ReDi: Rectified Discrete Flow
Authors: Jaehoon Yoo
Wonjung Kim
Seunghoon Hong
Venue: NeurIPS 2025 poster ()
Keywords: discrete generative models
discrete flow matching
discrete diffusion
few-step distillation
Primary area: deep_learning
TL;DR: Rectified Flow for Discrete Flow-based Models

Abstract: Discrete Flow-based Models (DFMs) are powerful generative models for high-quality discrete data but typically suffer from slow sampling speeds due to their reliance on iterative decoding processes. 
This reliance on a multi-step process originates from the factorization approximation of DFMs, which is necessary for handling high-dimensional data.
In this paper, we analyze the factorization approximation error using Conditional Total Correlation (TC), and reveal its dependence on the coupling.
To address the challenge of efficient few-step generation, we propose Rectified Discrete Flow (ReDi), a novel iterative method that reduces the underlying factorization error (measured as Conditional TC) by rectifying the coupling between source and target distributions.
We theoretically prove that each ReDi step guarantees a monotonic decreasing Conditional TC, ensuring its convergence.
Empirically, ReDi significantly reduces Conditional TC and enables few-step generation.
Moreover, we demonstrate that the rectified couplings are well-suited for training efficient one-step models on image generation.
ReDi offers a simple and theoretically grounded approach for tackling the few-step challenge, providing a new perspective on efficient discrete data synthesis.
Code is available at https://github.com/Ugness/ReDi_discrete.

ReDi: Rectified Discrete Flow
Jaehoon Yoo KAIST wogns98@kaist.com kr
Wonjung Kim
Seunghoon Hong KAIST
KAIST
wjbj16@kaist.ac.33 ac.
seunghoon. seugghon.bog@kkasst.ee www.ghoisst.tc.................. kr
Abstract
Discrete Flow-based Models (DFMs) are powerful generative models for high- quality discrete data but typically suffer from slow sampling speeds due to their reliance on iterative decoding processes. This reliance originates from the factorization approximation of DFMs, which is necessary for handling high-dimensional data. In this paper, we analyze the factorization approx- imation error using Conditional Total Correlation (TC), and reveal its dependence the coupling. To address the challenge of efficient few-step generation, we propose Rectified Discrete Flow (ReDi), a novel iterative method that reduces the underlying factorization error (measured as Conditional TC) by rectifying the coupling between source and target distributions. We theoretically prove that each ReDi step guarantees monotonic decreasing Conditional TC. ensuring its convergence. Empirically, ReDi significantly reduces Conditional TCC and enables few-step generation. Moreover, we demonstrate that the rectified couplings are well-suited for training efficient one-step models on image generation. ReDi offers simple and theoretically grounded approach for tackling the few-step challenge, providing new perspective efficient discrete data synthesis. Code is available at hhttps://githut http://gttuub.commmmeeeeeeee com/Ugness/Rel) /eeii discrete
multi-step process
1
Introduction
Discrete data synthesis is fundamental task across many domains, including texts, images, and biological sequences. Recent advances in deep generative models have shown remarkable success in synthesizing high-quality data [2 29 39]. Among these, Discrete Flow-based Models (DFMs) [[] 31 34 33), which typically achieve this by modeling probabilistic process that transforms simple initial state (e.g., a masked or random state) into complex data, have emerged as particularly effective for this purpose, demonstrating strong performance in generating high-quality discrete data.
6 13 26
30
However, despite their quality, the reliance of DFMs slow, iterative multi-step sampling pro- prohibits efficient few-step generation. This reliance originates from factorization approximations applied model high-dimensional discrete data 17). While data often exhibits high inter-dimensional correlation, the factorization approximation assumes independence of dimensions given the previous state. This assumption becomes increasingly inaccurate for the large steps required in few-step generation, undermining DFMs effectiveness in few-step scenarios.
cess 5 6
31 often involve techniques To alleviate the slow sampling of DFMs, existing approaches [0
like knowledge distillation where a multi-step teacher model trains a few-step student model. These methods commonly require maintaining both teacher and student models simultancously during training and may introduce new, specialized training objectives distinct from standard DFM training, adding complexity.
In this paper, we address the few-step generation challenge in DFMs by analyzing the underlying factorization error. We characterize this enror using Conditional Total Correlation (TC) (38 and
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

reveal its dependence on the coupling. Inspired by Rectified Flows [24 25 42 in continuous domain, we propose Rectified Discrete Flow (ReDi) enable efficient stw- generation by rectifying the coupling of discrete data, which in turn reduces the Conditional TC. By focusing on coupling rectification, our method provides a simpler alternative to prior works To 17 [T as it requires neither a specialized training strategy nor the handling of separate teacher-student models, which in turn reduces memory requirements. This simplicity enables ReDi to be broadly applicable to various DFMs including other distillation frameworks.
We demonstrated our method's effectiveness both theoretically and empirically. We theoretically prove that each ReDi iteration guarantees monotonically decreasing Conditional TC and empirically show that each rectification significantly reduces it. We evaluated our method on class conditional image generation and text generation. On image generation, ReDi shows comparable few-step generation performance against existing distillation methods, and significantly outperforms in one- step generation, due to direct rectification of the couplings contributing to the factorization error. On text generation, we observe that iteratively applying rectification improves the sampling efficiency and that ReDi can also be applied with existing distillation methods.
2 Related Works
2.1 Discrete Flow-based Models
Discrete flow-based models (DFMs) are used to generate discrete data such as images 2 8 16
39
videos 44 text [ [1 34 [5 : and protein (6. DFMs generate data by learning the flow from
initial states (often set as masked states or uniform random states). The generative flow is learned by
two primary formalisms, reversing a corruption process (e.g., masked generative models [2] 7 g 39], discrete diffusion [ 26 30 34 35), or constructing bridges between initial distribution and data Although they
distribution (e.g., Discrete Flow Matching
6 3 Schrödinger Bridges 21
221)
show powerful performance on discrete data synthesis, and efficient sampling cost compared to
autoregressive models as they support generating multiple states simultaneously [10
30 31
they
still require slow multi-step decoding process for successive generation [7]
2.2 Distillation of Discrete Flow-based Models
To address the slow sampling speeds of multi-step DFMs, prior works 110 17 31 have explored methods for distillation and faster generation. These approaches typically aim to distill a slower, multi-step teacher model into faster, few-step student model. They primarily focus modifying the training objective or designing specific training procedures tailored for distillation, and are sometimes specific particular DFM framework. For instance, SDTT [10] is tailored for masked diffusion models, and the dual consistency distillation method suggested in DUO (31 is tailored for uniform diffusion. While Di4C (17 suggested an objective function that is applicable for various discrete diffusion models, it utilizes four loss terms, requiring tuning of weights.
Alternative approaches 23 41l introduce auxiliary models reduce decoding steps. Discrete Copula Diffusion [23], for instance, requires a pretrained autoregressive model as an additional copula model. EDLM 41 takes another approach, using an energy-based model to guide sampling; however, its practical sampling efficiency remains limited as its algorithm requires sampling multiple candidates and selecting the most probable one. In contrast to the prior works, our method improves few-step generation in DFMs by focusing on the coupling itself, rather than solely on modifying the training process or model architecture.
2.3 Rectified Flows on Continuous Data
represent a significant development in achieving efficient few-step generation for bawe models in the continuous domain. These methods address the limitations of standard ODE solvers for faster sampling by characterizing the error as ntnnstraighttess of the path and proposing techniques to rectify or straighten the flow defined by the coupling between distributions. While Rectified Flow 24 is popular and well-used techniques in the continuous domain, the rectification of discrete flows has not been explored prior to this work. This is primarily because the concept of straightness is difficult to define meaningfully in a discrete space. Furthermore, the core challenge for few-step generation in DFMs stems not from
Rectified Flow Z and related techniques 20
25 42

---

continuous- time path straightness, but from the factorization approximation inherent in modeling high-dimensional discrete data. In this paper, we bridge this gap by characterizing this factorization emor using Conditional Total Correlation.
3 Method
This section describes our proposed method, Rectified Discrete Flow (ReDi). We first provide preliminary background on Discrete Flow-based Models (DFMs) in Sec. 3.1 Then we detail the problem of factorization error, which is characterized by Conditional Total Correlation (TC), depends on the coupling or dataset in Sec. 3.2 Finally, we present our novel method, ReDi, which rectifies the coupling reduce factorization error in Sec. 3.3 We also provide the theoretical analysis that single step of rectification process monotonically reduces the Conditional TC.
3.1
Preliminary: Discrete Flow-based Models
We denote discrete data sequence of discrete random variables X = (X), .....,,NNN)) where each dimension X takes values from a set of size D. The primary objective of DFMs 5 [3
26
is learn the probabilistic mapping from the source distribution p(Xo) to the target distribution (X1) by modeling the conditional distribution (X)) For data generation, the source distribution p(Xo) is typically chosen a tractable distribution (e.g., uniform).
30
34
To achieve this transformation, DFMs define a probability path over time E [0, 1] as a marginal distribution (x,) = Exxo..x,>>n ( Xi)]. This path is defined using coupling and conditional probability distribution pe (2t [ .1) that specifies the path bridging specific states zo and #1- The coupling ı joint distribution over (Xo, X1) from which training pairs are drawn, which is commonly chosen independent coupling i.e., πT ( x1) p(ro) (21)). A common choice for (r z1) is a convex sum bridging the two endpoints. For instance,
Pr(2z # = (1 11-a))6,, 02 (zt) + ato,, (zt),
(I)
where ag is a time-dependent coefficient with ax = 0 and aj = and å a Kronecker delta function, i.e., 8eg (Ira) = 1 if zo Iz and 0 otherwise.
DFMs are trained to model the conditional transition probability (ajo(xs (xx)) for s These transitions are mathematically derived from the defined probability path and describe the probabilistic dynamics of moving from state X, X, along the path as:
Poje (oje (x))))))
Σ Po I x (t) x1 x1(2)),
Za.21
where ps ( 3i; 21) is the conditional probability of X, x, given the path passes through æ; at time and has endpoints Zo, It. Also, p(zg, x1 (xx)) is the posterior distribution over endpoints given X, z4 and is computed using Bayes' rule:
(r an (2x.) Eng,at Pe (e ( (1)) (1)
As both terms on the right side of Eq. 2 are ultimately derived from the probability path definition and the coupling म. the conditional transition probability (=)e(==(2,) directly depends on æ. By learning to model these fundamental step-wise transitions, DFMs acquire the ability to simulate the entire path and thereby transform samples from the source distribution to the target distribution.
However, modeling the full conditional distribution Pajt (x,//2)) over the entire D state space is intractable for high-dimensional data. Therefore, to make modeling feasible, DFMs assume that the true transition (aje(x, (x;) in Eq, can be approximated by factorization across dimensions:
N
(2)(2)))))) ~ II (1
where (xf (xx)) the marginal distribution of Pate (xr)) along dimension By relying on this factorization, DFMs reduce the complexity of the output space representation. While this factorization necessary to deal with high-dimensional data, the resulting approximation error may induce few-step decoding challenge, which discuss in detail in the following.
3

---

Tof To(Xo. X1)
T X1)
10
10
11
00
1/4
00
1/4
8
01
1/4
01
1/4
8
(OX
(0X)
1
10
4
1/2
0
0
1/2
1/2 0
0 1/2
p(X1)
p(X1)
Figure synthetic example that illustrates two different couplings mo and T1. p(Xo) is defined as uniform distribution over (00, 01, 10, 11} and p(X1) is defined as uniform distribution over {00, 11}. While the two couplings To and ø1 share the same marginal distributions (p(Xg) and I), due to the difference between them, the Conditional Total Correlation of xo is higher than that øj. Detailed explanation about the example Sec. 3.2
3.2 Factorization Error of DFMs
Factorization error induced by the approximation in Eq. hinders few-step generation in DFMs. Specifically, since the factorized model treats dimensions independently, it fails to create the inter- dimensional correlation needed as the state changes from uncorrelated Xo correlated X1. This error grows with the time step = I, as larger steps involve more significant changes in the distribution being modeled, making few-step generation difficult.
We characterize the factorization error using Conditional TC, which is defined as the expected KL divergence between the conditional distribution and the product of its marginals:
N
TX,( = Ez, DEL
( (X, = x,)1 II & = =.:)]
(5)
This metric is particularly suitable for analyzing the factorization error because it directly quantifies the inter-dimensional dependencies that the factorized approximation neglects. also shows that several prior distillation objectives [10 17 31 implicitly minimize Eq. 5 by reducing the KL divergence between the multi-step teacher transition (serving as the approximated joint distribution) and the few-step student transition. Importantly,
note that the above TC is dependent coupling n, since the true transition (2) (x,) depends on т as shown in Eq. 2 and Eq. This dependency reveals that rectifying the coupling reduce the Conditional TC, which characterizes the factorization error.
To provide further intuition, present simple example in Fig. Dig.Dexxpin explaining the dependency between coupling and the factorization error. For simplicity, we consider a task that models the distribution of 2-bit sequences, where p( p(Xo) is defined uniform distribution over 4 states {00, 01, 10, 11] and p(X,) ) defined as a uniform distribution over {00, 11). We then compare the two couplings To and TT. which leads to the same marginal distributions pi Xx) and p(X,) ) but different Conditional TC. For instance, for the coupling To. p(X; = 00) X, = 00) = 0.5 diverges with its factorized distribution 0/00) 0.25, while the factorization error is zero for the coupling by p(X, = 00)00) = x = 0 (0) = 0/00) = 1. This simple example demonstrates that the factorization emror indeed depends the coupling and can be reduced by updating the coupling.
p(X] (XX?
3.3 Rectified Discrete Flow
To enable efficient few-step generation, we propose Rectified Discrete Flow (ReDi), method designed to directly tackle the underlying factorization error. Based our finding that this error closely depends on the coupling, ReDi provides a mechanism to rectify the coupling, thereby removing this primary bottleneck to fast and efficient synthesis. Given coupling g. (initially mo).

---

the rectification process involves training a DFM using x and subsequently generating sample pairs (1) by sampling from the source distribution and transforming X, through the trained DFM. These generated pairs collectively define new coupling, Th+1 with monotonically decreased factorization error.
Formally, at iteration k. the rectification process takes a coupling Tu (Xo, X1) and produces a new coupling TK++1 (Xo, X1). This is achieved by first training a DFM, denoted by its conditional distribution pe (Xo) to model the transition probability defined by the current coupling. Once the DFM is trained, the rectified coupling Th+1 is formally defined as the joint distribution resulting from sampling Xo p(Xo) and X1 ~ pe (x).
(X41 X1) - p(Xo) (X)
(6)
Then we can show that the rectification process of Eq. 6 is monotonically decreasing the Conditional TC. For brevity, we present an informal statement of this guarantee below. The formal version with assumptions is provided in Appx. A allong with its proof.
Theorem 1 (Informal). Let Th (Xo, x1) be coupling at iteration k. and let Th+ ( X1) be the "rectified" coupling obtained via the ReDi procedure at iteration k. Then, under certain assumptions, it satisfies the following:
TC (1 (X1(X)) s TC
In addition to the theoretical support, we also demonstrated the decrease in factorization error through the rectification process in Sec. 4.4
Given Thm. [ we can apply the rectification process iteratively to reduce the factorization error. By repeatedly applying the rectification process for K iterations, starting with the initial coupling to. we obtain a sequence couplings To, IT, मH with monotonically reduced factorization error. Having small factorization error, this rectified coupling * is then well-suited for training an efficient one-step generative model, addressing the few-step generation challenge highlighted carlier.
At the same time, the rectification process has to be performed with care: while cach step reduces the factorization error (Thm. [T. the marginal approximation error between the target marginal p(X,) and the rectified marginal (XX; can accumulate in practice because each T is estimated from model- gedee-eeee pairs rather than real data. This accumulation becomes more pronounced in high-cardinality data (e.g., text). where modeling the target distribution is inherently more challenging. To address this issue, we introduce a perturbed rectification strategy. Instead of using samples from the source distribution p( (Xa)). the perturbed rectification first sample clean target X1 ~ p(Xi) and then perturb it to X, ~ p( (1) at a random time l. The rectification is then performed by generating X1 pe (Xg Xx). By replacing the source distribution samples with perturbed ones for rectification, we reduce the empirical discrepancy between the true target distribution p p(X,) and the rectified marginal nu (X1). ). Detailed algorithm can be found in Appx.
and Iterative Marko- which progressively straightens the transport path between distributions in continuous space, ReDi iteratively refines the coupling to minimize the factorization error, a metric tailored for the discrete domain. Furthermore, the iterative training and data generation steps ReDi are analogous to the Markovian and reciprocal projections of IMF. These parallels suggest that ReDi can be viewed as an application of the broader principle of iterative coupling refinement, adapted for the discrete domain.
ReDi's iterative process closely resembles rectification methods
20 24
vian Fitting (IMF) procedures [21 22] 36) Similar to Rectified Flow
24
ReDi presents several key advantages that distinguish it from existing distillation methods [10 17 31 on discrete data. A primary benefit its notable simplicity and ease of implementation. ReDi does not introduce specialized objective functions for improving few-step inference, allowing broad applicability to diverse DFMs. Also, unlike some distillation methods that require both teacher and student networks during training, ReDi only handles single DFM, reducing the training memory requirements. Since the rectification process of ReDi orthogonal to distillation methods, can be also applied with the existing approaches to further boost the few-step performance. Finally, we can employ the rectified couplings directly to train one-step generative models. These advantages make ReDi practical and powerful approach for tackling the few-step generation challenge.
5

---

Experiments
4.1 Experimental Setup
Datasets We conduct experiments on two benchmark datasets: ImageNet [9] for class-conditional image generation and WebTet dataset [14] for text generation. For the image generation lask, following prior works 88 17 we represent 256x256 images as 16x 16 vector-quantized tokens using a pretrained VQGAN model generation, following prior works TO 30 31]. we use sequences of 1024 tokens, tokenized with GPT-2 tokenizer
Each token can take one of 1024 possible values. For text
4 which has vocabulary size of 50257.
Baselines We compared our method with SDTT [0] and Di4C 17 prior works for efficient few-step generation in DFMs. Both SDTT and Di4C distill the teacher model's multi-step behavior into a few-step student model. Di4C suggests its variant, di4c-d, which additionally utilizes data prediction loss to enhance the performance. For all in our experiments, we used di4c-d as Di4C baseline. As SDTT is tailored for masked diffusion models, we compare SDTT only if the teacher model masked diffusion model.
Teacher Models To compare with distillation methods pretrained models teacher models for the baselines. These models served the initial models for the first rectification process of ReDi. For ImageNet experiments, we adopt pre-trained MaskGIT as a teacher model based on masked discrete diffusion framework 45]. For OpenWebTest experiments, we adopt DUO+DCD [1 as teacher model that utilizes uniform random state as initial state.
we utilized publicly available
Implementation For ReDi, we use the same training objective the teacher model g and finetune the model from the previous iteration. For the one-step distillation model, training uses the same objective while sampling Xo instead of X, during training, so that the model specifically targeting for direct transition from f = 0 to t 1. For image generation, we rectified the coupling with 50k pairs with 16-step heuristic sampling method of MaskGIT 3 For text generation, we utilized 20k pairs with 1024- and applied perturbed rectification strategy. For each experiment, we denote R if the model is trained on πg. For instance, teacher model in each experiment can be denoted as ReDi". In similar, we also denote our one-step distillation model as ReDit-distill. The experiments in Sec. are conducted on the ImageNet dataset unless specified. Training and implementation details are provided in Appx.
Metrics For image generation. we generate 50k images and measure Fréchet Inception Distance (FID) [18), Inception Score (IS) 33 following the procedure of prior works B 28 We also measure precision, recall, density, and coverage 27 32 to further assess the fidelity and diversity of generated images. As DFMs can utilize classifier-fre free guidance (CFG) [9 to control the generation quality, we report the score with best FID among the CFG values in [1, 2, ... 8). We measure the generative perplexity I of 1024 texts using the LLaMa 3-B model [15] To complement the metric, we also assess I I-gram entropy to detect failure modes involving simple, repetitive generation [37]
4.2 Results on Image Generation
Few-step Generation As shown in Tab and Fig. ReDi improves 4-step generation performance over its teacher model and achieves comparable results against distillation baselines. Specifically, both ReDi and ReDi2 outperform MaskGIT's s four-step generation, yielding FID scores of 7.58 and 7.86 (vs. 10.90) and Inception Scores of 228 and 240 (vs. 184), respectively. Against other baselines Di4C and SDTT (4-step), ReDi exhibits comparable performance. For instance, while Di4C records a lower FID (6.20), ReDi achieves a higher IS (228 vs. Di4C's 216). Additionally, ReDi maintains comparable scores across Precision, Density, and Coverage metrics.
Fig. 2 further illustrates ReDi's generation quality, which appears on a similar level when compared to distillation baselines [7] while improvement against MaskGIT by reducing the factorization error is clearly shown. To be specific, due the factorization error, MaskGIT's 4-step generation often leads to artifacts in structures in generated samples. For instance, the goldfish image generated
'To enable the update of coupling, for ReDi experiments, we finetuned MaskGIT while changing the initial states with stochasticity, Details can be found in Appx. B.1

---

Table 1: Performance of discrete flow-based models on ImageNet at different generation steps. ReDi denotes the model trained on k-th rectified coupling The and ReDi distiil denotes the model specifically targetting for one-step generation. We reproduced SDTT on ImageNet with round of
distillation and denoted as SDTT
Step
Model
FID (4)
IS (1)
Prec. (1)
Rec. (f)
Den. (1)
Cov. (?)
MaskGIT 3
95.16
12
0.26
0.12
0.17
0.35
SDTTI 10
90.40
14
0.31
0.13
0.21
0.34
Di4C 17
90.32
13
0.26
0.24
0.17
0.33
ReDi
37.43
49
0.63
0.51
0.78
0.86
ReDi
21.80
90
0.74
0.52
1.05
0.93
ReDi
11.68
182
0.83
0.44
1.25
0.96
MaskGIT 3
10.90
184
0.83
0.46
1.18
0.96
SDTTI T0
8.97
205
0.88
0.41
1.43
0.97
4
Di4C 17
6.20
216
0.87
0.52
1.33
0.98
ReDi
7.58
228
0.87
0.46
1.33
0.98
ReDj2
7.86
240
0.87
0.44
1.31
0.97
8
MaskGIT
3
6.51
227
0.89
0.48
1.38
0.98
1-step Generation Results
4-step Generation Results
MaskGIT
SDIT
Di4C
ReDi
ReDi
ReDi-disttll
Figure 2:
Generated images from various discrete flow-based models. ReDi successfully generate images with natural structures even under one-step generation settings.
by MaskGIT omits the tail of the fish and has heads both sides. We conjecture that simultancous sampling of tokens at both sides fails because their joint distribution is poorly factorized when conditioned on the tokens from the previous step. Compared to MaskGIT, the images generated by ReDi shows improved strucutural coherence. This improvement structural coherence is attributed to the reduced factorization error achieved by ReDi, allowing the model to better capture inter- dimensional dependencies during generation. Additional qualitative results are shown Appx. D.5]
One-step Generation As shown in Tab. [ the one-step distillation model trained on the rectified coupling ( -distill) achieves remarkable performance against 1-step generation other methods. With an FID of 11.67 and IS of 181, it significantly outperforms their one-step results (e.g., SDTT 1-step FID 90.40, 14; Di4C 1-step FID 90.32, IS 13). Interestingly, while the performance ReDil and ReDi2 is inferior to ReDi - distill, they show significant improvement on one-step generation

---

DUO+DCD
ReDi
ReDi
ReDi
D14C
Di4C+ReDi-
100
5.6
Dataset
80
Gen. PPL
Entropy
5.3
60
5.0
40
20
4.7
4
16
128
1024
16
128
1024
Sampling Steps
Sampling Steps
Figure 3: Comparison of various discrete flow-based models on OpenWebText. The blue horizontal line denotes 1024-step generation performance of DUO+DCD. Lower generative perplexity (Gen. PPL) indicates more natural texts. Following Wang et al. [37], we additionally assess the entropy of generated samples monitor the pitfall of generative perplexity. We provide the exact values for each metric in Appx. D.I
by iteratively applying the proposed rectification process. Compared to our method, SDTT and Di4C fails improve the performance of one-step generation reasonably. This discrepancy between ReDi against SDTT and Di4C demonstrates that ReDi successfully enables efficient generation in single step, supports that the rectified coupling is well-suited for one-step generation by reducing the Conditional TC. Furthermore, the performance of our one-step model reaches close to that of the multi-step teacher model (e.g., MaskGIT 4-step FID 10.90, IS 184), highlighting that the better coupling achieved through ReDi's iterative rectification process is highly effective in bridging the gap towards efficient, high-quality one-step generation.
Fig. also visually supports the discrepancy between ReDi and other methods. By iteratively rectifying the coupling, the fidelity of generated images shows significant improvement, while MaskGIT, SDTT, and Di4C fails to generate images in one-step manner. Additional qualitative results are presented Appx. [D.5]
4.3 Results Text Generation
Compared with the image generation, text synthesis is much more challenging due to its significantly larger state space (1024256 vs. 502571024), For text generation, we compare the DUO+DCD which is used as a teacher model, its ReDi-rectified variants (ReDi*), the Di4C-distilled model (Di4C), and a hybrid Di To train the hybrid model, we generate the rectified coupling with Di4C and finetune the Di4C model with the training procedure of DUO 3I
The results are shown in Fig. The blue horizontal line denotes the performance of 1024-step generation with the teacher model. Iterative rectification with ReDj (k=1,2,3) consistently yields lower perplexity across few-step settings and boosts the required decoding steps for such quality. Specifically, compared to the teacher model (DUO+DCD)ss 1024-step performance, ReDi", ReDi² and R show equivalent performance at 16 and 8 steps, achieving significant speedups of x64 and x x128. This is consistent with our hypothesis that iterative rectification additionally reduces the factorization error, identified main challenge for few-step generation.
To demonstrate ReDi's applicability, we additionally apply it to the distilled model with Di4C, resulting in the combination of ReDi on Di4C (Di4C+ReDi)), Compared Di4C, applying ReDi (Di4C+ReDi enhances few-step performance. These results suggest that ReDi is effective in enhancing few-step generation performance and is applicable across different base models.
To avoid misleadingly low perplexity scores caused by repetitive tokens 37 we also measure the entropy of the generated samples. As shown in Fig, (right), the ReDi variants achieve entropy values

---

0.0
40
0.6
0.9
Precision
Recal
030
30
Precision
Recall
g 100
0.85
600
10
500
200k
0.8
0.3
Rectification Ineration
Number
(a) TC* over rectification iteration
(b) FID number of pairs
Precision and Recall over CPG
Figure 4: Ablation studies of ReDi on ImageNet. We conducted ablation studies about iterative
rectification, number of pairs to represent the coupling, and the effect of decoding strategy.
Table 2: Ablation study the perturbed rectification strategy. The perturbed rectification strategy reduces the discrepancy between original dataset and rectified coupling.
Data PPI.
Data Entropy
1024-step PPI. 1024-step Entropy
Orig. Dataset
9.75
5.44
36.21
5.23
w/ Perturbed
26.02
5.46
24.13
5.28
w/o Perturbed 36.21
5.23
44.17
5.11
comparable to the teacher model, indicating ReDi avoids the failure mode. Qualitative inspection of the samples Appx. D.5 further suggests that ReDi generates plausible text.
4.4 Analysis and Ablation Studies
Empirical Analysis Conditional TC To support the theoretical findings in Sec. 3 pirically demonstrate that the proposed rectification process iteratively decreases Conditional TC. To approximate the Conditional TC, we generated 10 samples from the same initial state 5k times (which resulted in 50k samples) and then approximated the Conditional TC based on their frequencies. The progressive decrease of Conditional TC over rectification iterations is shown in Fig. 4a While Thm. Tom.[hha has shown that the Conditional TC is monotonically decreasing, we empirically find that it usually strictly decreases iterations progress. While the Conditional TC decreases over rectification iterations, we empirically observe that DFM performance degrades in practice due to error propagation arising from sampling pairs from the trained DFM. A similar observation can also be found in the rectification of continuous flows 46
Ablation on the Size of Dataset the proposed method, the rectified coupling Ty is determined by pairs sampled from the distribution defined by pretrained 0, shown in Eq, To determine sufficient number of pairs for reliably estimating the rectified coupling æ1, we conducted an ablation and subsequently trained a DFM using Th-
study the number of sampled pairs used to compute The result is shown Fig. [4b] We reported the 4-step generation performance of the trained model the number of pairs used to define TT Interestingly, the model's performance starts to
with FID saturate with 20k samples and is fully saturated with more than 50k samples. This demonstrates that ıi can be effectively defined with remarkably smaller number of pairs compared to the original dataset, which consists of 1.3M images.
Ablation on CFG Level for Rectification Pair Sampling The proposed rectification process relies on (1) pairs sampled from a pretrained DFM 0 to define the coupling To investigate how the DFM's decoding strategy impacts the generation performance of a new DFM subsequently trained, conduct ablation study by controlling the Classifier-Fres Guidance (CFG) level [19] during sampling pairs. As shown in Fig. 4c we empirically observe that controlling CFG affects the fidelity- diversity tradeoff of trained DFM. This observation is similar the observation in rectification of continuous flows despite the inherent differences between discrete and continuous flow mechanisms.
Ablation on Perturbed Rectification in Text Generation As discussed in Sec. 3.3 we empirically find that perturbed rectification strategy required when treating bigh-dimensional data. The results in Tab. 2 demonstrates that the perturbed rectification strategy narrows the discrepancy between

---

original dataset and the updated coupling. Without the strategy, the trained model's s generative perplexity (44.17) degrades compare to the teacher model's perplexity (36.21) due the poor quality of training dataset. In contrast, the perturbed strategy enhances the training dataset, leading to the trained model achieving a superior perplexity of 24. 24.13, surpasses the teacher model.
Limitations and Future Work
While ReDi demonstrates promising results in enabling efficient few-step and one-step generation for discrete flow-based models by rectifying the coupling and reducing factorization error, this work has several limitations that open avenues for future research.
First, the connection between the rectification process in discrete flows (ReDi) and its counterpart in continuous flows (e.g., Rectified Flow (24] ) is not yet fully elucidated. While we characterize the error in DFMs using Conditional TC and show its monotonic decrease, a deeper theoretical understanding of the parallels and differences in how rectification improves paths or reduces errors in these two distinct domains could provide further insights and potentially lead to unified frameworks.
Second, our current work primarily focuses on non-autorcgressive or parallel decoding DFMs. However, suggested by recent interpretations (c.g.. D3PM ([), autoregressive (AR) models can also be viewed within the broader DFM framework. Future work could explore extending the ReDi framework to AR models, potentially offering new ways to accelerate their notoriously slow sequential sampling process by rectifying their implicit state transition couplings could be an interesting direction for future research.
Conclusion
In this paper, introduced Rectified Discrete Flow (ReDi), novel iterative method designed to address the challenge of slow, multi-step sampling in Discrete Flow-based Models (DFMs). We identified the factorization approximation, necessary for handling nigh-dimensional discrete data, as a primary source of error that hinders efficient few-step generation. We rigorously characterized this factorization error using Conditional Total Correlation (TC), highlighting its dependence on the coupling between source and target distributions.
ReDi tackles this issue by iteratively rectifying the coupling. Each ReDi iteration involves training a DFM on the current coupling and then using it to generate new, rectified set of paired samples, which defines the coupling for the next iteration. We theoretically proved that each rectification step guarantees a monotonically decreasing Conditional TC, ensuring convergence towards a coupling with lower intrinsic factorization error.
Empirically, through experiments on image and text generation benchmarks, we demonstrated that ReDi reduces Conditional TC and improves few-step generation performance. Notably, the rectified couplings obtained via ReDi are particularly well-suited for training highly efficient one-step generative models, achieving remarkable performance that often surpasses or rivals more complex distillation techniques. ReDi offers a simple, theoretically grounded, and broadly applicable approach that avoids specialized training objectives or teacher-student architectures typically required by existing DFM distillation methods.
By providing new perspective on improving discrete data synthesis through direct coupling manipu- lation, ReDi advances the development of faster and more efficient generative models applicable to a wide range of discrete data modalities.
Acknowledgments
This work was in part supported by the National Research Foundation of Korea DSS-2024-00351222 and DS-2024-00436166 the Institute of Information & communications Tech- 2--0224005092799
nology Planning & Evaluation (ITTP) 2--2222222222222 RS- RS-2022-I1220959, and RS (89-0019-11100075) and the "HPC support" project funded by the Korea gov- ernment (MSIT), and the Korea Meteorological Administration Research and Development Program "Developing Service Platform Technology for Al and Data Convergence" RMMA2021-00122)
10

---

References
[1] Jacob Austin, Daniel D. Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg. 2021. Structured Denoising Diffusion Models in Discrete State-Spaces. In NeurlPS.
[2] Jinbin Bai, Tian Ye, Wei Chow, Enxin Song, Qing-Guo Chen, Xiangtai Li, Zhen Dong, Lei Zhu, and Shuicheng Yan. 2025. Meissonic: Revitalizing masked generative transformers for efficient high-resolution text-to-image synthesis. In ICLR.
[3] Victor Besnier and Mickael Chen. 2023. A pytorch reproduction of masked generative image transformer. arXiv preprint arXiv:2310.14400
[4] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D) Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et 2020. Language models are few-shot learners. In NeurlPS.
[5] Andrew Camphell, Joe Benton, Valentin De Bortoli, Thomas Rainforth, George Deligiannidis, and Arnaud Doucet. 2022. A Continuous Time Framework for Discrete Denoising Models. In NeurIPS.
[6] Andrew Campbell, Jason Yim, Regina Barzilay, Tom Rainforth, and Tommi S. Jaakkola. 2024. Generative Flows on Discrete SpaceSS Enabling Multimodal Flows with Applications to Protein Co-Design. In ICML.
[7] Huiwen Chang, Han Zhang, Jarred Barber, AJ Maschinot, Jose Lezama, Lu Jiang, Ming- Hsuan Yang, Kevin Murphy, William T Freeman, Michael Rubinstein, et al. 2023. Muse: Tex--to-image generation via masked generative transformers. arXiv preprint arXiv:2301.00700
[8] Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T. Freeman. 2022. MaskGIT: Masked Generative Image Transformer. In CVPR.
[9] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. ImageNet: A large-scale hierarchical image database. In CVPR.
[10] Justin Deschenaux and Caglar Gulcehre. 2025. Beyond Autoregression: Fast LLMs via Self- Distillation Through Time. In ICLR.
[11] Sander Dieleman, Laurent Sartran, Arman Roshannai, Nikolay Savinov, Yaroslav Ganin, Pierre H Richemond, Arnaud Doucet, Robin Strudel, Chris Dyer, Conor Durkan, al. 2022. Continuous diffusion for categorical data. arXiv preprint arXiv:2211.15089.
[12] Patrick Esser, Robin Rombach, and Bjorn Ommer. 2021. Taming transformers for high- resolution image synthesis. In CVPR.
[13] Itai Gat, Tal Remez, Neta Shaul, Felix Kreuk, Ricky T. Q. Chen, Gabriel Synnaeve, Yossi Adi, and Yaron Lipman. 2024. Discrete Flow Matching. In NeurIPS.
[14] Aaron Gokaslan and Vanya Cohen. 2019. OpenWebText Corpus. httpp//Skyllioo007
sttaub..o//pennWeTTetttoom
[15) Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, al. 2024. The llama 3 herd of models. arXiv preprint 2rXiv:240..27788
[16] Shuyang Gu, Dong Chen, Jianmin Bao, Fang Wen, Bo Zhang, Dongdong Chen, Lu Yuan, and Baining Guo. 2022. Vector quantized diffusion model for text-to-image synthesis. In CVPR.
[17] Satoshi Hayakawa. Yuhta Takida, Masaaki Imaizumi. Hiromi Wakaki, and Yuki Mitsufuji. 2024. Distillation of Discrete Diffusion through Dimensional Correlations. CoRR, abs/2410.0874
[18] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. 2017. Gans trained by two time-scale update rule converge to local nash equilibrium. In NeurIPS.

---

[19] Jonathan Ho and Tim Salimans. 2021. Classifier-Free Diffusion Guidance. In NeurlPS 2021 Workshop on Deep Generative Models and Downstream Applications.
[20] Beomsu Kim, Yu-Guan Hsieh, Michal Klein, Jong Chul Ye, Bahjat Kawar, James Thornton, et al. 2025. Simple ReFlow: Improved Techniques for Fast Flow Models. In ICLR.
[21] Jun Hyeong Kim, Seonghwan Kim, Seokhyun Moon, Hyeongwoo Kim, Jeheon Woo, and Woo Youn Kim. 2025. Discrete Diffusion Schrödinger Bridge Matching for Graph Transforma- tion. In ICLR.
[22] Grigoriy Ksenofontov and Alexander Korotin. 2025. Categorical Schrödinger Bridge Matching. In ICML.
[23] Anji Liu, Oliver Broadrick, Mathias Niepert, and Guy Van den Broeck. 2025, Discrete Copula Diffusion. In ICLR.
[24] Xingehao Liu, Chengyue Gong, al. 2023. Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow. In ICLR.
[25] Xingchao Liu, Xiwen Zhang, Jianzhu Ma, Jian Peng, et al. 2024. Instaflow: One step is enough for high-quality diffusion-based text-to-image generation. In ICLR.
[26] Aaron Lou, Chenlin Meng, and Stefano Ermon. 2024. Discrete Diffusion Modeling by Estimat- ing the Ratios of the Data Distribution. In ICML.
[27] Muhammad Ferjad Naeem, Seong Joon Oh, Youngjung Uh, Yunjey Choi, and Jaejun Yoo. 2020. Reliable fidelity and diversity metries for generative models. In JCML.
[28] Alexander Quinn Nichol and Prafulla Dhariwal. 2021. Improved denoising diffusion probabilis- tic models. In ICML.
[29] Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, JUN ZHOU, Yankai Lin, Ji-n Wen, and Chongxuan Li. 2025. Large Language Diffusion Models. In ICLR 2025 Workshop on Deep Generative Model in Machine Learning: Theory, Principle and Efficacy.
[30] Subham S. Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin T. Chiu, Alexander Rush, and Volodymyr Kuleshov. 2024. Simple and Effective Masked Diffusion Language Models. In NeurIPS.
[31] Subham Sekhar Sahoo, Justin Deschenaux, Aaron Gokaslan, Guanghan Wang, Justin T Chiu, and Volodymyr Kuleshov. 2025. The Diffusion Duality/ In ICLR 2025 Workshop on Deep Generative Model in Machine Learning: Theory, Principle and Efficacy.
[32] Mehdi SM Sajjadi, Olivier Bachem, Mario Lucic, Olivier Bousquet, and Sylvain Gelly. 2018. Assessing generative models via precision and recall. NeurIPS.
[33] Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. 2016. Improved techniques for training gans. Advances in neural information processing systems, 29.
[34] Yair Schiff, Subham Sekhar Sahoo, Hao Phung, Guanghan Wang, Sam Boshar, Hugo Dalla- torre, Bernardo P. de Almeida, Alexander Rush, Thomas Pierrot, and Volodymyr Kuleshov. 2025. Simple Guidance Mechanisms for Discrete Diffusion Models. In ICLR.
[35] Jiaxin Shi, Kehang Han, Zhe Wang, Amnad Doucet, and Michalis Titsias. 2024. Simplified and generalized masked diffusion for discrete data. NeurIPS.
[36] Yuyang Shi, Valentin De Bortoli, Andrew Campbell, and Arnaud Doucet. 2023. Diffusion schrödinger bridge matching. In NeurIPS.
[37] Yequan Wang, Jiawen Deng, Aixin Sun, and Xuying Meng. 2022. Perplexity from plm is unreliable for evaluating text quality. arXiv preprint arxiv:2210.05892.
[38] Michael Satosi Watanabe. 1960. Information Theoretical Analysis of Multivariate Correlation.
IBM J. Res. Dev, 4(1):66-82. 82.
12

---

[39] Mark Weber, Lijun Yu, Qihang Yu, Xueqing Deng, Xiaohui Shen, Daniel Cremers, and Liang- Chich Chen. 2024. MaskBit: Embedding free Image Generation via Bit Tokens. TMLR.
[40] Geoffrey Wolfer and Shun Watanabe. 2024, Geometric aspects data-processing of Markov chains. Transactions of Mathematics and Its Applications, 80 Itna:n01
[41] Minkai Xu, Tomas Geffner, Karsten Kreis, Weili Nie, Yilun Xu, Jure Leskovec, Stefano Ermon, and Arash Vahdat. 2025. Energy- -arge Diffusion Language Models for Text Generation, In ICLR.
[42] Hanshu Yan. Xingchao Liu, Jiachun Pan, Jun Hao Liew, Jiashi Feng, et al. 2024. PeRFlow: Piecewise Rectified Flow as Universal Plug-and-Play Accelerator. In NeurIPS.
[43] Jachoon Yoo, Semin Kim, Doyup Lee, Chiheon Kim, and Seunghoon Hong. 2023, Towards end-to-end generative modeling long videos with memory- bidirectional transformers. In CVPR.
[44] Lijun Yu, Yong Cheng, Kihyuk Sohn, José Lezama, Han Zhang, Huiwen Chang, Alexander G Hauptmann, Ming-Hsuan Hsuan Yang, Yuan Hao, Irfan Essa, et al. 2023. Magvit: Masked generative video transformer. In CVPR.
[45] Kaiwen Zheng, Yongxin Chen, Hanzi Mao, Ming- Yu Liu, Jun Zhu, and Qinsheng Zhang. 2025. Masked diffusion models are secretly time-agnostic masked models and exploit inaccurate categorical sampling. In ICLR.
[46] Huminhao Zhu, Fangyikang Wang, Tianyu Ding, Qing Qu, and Zhihui Zhu. 2025. Analyzing and Mitigating Model Collapse in Rectified Flow Models
13

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No]. or [NA].
[NA] means either that the question Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will published with the paper.
The reviewers of your paper will be asked use the checklist as one of the factors their evaluation. While "[Yes] is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] providedd proper justification is given (e.g.. "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering binary way, we
"[No] or "[NA]" is not grounds for rejection. While the questions are phrased acknowledge that the true answer is often more nuanced, please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The claims are in the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitation in Section 5.
14

---

Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, c.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification:
The proof is in the Appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
The details for reproducibility of result are in Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
15

---

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. w
For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
The code will be submitted with supplementary materials.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (nttps://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ 7/nips. /ttppp.cc/pbllc/guideeee http://oeeuuiiisssoooooooooo
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
16

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting//etails
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
In appendix
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Due the heavy computation cost of evaluation protocol, we didn't report the statistical significance.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-simma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time exccution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: In appendix
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
17

---

The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer:
[Yes]
Justification:
We reviewed Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
No potential societal impacts
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Answer: [NA]
Justification: No such risks.
Guidelines:
The answer NA means that the paper poses such risks.
18

---

Released models that have high risk for misuse or dual use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: In appendix and supplementary materials.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: No assets introduced.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: no human subjects
19

---

Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: No human subjects
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods this research? Note that if the L.M used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: We utilize LLMs to improve the clarity of the sentences in this paper.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM for what should or should not be described.
20

---

Appendix
A Theoretical Analysis
A.1 Preliminaries
We first begin by introducing the definition, assumptions, and the property of KL. divergence for our formal proof.
Definition 1 (M-step Decoding Process). process is defined as:
With discrete flow-based model pe, M-step decoding
Po (o(Xi(Xo)
Σ
IIII
Pa.ua alte (Xi, (Xee)
X No. Nex--
Assumption 1. Let P be the family of M-step decoding processes. We assume that our model (o) lies within the log-convex hull of P.
This assumption is justified because for a sufficiently large number of steps M, the family of processes P can closely approximate the hypothesis space po (o(X (o).
Assumption We assume that at each rectification step, the model pa is the minimizer of the objective function, D (Pr,, (Xo) [lpe (xo)).
Property (Pythagorean Inequality for KI. Divergence 400).. For distribution q in a log-convex set of distributions Q. if q² arg mingeQ ( and e Q then
DkI(pllr) DRi(plla") DRi(d"ll).
A.2 Formal Theorem and Proof
Theorem 1. Let (Xo. X1) be a coupling at iteration k., and let (X,, X2) be the "rectified" coupling obtained via the ReDi procedure at iteration Then, under Assumptions I and 2, the following inequality holds:
TC (Xi(Xo) & TC,, (X))
Proof. The proof proceeds as follows:
TC = Ex, (PT. II Pr., 110,
2 Ex, D (P+. (o
+ +Ex,IDKI (X, (xx) II Pr.. 110 (X:
2 Ex, (Xx
Pm (oxii((X((Xo)))
=xx P
PT.. (X))
(8)
Ex.,DKL Dx.
Σ DKL(P (x (Xx](
2 Ex, DKL 10
X II Pa, (X
(X
The inequality Eq. follows from Property 1, which applicable under Assumptions and while the equality in Eq. 8 follows from the definition of rectification process in Eq. 6
21

---

25
TC
FID
800
ReDi"-FID FID
20
10.5
750
9.5
ReDi
15
700
8.5
10
650
0.1
0.3
0.5
0.7
0.9
1.0
Interpolation ratio
Interpolation ratio r
(a) 16-step FID over interpolation ratio
(b) TC,, and 4-step FID over absorbing ratio
Figure 5: Ablation studies about finetuning MaskGIT with stochastic initial states.
B
Implementation Details
This section outlines key implementation details, covering the procedure for finetuning the masked state diffusion model used in Sec. f.2 rectification, and the hyperparameters for both generating the rectified coupling and training ReDi models used in Sec.
B.1 Finetuning MaskGIT with Stochastic Initial States
While Thm. Tholds for any coupling иa between a source X, and a target X,, the equality condition met if the source is masked state (i.e., (X)) 6m (Xo)). This occurs because л (Xo, X1)
p( X1 | X (o)p(XX) = p(X,) for any p( X2 Xo) preserving the target distribution. Therefore, to handle
models with masked initial state 13 g finetune the models by modifying the initial distribution p(Xo) as:
(X.) = (1 - r))( + (Xm)((x))
(9)
where (Xo) is the uniform distribution over possible states, and r E [0, 1] is a interpolation ratio interpolating between the delta distribution 8m (Xo) and the uniform distribution u(Xo).
To analyze the effect of the interpolation ratio т. we finetune MaskGIT models with r e (0.1, 0.3, 0.5, 0.7, 0.9). which denote ReDi After training, evaluate the models by 16-step generation FID using a CFG level of We also measure the Conditional TC of the rectified coupling 1 and the 4-step FID of the corresponding models, ReDi The original MaskGIT model B] corresponds to = 1.0 in this setup. As shown in Fig. 5a decreasing r degrades the performance (FID) of ReDi We conjecture the degradation is due the distributional mismatch between the initial state of the original MaskGIT (r = 1.0) and that of finetuned ReDi models (which use 1.0). Although the performance of ReDi degrades with smaller r, observe that reducing r also decreases the Conditional TC of (Fig. 5b). Consequently, trade-off emerges between ReDi and the Conditional TC of π. This balance yields the optimal ReDi model performance (ReDi--FIl at r = 0.3. Accordingly, we use = 0.3 for all image generation experiments in the main paper.
B.2 Hyperparameters
with stochastic initial states, we use the Adam optimizer with a learning rate of 1c-4 and a weight decay le-5, The model is trained for 13 epochs the original ImageNet dataset [9], with each epoch consisting of 1.3M images. The global batch size is set to 512, and the training take 96 GPU hours with A6000 GPUs.
Hyperparameters for Image Generation For finetuning MaskGIT 3 K
For training the ReDi variants, we set the global batch size of 512 and applied a cosine learning rate scheduler with decay over 100 epochs. Each epoch consisted of 50k pairs. To rectify the coupling, we control the decoding parameters during pair generation. For ReDi² and ReDi2, pairs for the coupling were generated with CFG level of and 16-step decoding. For the ReDis-distill model, pairs were generated with CFG level of and 16-step decoding. Fach of ReDi variants take 16 GPU hours with A6000 GPUs.
22

---

Hyperparameters for Text Generation For all ReDi variants, we use learning rate of 3e-4. The models are trained with a linear warmup over the first 2500 steps. The global batch size is set 128. For Di4C+ReDi', we fixed the random variable À for the control variates in Di4C [17] as 0. The best-performing model selected and used after completing 25,000 training steps. Each training takes 12 GPU hours with H100 GPUs.
C Standard and Perturbed Rectification
Algorithm Coupling Update (Standard)
Algorithm Coupling Update (Perturbed)
1: Input: p(Xo). pa, dataset size N
1: Input: p(z1), pe. dataset size N
2: for 1 to do
for 1 to N do
3:
20
p
3:
p(z1)
4: 21 poloil
t Uniform(0, 1)
5: end for
5:
6: Return: {100) (1(( 1
ppo( (2x
7: end for Return: {11 (1) ((1) )
Algorithm 3 Training (Standard)
1: Input: ((=60) (z))) )
Algorithm 4 Training (Perturbed)
2: while converge do
3: Sample (Io,, 2i) from {(10 ( (
1: Input: ((((") (0))
4: &~ Uniform(O, 1)
2: while converge do
5:
27 p( (zo
3: Sample (Iz, 21) from ((x x
6: C CrossEntropyypp m1), x1)
4: L= CrossEEtropy (po (r)).
7:
Backpropagte and update parameters
5: Backpropagte and update parameters
8:
end while
6: end while
Algo. m highlight the key difference between standard and perturbed rectification. The standard rectification updates the coupling of (Xo, and samples X, from (., X1) during training. In contrast, the perturbed rectification updates the coupling of (X,, X1) and use X₁ directly during training. As the time is randomly sampled from [0, 1), the perturbed approach also covers the standard rectification case at
D Additional Results
D.1 Detailed Values for Fig.
We provide the detailed values for each metric that corresponds to Fig. in Tab. research.
for future
Table 3:
generative perplexity scores.
DUO+DCD
ReDi
ReDi²
ReDj
Di4C
Di4C+ReDi
97.22
79.35
69.01
53.24
55.42
47.50
8
72.18
48.53
45.11
36.33
36.52
30,92
54.82
37.13
36.42
30.34
27.66
24.81
32
46.05
31.21
31.56
27.75
23.04
21.88
42.38
29.12
29.85
26.78
21.54
20.32
128
39.74
27.16
27.53
25.93
19.82
19.38
256
38.50
26.42
27.11
26.96
19.79
19,00
512
37.48
25.38
26.80
25.41
19.16
18.70
1024
36.21
24.13
25.84
24.80
18.68
18.44
23

---

Table Open entropy scores.
DUO+DCD
ReDi
ReDi
ReDi
Di4C
Di4C+ReDi
4.82
5.43
5.43
5.33
4.81
4.96
5.25
5.49
5,44
5.3
5.02
5.07
16
5.35
5.48
5.44
5.28
5.07
5.08
32
5.36
5.44
5,41
5.26
5,05
5.09
5.35
5.42
5.39
5.25
5.03
5.07
128
5.30
5.37
5.35
5.24
4.98
5.07
256
5.28
5.34
5.34
5.26
4.95
5.06
512
5.25
5.31
5.31
5.23
4.92
5,06
1024
5.23
5.28
5.29
5.21
4.89
5.03
D.2 Training Cost Analysis ImageNet
Table 5: Training cost comparsion ImageNet.
Dist. Iteration
GPU Hour Iter.
Total Training Time
MaskGIT 3
1800h
1800h
SDTT* 10
68h
204h
Di4C [1
50h
50h
ReDi ReDi
15h
30h
15h
45h
We further analyze the efficiency of ReDi by measuring GPU hours per iteration with A6000 GPUs. include costs to generate the rectified couplings ReDi variants. As shown in Tab. 5 ReDi demonstrates its training efficiency in both iteration and total cost. This efficiency is achieved by following reasons. First, as discussed in Section 4.4. and Figure 4(b) in the main paper. the rectification process can be trained with only small portion of the entire training data (50K images vs. 1M full training data). Therefore, it greatly reduces the cost of forwarding pre- pre-trained trained models and enhances convergence speed of each rectification iteration. Second, unlike other distillation approaches, ReDi requires only the student model during training, avoiding the cost of operating two models simultaneously.
D.3 Rectification CFG Ablation ImageNet
Table 6: ReDi rectification CFG ablation.
Table 7: ReDi distill rectification CFG ablation.
ReDi (4-step)
FID
IS
ReDit-distill (I-step)
FID
IS
CFG
9.76
163
CFG
14.11
139
CFG
7.52
228
CFG
13.25
150
CFG
7.77
252
CFG
11.68
182
CFG
8.63
283
We ablated the CFG level that is used during rectification. As shown in Tab, 6 optimal CFG value required for rectification and distillation differ. We used the best CFG values for rectification in our main experiments.
7
we found that the
D.4 Inference CFG Ablation on ImageNet
ablated the CFG level inference time and reported in Tab. 8 Similar to other discrete
flow-based models g 17 34 39), the performance of ReDi varies with different CFG values.

---

Table 8: ReDi- inference CFG ablation.
CFG
5
6
7
8
1-step
55.35
39.33
37.43
39.62
42.46
45.27
47.75
49.90
4-step
22.39
12.49
9.22
8.06
7.69
7.58
7.59
7.62
Table 9: ReDi inference CFG ablation.
CFG
5
6
7
8
1-step
26.59
21.80
22.30
23.38
24.54
25.51
26.33
27.22
4-step
11.47
8.68
8.03
7.86
7.86
7.90
7.92
7.95
D.5
Additional Qualitative Results
In addition Fig. 2 in the main paper, additionally visualize the 1-step and 4-step generation in
Fig. and Fig. 7 As discussed in Sec. 4.1 one-step generation result of ReDis-distill shows compa-
rable fidelity against 4-step generation result of MaskGIT [3, and ReDi models show comparable
4-step generation quality against SDTT 10 and Di4C 17
We also provide qualitative results for text generation in Fig. 13
As discussed in Sec, 4.3 ReDi
generates plausible texts. We use the unidecode package for decoding Unicode characters.
25

---

MaskGIT
SDTT
Di4C
ReDi
ReDi
ReDii-distlll
Figure 6: 1-step generation results on ImageNet. Visualized class labels: 11, 108, 933, 28
26

---

MaskGIT
SDTT
DICC
ReDi
ReDi
Figure 7: 4-step generation results on ImageNet. Visualized class labels: 11, 108, 933, 28
27

---

<lendoftextl>, Recently, my older sister showed up in Scoul,, she was shown some of the monuments and and and buildings. I was 12 yearyear-old girl. and she was told that these monuments have be be damaggeclendoflxxt>>>>> I would definitely like to make any thoughts on these monuments, as in my experience these put in the plans, and so it very impossible to ensure they are be removed, and not, they not part the new level. However, however, even though the project took place in the major capital city, and one of the recent studics found in it has reduced its entire carbon footprint, in the years since he took office. On the other hand, if you don't t act this, there will be more more and Also, it has changed so much since you there, many lots of construction and new state the art high-rises.
He has special interests in urban design and the cearronnennt.eeddffexxittt more people are reading The Jerusalem Post than ever before. Nevertheless, traditional business models are no longer sustainable and high-quality publications, like ours, are being forced to look for new ways to keep going, Unlike many other news organizations, have not put up a paywall. We want keep our journalism open and accessible and able keep providing you with news and analysis from the frontlines of Israel, the Middle East and the Jewish World.
Reader, As you can imagine,
St. Louis, that Paris and Paris signed plans for the settlement. Beijing said the is is isancing progress on building settlement of the Jewish State and the claimsand there will be more support. the city is working up new percent000-ffee free "itary buffer wall" Jerusalem late last week, an Paris city official said. Laredo, and Tel Aviv.
Israel announced development for economic and economic development of the construction,, it was contacted by Ma'a, aa.a. the Gaza Authority that it the going to to., Avi Meirman, head of thes UN agency in Gaza Israel the Israel and international other have in the and and the aid and to focused on the development in Gaza's local sector. End's note on May a was written by The deee-eeeedoiixx--EEBEEDD Portland, Ore. If his memory doesn't translate well, a fifth grad is diagnosed with Alzheimer's 's dementia- - is the target of a local high student.
The root project, which was placing a log, was that he wanted to do, to of of photographic memory, as well. It's enough for him.
But the fourth grad, who died Tuesday, had own own needed worst moment. When Paul McCallum in the classmates at a, of summer summer camp to keep him from having a memory-like memory, it was figured, who was placing the place the theighthouse, and the event to take place. It to everyone at the camp.
"I said 'He's gone.' I't tell him to go," " said said at the camp. of the other people, mc said, 'Where's me?' I want him to see I've really graduated a student, I've even done my my yet.
See more the let's thoughts. Ben McDermott
Milwaukee, Wis, in good state of his young mind and seff-consciousjection,, with the 4-2 record (and 3.2 scoring record) on the team, he a senior year senior, He one the best players of college young career, an incredible talent that, to making a number plays in game-day. practices and games. Consistently. Looking the him, he has, crunching, outages, doing doing very good job completing it. Taking over the bulk of the team. with a very good things, doing is doing is job. More info: Wisconsin's 's transition from duc diligence to practice
Speaking to the media, and said he is his to in offense, and he's perfect fit, and the president of the team that he was going to step down the they thought of him in the NBA draft too Ben O' o'an wants to break out with the team,, he he is starting center in the team, and he's s a kind of starter of quarterback. But isn't It the not of hope in sight now.
"He is not the smartest intelligent. Not that the smartest kid but but but the smartest kid," insider told the
Sentinel.
"He went through a period last year where he was slow, and he lost his high sense of alertness. Really," Ron source said, and with the "cuteure to be done." After theclendoftextl>
Figure 8: A text sample generated by 8-step decoding with DUO+DCD 31
28

---

<lendoftextl> and fift to in the country of sweeping the country, and he says, "What am very passionate about is trying to spread this country around the world, and this is because have spent lot of years and years, in the process of studying the character and growth and character of this country, of the history of it. These discoveries are published in many of the books, books and magazines around the world, and also the history of its history and development, in books written by the people of the Netherlands, the Netherlands, Germany, and, and now, in the United States."
Holland he says, at the moment he read the book, "One of the greatest people in the world, who know who who lived in the world, and who shaped the character of the country, the modern country, it's one the most powerful books in the world, and has attracted a great number of people to America, It's one of the most important books that I read written the America.
It is the perfect book to look the history and the character of the country, of America," " he says, in "dark very different way." a difficult place. "It is the most important book of this time, and to some extent, the book is the willingness of the American people to the struggle to find their place in the history and development of the development of this country all over the world. For the young people, there has been a lot of the power and potential of the country, in the world, to the world, if you are putting your place in America and America." Advertisement
Holl, the African American in the book., in the U.S., has spent so much of the rest of his life sitting in the chair hall of the universities, in thinking of, and helping the students in get their degrees, listening to their ideas, thinking about the character, all these countries, and studying them, and some of the basic ideas of how young man ought better succeed in the universities, in the United States, in Europe, and around the world in the education, teaching. the education and the work of young people.
"But in my youth, the people of the Europe, which is now one of the most powerful countries, has come to be European and European," he says, are pushed to the extreme right of the American political system and start
to think in some way, and George he once said, "and for time we didn't 't know the truth of what is was going to happen."
In fact, George Bush first became elected president of the United States, Jan June 8, 1968.
Advertisement
Holland, a professor who now at the University of London., when he first reading the book, that he read him with being "graat man, good citizen of this country, of this great and great country." He says, however, "From the very beginning, Bush's life was an ordinary man, and he followed the history of the current world in Eastern Europe and Europe, and what it mean if Europe didn't no longer dominate the world. He was young man, and he followed the course of Europe, and the development of the modern world."
He has spent almost his entire career writing, he says, "Bush's writing said that, and that, the European Union was great, and, the success a lot people around the world. It was because of other people's success, that he became an great citizen. From Paris to Paris, he came to the United States. Advertisement
"From there, he, with the help of the people of the continent and around the world, realized that for many people, America was a powerful country. The power of these people, people from other countries and Europe, and people from the U.S. together, these people became an important part of an American." Brown, the only American in the book in the United States is in the Netherlands. Last year, in Brown family lost Daniel and James, as boy, in the streets of the street, the horrors of living and traversing their country with the world, real and unseen, everywhere, and everything that surrounds them and haunts them. Advertisement
Last year, he says, in Baltimore, the heart of a still- unlle-niverse Baltimore, that was the beacon of hope. "It was all about our life, and when we got on the plane to the United States, just a few years ago, and this thing happened, and, it had come to the whole States, in the U.S. and became one of the most important stories of the timeclendoftextl>
Figure 9: A text sample generated by 8-step decoding with Di4C [17]
29

---

<lendoftextl> follow a meteorite-Earth-tr axis of motion, even as the Sun away from the Turbine hit the Earth.
The planet nucleus and some other objects are puzzled by the shift and flatterings, according the recent paper "The Population of Comet Objects," because objects between bodies have interplanetary mass. For example, some matter belonging of smaller objects reside in the planet the nucleus, in groups with masses of up to 400 times more mass in common.
The core of the earth life- gifenn has come into contact with host of other celestial bodies, big one from Earth. Every year since, increasing numbers of people, astronomers and scientists have steadily expanding the search for the universe, turning from theories and hypotheses, the theory that it would "diff easy" in 1,000 ago on the hasis of evidence that we have a few billion yet to see it. Image:
(c) Gravity and Space, Inc/ / Mark Tinglin
Mostories alike remain cautious. Details remain as to how big or massive will become. And yet there is one place where a lid keeps a lid on the rate ofcheting-up.
We don't 't have any good answers about what we would do after the extinction took hold, but we don't 't think the extinction is still filling the void left by asteroid, which swept the world cast of the in 2000. March 23, 2001, recorded on Terra, saw that record low and cold temperatures, producing series of dust clouds of low pressure that caused severe and likely extinction. Life at that was more than 250 million cubic kilometres cubic miles) in total. At this time, it seems, the broadening of the Sun's 's away from the core of our core, which transgresses the basic tenets of Islamic faith, threatens our planet and surrounding regions well According to a report released by Saudi Arabia's 's Health Ministry on April 3. new cases of infection are reported around 1,100 cases each day under the new Saudi kingdom's 's sharia law, designed to restrict the supply of medicine in the Kingdom of Arabia.
Those with long histories of Islam will note that some leaders of Riyadh fundamentalist Salafafist movement in the government of Saudi Arabia, have tried to lead the movement underground by receiving millions of dollars in cash from Iran and al-Qaeda of the Muslim Brotherhood (AQAP), among other things. But betters, things gets better unless these clerics stand up and take action. Read more about today's 's reenlisting
Subscribe to the Guardian 's English-style news email every Monday 31 December
In interview, Dr. Juan Martinez Rubens, the Geneva-based regional director, told the Financial Times in Geneva. "A human cell is spinning ahead of gravity, the sense of the cell's rotating mass, so that the incontradable processes are managcable and you can get the good results. This highlights the fact that in the body's 'ss evolution through evolution, something new material either accumulate in the body else it undergoes damage by the elements-the same thing goes for the brain. The shape of the limbs is affected these changes," he said,< bidd.<lennoftextt those who know the thermoklosan (EGS) compound found in many plants is responsible for the growth and destruction many proteins and organ systems (Li and Wobler, 2003). ERS proteins are responsible for the surface proteins and have been characterized by attractive "dynamic dynamics" mixing of dissolved biomolecules on surfaces. Egel is similar, affectionately being referred to as "all else's such particular lipids and hemispheres."
Egel calls these mechanisms an "illustance chemistry" with strong mixing behavior and fundamental electron rotation. What Conrad et al. and Bayes In-Seq -aabuin- vol. (Figure 5; Herschel et al. 1999) call the 'static' effects of heating and 'interration' is the oxidization to which the floor's lipoprotein II and metallophosphate [pH1202] at room temperature swells with dissolved organic compounds as far away as the floor's solenvironments." "
Today, room temperature is generally reported to drop to 10-2 degC (69.5/C) in preference to 40 degC F. Not surprising has alteration of the oxygen oxygen-fogal state over centuries taken a such global view of all organisms. For example, MEAs are damaged by CO2 excretion in Type I and Type 2A; Bartletta, 2005. Whereas oxidative damage in Molybic-sensitive MEAS has increased for many decades (Durkin et al., 2005). it is relatively global view of all changes in the environment. Indeed, in this respect it is challenge understand molecular processes order to explain theclendoftexti>
Figure 10: A text sample generated by 8-step decoding with ReDi
30

---

<lendoftextl> home in Highland Park in August 2007.
Police said Frank sat down with some employees Buickler Industries who said they found several items that they said had managed to disappear from behind.
"Several items in the yard had gone missing including a suit, wire harness and chair, one thing missing," police said, "The other was missing as well, a found rug, skull, jewelry worn by man schnapeer had gone missing and all that he was doing had gone missing."
As soon as Frankley came home, employees quickly tracked down the man and discovered many items that he owned until he was at a building insurance company, police said.
To his surprise, the debris was left there for days but which is when the company asked appraiser for help an explanation.
He said he asked "many questions" and received expert feedback, including the project supervisor who did excavation and tests.
"Their report is so much better than the situation" said Lockhart. "CHASE, "Part and Part II: A Criminal Investigation," "" reads the official report.
But the project foreman contacted real rep who for months negotiate the appraiser's 's ownership of the yard as dumping ground months later. She and the Buildersers Basingers eventually agreed to include Frank's name into the report.
When Frank Curley, who worked a high school teacher with three managers moved to Chicago in 2007 and flew the airport, Frank was found abandoned with stolen stolen toys all while trying to find new home his feet.
He's leading a crime task force and said he reportedly found himself caught up in crime. "There's 's a two-word movement linking him to organized crime," says Lockhart. CNN went to visit his home and spoke with reporters. Van Huey, where Frank lives is seen watching helplessly with his daughter on TV.
His family photo still shows Frank's 's home. (Photo: FASHION HAYY) "He sure dressed things up with that ace on on screen H Huey tells FOX News.
Although Frank has been identified by police, there have been suggesting he has been used to appear on Facebook. YouTube and more, clendofiiiit light is very dark in the environment but when deerred in in room, often it is not visible, especially on the console version of Destiny.
Jiteun says the hypothesis is crucial to make sure players work in conditions that avoid acidity and extreme stress.
"The idea of the light is IR," he told Ars. "This really is all about infrared exposure. It makes it really important to just have in there that you can paint picture." "
With the hypothesis accepted, the development team says they'r rr moved one step closer to working with Hough, Nell and Guillamics, and they's 's working in both California, Australia and Japan.
"It's an enormous enthronement that gives very a lot of energy to important exploration and action," " Chun said. "This holds that the world can be shaped by our ability to get better images of ourselves in order to do other things than human contact." "
In the console version of the game, players can take you around a cave-like world with set of environmental stables crafted with a cast of material. You duld the object with various materials to tear down or one of the objects sets on fire for health or sets another to mark its crumbs. It demo demoed by Tom Hough, producer Andrew Nell on Sony Sony Blu-ray 6.0 and PS4 Compact. PC
version is available here.
We'd d have a program in which it was really compelling to see how the mission would work." Jiteun said. "Where we' 've had really good success is with some of the Caribbcan's 's blue white ships rolling in the occan off the coast.
"The actual blue ships rolling around are different from the environments of the game and the footage." " If you choose to use pure blue or not really blue, should be real instead of dark black, the team says. "If the environment is completely red, the sky should be pure yellow instead of dark black; the blue should be visible," Jiteun said. "If you can't 't see below the surface, you just put in some dark black. You can't see it and then you see it because it's completely there." "eere."<lendor" "<lendoftextt> "oow it could be more convenient to have healthy air than buy books online?" Dominic Scherbacher said.
The German striker added and new technologics such as haze could lead reduce the damaging effects of heat and pollution. The landmark study, published today by an international team found more than 77,000 people spent PS86m as part of the airclendoftext!>
Figure 11: A text sample generated by 8-step decoding with ReDi²
31

---

<lendoftextl> in search those in need Copyright by WKRN All rights reserved Mike Miller- In Nashville's Maaityy-raiinn reported Larry Wright received a phone call that told people the city is paying as little $4 an hour for 36 hours and free to alleviate their needs on the strects is just what a truck can get there. (4:30 Uber and MAX)
Another man was killed in his life, but growing concern is growing as the situation ramped up Wednesday morning when rescue worker was driving his Countachio Cavalry. Baggi, 43, recently owned an Italian home and began work in Central Italy. He hopes the relief money will help him make some money and in addition to few of road trips he makes this year
B Baggi said he'd d like to receive supplies as order to anticipate food supplies and damage. "I don't t feel the need to drive any more. don't I get drinking drinking water," he said.
Even if the man doesn't want to get some precious food after finding firewood, he's talking about driving cars without running water, "without shelling" out of his car.<lendoffextt-In the concern that the crisis is likely to caused more suffering in Europe, four top international aid groups from the University of California. Santa Barbara have held Mass at schools in Washington. On Tuesday, the groups with officials from Belgium and Kuwait at the G2020, the G20 economic summit for the what-wealhhrich nations.
The United States government and foreign administration have also to normalize the current post-The Security and Climate Council accord between the member states and the EU member states to bring peace to Europe. The negotiations take place in 50 states, with roughly 8,000 men, 700 and 700 countries and governments. U.S. Rep. Bill E. Lee. member of the accord accord, threatens to cost more than 3,000 lives and has become the U.S. target of the humanitarian crisis.
Davies of the Army called the "Warrior" led the efforts of states affected by war in World War II under the Initiative of 1789, and a total of 44 states led one of the 672 Rep. Lee forces to the position they had fallen. The majority of the victims are women. At least 39 children guarding the victims are now from overseas, the Army confirmed late Tuesday.
Michael J.L.. Wilson, an assistant professor of political and cultural studies the University of California Santa Barbara, said Mr. Lee was committed helping people in severe, life- difendefen conditions. Mr. Wilson appeared on Fox News in 2014, drawing extensive social media coverage and the attention of politicians. He also wrote federal papers at Congressional hearing, and he did so in a public May meeting with friends of the Falmouth Fish and Wildlife Society.
Mr. Lee's family said he was conscious and had taken part hospitalized treatment. His attorney, Falmouth County, declined to comment. Late Tuesday, officials on the health.gov gov website said the agency's programs in all 99 counties worked equally. The website highlighted the differing views on several crucial issues, as the events take on expanded response to the country in the wake of the crisis.
"We have these people who are out in these rains and valleys these deep jungles, these people in sinkholes, these people with cattle," one resident, C.R.M. Fort Lee, Ga, blamed the state the flooding. "And, we have internally displaced people, obviously. Hopefully those people have a better chance to get back on their feet at some point.
We know that some people have been featured in the movies or in the news while watching them. For example, movie will show Captain William Jr. Hoyt (taken a modern repellum meldass, photorealized) as he is told about his doctor: a former doctor, a depressive, and now one crucial witness he's lived with. And if it doesn't "tssnttjji just end with the character. Hoyt will spend months with Christopher Nolan as the titular, and in yet another upcoming movie even more. George Reisingerly, director of the late John Segel and George Clo, has also direeted Red-Hurts. The film is one of the most iconic Star Trek films of the 1960s, giving the Starflcet actor better idea of who he is than most people realize. Over the weekend Washington, he has found - not that he did, but than many would - that he will be replacing his star with the beginning of larger group in him that includes World Health Organization's 's Director Tom bubbed in broad swath of Washington political establishment, with of the first time lawmakers criticized Obama administration in the wake of an economic research study that that 400 million Americans die from heart attacks and birth defects. "<lendoftext>>
wew.iiosoonnenootttttttt>. Word has
Figure 12: A text sample generated by 8-step decoding with ReDi3
32

---

<lendoftextl>, in the NBA, foothall and Kurt Ramon on HBO's The Wire.
"It sends a message we're sending and we're sure that [the industry] doing their job." O
Cameron is an actor and is one of the movie stars of the women's 's baseball union union (L..A). He will co-star Dustin Sheen and Ted Smith in The Jack Black, too. The movie will mark the 50th anniversary this year of the centennial of baseball. An Australian writer is writing the screenplay. the follow-up, the movie, and is making arrangements to make the film.
Cameron will see TOG: Batty time-traveling through sports, action, grills and music. The film was first made in 2003 Australian release!
"In the movie, we really capture some of those spirit of L..... The film is populated, the players and the film stars stars as players, and tells stories that will make people remember stories about sports. "We use sports movies as a way to connect people, and we want them to remember what they do no matter how they finish. It is told through a female lens."
TAG; Batty is 0.4
"Women wake up now to read this movie, is is. very, very important news. Much research is important, and especially science fiction and requires great preparation preparation and effort.
"Men can make bad predictions, and so when approach L.A. we should be careful and make sure that the industry are doing their job." "ob."<lendde"" "<lendoftexi)> When the U.S. was to be building the tallest building of the U.E. Building two centuries it, there was online marketplace called Bazaar, which first was launched in 1994. The site since become a popular platform for investors in properties like Dr. Seuss, The Real Estate, and Justin Bieber, and considered considered a boom time. Many other sites in the United States are offering the same deals to owners of government buildings for deals related to housing issues.
The site was a hassle to set up and promises to be one-stop experience for all the journalists looking to get good look the major U.S. idea behind the site was built, specifically for those looking to make the switch hetween movie with Simon Johnson, NBC News, TV show, and spaceship. We're theree our findings, can check out the website about the site here. Of course, even Dr. Who is is famous when he asked: How and where is he? Where where it all come from?
The Superman, R. Goldberg the guy who directed the Batman movie Superman, and is also an actor He is well-known author several films, Martin Gaerdes: The Return.
Goldberg, Mr. Supreme a brilliant former Federal Reserve, and former U.S.T. senator and Attorney- General of the U.S. Hc was law and a field professor at Harvard in 1995, He graduated M.O. Harvard Harvard in 1998, and M.D. in 2000. Hc's degrees, one Johns Princeton University in 2001, and the Johns Hopkins University in 2002.
Other things to know about this site:
Flaset Tau: Mr. Oz, up together with a group of Bible Studies students in early June of this year,. J. Dozen, formmryy the family patriarch, is the fourth-Secretayy Secretary of the United States. Fired by CBS in November 2014.
T-fus
Time: JJJ. has made some of the most controversial decisions to right-wing politicians, including The Wolf of Meand, the president of the United States, in recent films. Advertisement
Read the more on Slate: Michaela Pmana
Now. it lets you get all the latest information you need about the Star Trek universe. Later this week, you can expect to take part in the events of the 2017 Action Hero.
This is a great way so our fans can build on our work and invest in telling the real truth about the franchise. Let turn all of the voting machines for your favorite TV series, and what could could possibly think to say? The original Star, A.
The following clip is a trailer created to coincide with the episode and the 33th anniversary of the 2017 Action
Hero event.
The cast signed David Tennatii (91) and C.I.E. Shatner (R<lendoftextl>
Figure 13: text sample generated by 8-step decoding with Di4C+ReDi
33


Reviews:
[{"review_id": "sqXtw5Sghy", "reviewer": "Reviewer_riBK", "summary": "This paper addresses a key inefficiency in discrete flow-based generative models: the reliance on many sampling steps due to weak coupling between base and target distributions. While rectified flows have been used in continuous domains to straighten the path between source and target distributions, they have not been explored for discrete data. Moreover, most discrete flow models assume factorized transitions of the form $p(x_s \\mid x_t) = \\prod_i p(x_s^i \\mid x_t)$, which ignore inter-variable dependencies and thus lead to suboptimal approximations. To overcome this, the authors propose ReDi---a progressive coupling mechanism. At each iteration $k$, the model samples from $p(x_0)$ and trains a model to predict $p_\\theta(x_{k+1} \\mid x_k)$. The authors demonstrate that this strategy contracts the factorization error, defined as the KL divergence between the joint and factorized distributions. Empirical results on ImageNet (class-conditional) and OpenWebText show that ReDi achieves one-step generation with competitive results compared to distillation-based methods.", "questions": "Questions and Comments: Can this progressive coupling mechanism be extended to continuous flows? Why was the discrete setting prioritized? How does generation quality (e.g., FID for images) improve compared to the baseline factorized model in the multi-step regime? What is the actual training cost versus knowledge distillation? Finally, how sensitive is performance to the number of coupling iterations $K$? Could $K$ be adaptively chosen?", "limitations": "n/a", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths: The paper tackles a timely and nontrivial challenge in discrete generative modeling. The proposed progressive coupling mechanism is intuitive and theoretically well-grounded. The contraction result on the factorization error is an insightful contribution. The empirical results, especially one-step sampling, are promising.\n\nWeaknesses: Training ReDi appears to require multiple forward models, trained sequentially (at least 4 iterations as shown in Fig. 4a), which raises concerns about total training cost. It’s unclear whether this method is actually more computationally efficient than distillation. Moreover, improvements in generation quality appear modest, and the method has not yet demonstrated scalability to larger datasets.", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for your valuable comments and positive assessment for our work. Below, we address the questions and concerns you have raised.\n\n> **Q1.** What is the actual training cost versus knowledge distillation?\n>\n**A1.** We appreciate the comment. While our rectification process requires sequential training, we clarify that each rectification step is much more efficient than distillation for the following reasons. First, as discussed in Section 4.4. and Figure 4(b) in the main paper, the rectification process can be trained with only a small portion of the entire training data (50K rectification data vs. 1M full training data). As a result, it greatly reduces the cost of forwarding pre-trained models and convergence speed of each rectification step. Second, unlike distillation approaches, our method requires only the student model during training, avoiding the cost of operating two models simultaneously.\n\n**Table A** compares the training time of ReDi with the teacher model and other distillation approaches. It shows that the entire rectification process for ReDi$^3$-distill requires **only \\~3% of the time** needed to train the initial teacher model, and faster training than distillation methods up to three rectification steps. We appreciate the reviewers’ comment and will incorporate this result in the paper.\n\n**Table A.**\n||Iter.|GPU Hour / Iter.|Total Training Time|\n|:-|:-|:-|:-|\n|MaskGIT (Teacher model)|1|1800h|1800h|\n|SDTT|3|68h|204h|\n|Di4c|1|50h|50h|\n|ReDi$^2$|2|15h|30h|\n|ReDi$^3$-distill|3|15h|45h|\n\n> **Q2.** Moreover, improvements in generation quality appear modest.\n>\n**A2.** We would like to kindly remind the reviewer that our primary goal is to improve the few-step generation performance by reducing the factorization errors in discrete diffusion models. As shown in Table 1 and discussed in Section 4.2, our method achieves significant improvements in the one-step setting compared to both the teacher model and other distillation baselines. While it shows comparable 4-step performance to competing methods, our method still exhibits much lighter memory for training compared to other distillation baselines that require additional teacher networks, and faster convergence using only a small portion of data for rectification (Fig 4(b) in the paper. please also see A1).\n\n> **Q3.** The method has not yet demonstrated scalability to larger datasets.\n>\n**A3.** We would like to clarify that the ImageNet and OpenWebText are widely used datasets to evaluate the image and discrete diffusion models. Although time and computational constraints prevented us from testing on larger datasets during the rebuttal phase, we believe that our method can easily scale to larger models or datasets for following reasons. First, our rectification process and its theoretical foundation are model-agnostic and are applicable on top of pre-trained discrete diffusion/flow models agnostic to their architecture and scale. Second, unlike distillation approaches that require both teacher and student models during training, our method does not increase the memory footprint beyond that of the original pre-training. We plan to open source our code to facilitate the future community efforts in scaling our approach.\n\n> **Q4.** Can this progressive coupling mechanism be extended to continuous flows? Why was the discrete setting prioritized?\n>\n**A4.** We would like to clarify the connection of our work to continuous flows and the motivation for prioritizing the discrete setting.\n\nAlthough similar progressive refinement mechanisms have been explored in the continuous domain, as we note in our Related Works (Sec. 2.3), our work was prioritized for the discrete setting because the core challenges are fundamentally different. For continuous flows, the primary obstacle to few-step generation is the non-straightness of probability paths. In discrete flows, however, the concept of path straightness is not well-defined, as there are no intermediate states between distinct categories. The critical bottleneck we address is instead the factorization error inherent to these models.\n\nOur key contribution is thus to characterize this specific factorization error and resolve it with our proposed rectification process. Interestingly, this process, despite being derived specifically to address the unique problem of discrete factorization, closely resembles the rectification methods used for the different challenges in continuous flows. We will add a discussion of this point to our method section to further clarify this distinction and connection.\n\n> **Q5.** How does generation quality (e.g., FID for images) improve compared to the baseline factorized model in the multi-step regime? Finally, how sensitive is performance to the number of coupling iterations $K$? Could $K$ be adaptively chosen?\n> \n**A5.** We would like to kindly clarify our model's performance in the multi-step regime and its sensitivity to the number of coupling iterations, $K$.\n\nRegarding the generation quality versus the baseline in the multi-step regime, our method exhibits a trade-off between FID and IS, as shown in **Table B**. While the Inception Score improves compared to the baseline, the FID moderately degrades as $K$ increases. It is important to note that our primary contribution lies in the few-step regime, where the model shows significant gains from the rectification as demonstrated in Table 1\\.\n\n**Table B.**\n|8-step|FID|IS|\n|:-|:-:|:-:|\n|MaskGIT|6.51|227|\n|ReDi$^1$|6.57|249|\n|ReDi$^2$|7.37|261|\n\nConcerning the sensitivity to $K$, this behavior can be understood as a trade-off between two competing factors. On one hand, each rectification process reduces the Conditional Total Correlation (TC), which significantly improves performance with fewer-step regimes where the initial factorization error is large. On the other hand, each rectification can introduce a small amount of distributional drift due to imperfections in the base model. In the multi-step regime, this accumulated drift can offset the benefits of TC reduction, leading to the observed performance changes.\n\nFinally, regarding the adaptive selection of $K$, we agree with the reviewer that this is a viable approach. The general principle would be to monitor a chosen performance metric, such as FID or IS, after each rectification iteration and select the $K$ that provides the most desirable outcome based on that evaluation."}, {"review_id": "B0BkPLdYhd", "reviewer": "Reviewer_EMvH", "summary": "Discrete Flow-based Models (DFMs), while effective for high-quality discrete data generation, suffer from slow sampling speeds due to their reliance on iterative decoding processes. This paper first characterizes the approximation error from factorization using Conditional Total Correlation (TC), which quantifies interdimensional dependencies. Next, the authors propose Rectified Discrete Flow (ReDi), a novel iterative method that reduces factorization error by rectifying the coupling. ReDi theoretically guarantees a monotonic decrease in Conditional TC with each step and empirically demonstrates this reduction. Unlike traditional distillation methods that often involve complex multi-model training, ReDi simplifies the process by operating on a single DFM and directly rectifying the coupling, making it widely applicable and compatible with existing distillation frameworks. Empirical results show ReDi significantly improves few-step and one-step generation performance in image and text synthesis, outperforming or matching existing baselines.", "questions": "# Questions\n\n1. Could the authors kindly define $\\delta_{x_0}(x_t)$ in Eqn (1)? Do the authors mean something like $\\delta_{x_0}(x_t) = \\begin{cases}1 & \\text{if } x_t = x_0 \\\\\\\\ 0 & \\text{otherwise} \\end{cases}$?\n2. L168: How do you determine the stopping criteria for rectification? Do you measure some distribution-level metric like FID after each rectification? Or is K heuristically set to ~2 or ~3, following common practice in continuous rectified flows?\n\n3. How do the time/compute requirements of ReDi compare with existing approaches?\n\n4. Why are different CFG levels used for training ReDi$^1$+ReDi$^2$ vs ReDi$^3$-distill? (Appendix, L393)", "limitations": "Yes, addressed.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "# Strengths\n\n1. The paper is well-written. The proposed approach is well-motivated and important to the generative learning community.\n\n2. The paper provides a way to characterize the factorization error in DFMs by using Conditional TC. This can be a useful tool in subsequent works in the field.\n\n3. ReDi does not require maintaining a teacher and a student model unlike existing methods that follow a distillation paradigm, thereby easing memory requirements.\n \n4. The authors demonstrate improvements in few-step and one-step generation performance across image and text synthesis tasks, outperforming or matching existing baselines, highlighting its practical effectiveness.\n\n# Weaknesses\n\n1. The paper could use some more exposition on the suitability of Conditional TC for characterizing factorization error (L130).\n\n    - Are there any previous works that attempt to measure the gap between the conditional transition probability and the factorized product of marginals, or is this paper the first to do so?\n    - Are there other metrics that can be used to characterize/measure factorization error? E.g. Cond. TC measures the expected KLD between the conditional and the marginal.\n        - Could the authors briefly highlight why KL would be more appropriate than other methods in this scenario? (E.g. Cond. TC is directional, as $D_{KL}$(cond || factorized) $\\neq$ $D_{KL}$(factorized || cond). Is this better than evaluating something symmetric like JS divergence?)\n    - Some discussion like this would enrich the paper.\n\n2. In Fig. 4a, the authors show that TC keeps decreasing with rectification iteration (shown up to 4). But it is noted in L277-279 that the authors empirically observe that the actual quality of the DFM drops with more ReDi iterations.\n    - This empirical measurement seems to be absent; it could be shown alongside the TC in Fig. 4a.\n    - Even though TC is decreasing, why does the performance of the model get poorer over time? Does that mean there is something (e.g. accumulated error) that TC fails to sufficiently capture? Some additional discussion on this phenomenon is required.\n\n3. In L213, the authors mention they report the best FID score from trials with CFG values [1, 2, ..., 8]. Does each row in Table 1 correspond to a different CFG value? (and what are these values?) Does ReDi with different inference step count work better with different CFG values?\n    - Since the authors have evaluated trials over a large set of CFG values, it would be useful if these results are provided as a curve.\n\n4. Though ReDi's performance is outstanding for 1 step generation, for 4 step generation, it is mostly comparable or a little behind other methods. Do the authors have any intuition as to why?", "was_revised": true, "final_justification": "Discrete diffusion/flow matching is increasingly becoming a topic of interest to the generative learning community. I believe this work further expands the field in a productive direction, by translating some ideas from classic rectified flows (i.e. reflow techniques) into the discrete domain.\n\nI thus recommend accept.", "rebuttal": "We thank the reviewer for your valuable comments and positive assessment for our work. Below, we address the questions and concerns you have raised.\n\n> **Q1.** Are there any previous works that attempt to measure the gap between the conditional transition probability and the factorized product of marginals?\n> \n**A1.** We thank the reviewer for the question. We interpret the term \"measure the gap\" as having two potential meanings—(1) to empirically quantify it, and (2) to theoretically characterize it with a mathematical concept. We will address both aspects below.\n\nRegarding the empirical measurement of this gap, to the best of our knowledge, our work is the first to propose a method to quantify it. We accomplish this by estimating the Conditional Total Correlation (CTC) via Monte Carlo sampling.\n\nRegarding the theoretical characterization, previous works have indeed conceptualized this gap using various metrics. For instance, Di4C \\[1\\] used Total Variation to represent it. We also acknowledge that Discrete Copula Diffusion \\[2\\] uses a concept related to CTC to characterize the factorization error of model. We would like to note that our key contribution is distinct in that we focus on the CTC of the data coupling itself (rather than the model), and introduce a rectification mechanism to empirically minimize this gap by updating the coupling.\n\n\\[1\\] Hayakawa, Satoshi, et al. “Distillation of Discrete Diffusion through Dimensional Correlations.”\n\\[2\\] Liu, Anji, et al. \"Discrete Copula Diffusion.\"\n\n> **Q2.** Are there other metrics that can be used to characterize/measure factorization error? Why KL would be more appropriate than other methods?\n> \n**A2.** We thank the reviewer for this insightful question. We would like to answer it from a theoretical perspective and an empirical measurement perspective. We also wish to clarify that KL-divergence was not used as a direct training objective in our work, but rather as a tool for our theoretical and empirical analysis.\n\nFrom a theoretical perspective, the reviewer is correct that other metrics can be used. Any $f$-divergence is suitable for characterizing the factorization gap, as our Theorem 1 relies on the Data Processing Inequality (Appendix, L358-359), which holds generally for all $f$-divergences i.e., our rectification process reduces all $f$-divergences between the joint and factorized distributions, including Total Correlation.\n\nFrom an empirical measurement perspective, however, KL-divergence offers a crucial advantage in computational tractability. Our method for measuring the gap relies on Monte Carlo estimation. To compute the KL-divergence $D_{KL}(p||q)$, we only need to evaluate the term $p \\log(\\frac{p}{q})$ for the points sampled from the true distribution $p$. In contrast, estimating other f-divergences (such as JS-divergence) would require evaluating terms over the entire support of the factorized model $q$. In a high-dimensional discrete space, this is computationally infeasible.\n\nTherefore, while many metrics are valid in theory, KL-divergence was the practical and well-motivated choice for our empirical analysis. We will add this discussion to the appendix to clarify our reasoning.\n\n> **Q3.** Though ReDi's performance is outstanding for 1 step generation, for 4 step generation, it is mostly comparable or a little behind other methods. Is there something that TC fails to sufficiently capture?\n> \n**A3.** We thank the reviewer for this crucial question regarding the relationship between Conditional TC and generation quality. The full performance metrics are provided in **Table A**.\n\n**Table A**.\n||TC of $\\pi_{k}$|1-step FID|1-step IS|4-step FID|4-step IS|\n|:-|-|-|-|-|-|\n|MaskGIT|837|95.16|12|10.90|184|\n|ReDi$^1$|642|37.43|49|7.58|228|\n|ReDi$^2$|580|21.80|90|7.86|240|\n|ReDi$^3$|533|17.02|120|8.37|247|\n\nAs shown in the table, we observe different trends in performance. In the multi-step regime, the 4-step FID begins to increase after $K=2$. In contrast, the 1-step FID consistently improves as TC decreases. This phenomenon arises because TC captures the factorization error but does not capture the discrepancy between our model's learned distribution, $p_\\theta(X_1)$, and the true data distribution, $p(X_1)$. In an ideal scenario where the base model is perfect ($p_\\theta = p$), each rectification step would improve generation quality. However, this discrepancy may exist in practice, meaning each rectification step can introduce a small distributional drift even as it reduces the factorization error.\n\nThe net effect of this trade-off is highly dependent on the number of sampling steps. In the multi-step regime, the factorization error is already relatively small. Here, the marginal benefits of further TC reduction can be overshadowed by the accumulated distributional drift. In contrast, in fewer-step regimes (e.g., 1-step generation) where the initial factorization error is much larger, the significant performance gain from reducing TC far outweighs the minor impact of the drift. This explains the outstanding 1-step performance.\n\nWe are grateful for this question and will add this detailed discussion and the corresponding data to the main paper.\n> **Q4.** Does each row in Table 1 correspond to a different CFG value?\n> \n**A4.** The reviewer is correct that we report the results with the best CFG value for each method in Table 1. To ensure reproducibility, we will add the below table in our appendix.\n\n**Table B.**\n|Step|Model|CFG|\n|:-|:-|:-|\n|1|MaskGIT|6|\n||SDTT|5|\n||Di4C|5|\n||ReDi$^1$|3|\n||ReDi$^2$|2|\n||ReDi$^3$-distill|1|\n|4|MaskGIT|8|\n||SDTT|4|\n||Di4C|8|\n||ReDi$^1$|6|\n||ReDi$^2$|4|\n|8|MaskGIT|3|\n\n> **Q5.** Does ReDi with different inference step count work better with different CFG values?\n> \n**A5**. The reviewer is correct that the optimal CFG value varies with the number of inference steps. In response to the reviewer's request, we provide the full ablation study on this relationship below. This analysis will be added to the appendix as a curve, following the reviewer's helpful suggestion.\n\n**Table C.** ReDi$^1$\n|CFG|1|2|3|4|5|6|7|8|\n|:-|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|1-step|55.35|39.33|**37.43**|39.62|42.46|45.27|47.75|49.90|\n|4-step|22.39|12.49|9.22|8.06|7.69|**7.58**|7.59|7.62|\n\n**Table D.** ReDi$^2$\n|CFG|1|2|3|4|5|6|7|8|\n|:-|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|1-step|26.59|**21.80**|22.30|23.38|24.54|25.51|26.33|27.22|\n|4-step|11.47|8.68|8.03|**7.86**|7.86|7.90|7.92|7.95|\n\n> **Q6.** Could the authors kindly define delta function in Eqn(1)?\n> \n**A6.** We thank the reviewer for pointing out this omission. The \\'$\\delta_x(y)$\\' in Equation (1) refers to the Kronecker delta function, defined as:\n$$\n\\delta_x(y) = 1 \\text{ if } x=y\\text{, else } 0.\n$$\nWe will add this definition to the manuscript.\n\n> **Q7.** How do you determine the stopping criteria for rectification?\n> \n**A7.** The number of rectification iterations ($K$) is determined empirically by evaluating the generation quality after each step. Specifically, we monitor both FID and IS for 4-step generation to find the best trade-off, as shown in **Table E**.\n\n**Table E.**\n|4-step|FID|IS|\n|:-|:-:|:-:|\n|MaskGIT|10.90|184|\n|ReDi$^1$|7.52|228|\n|ReDi$^2$|7.86|240|\n|ReDi$^3$|8.37|247|\n\nBased on the trade-off between FID and IS, we identified $K=2$ as a \"sweet spot\" that provides a large IS gain without a significant FID penalty on 4-step generation. Accordingly, we reported the scores for ReDi$^1$ and ReDi$^2$ as they represent the most compelling performance points.\n\n> **Q8.** How do the time/compute requirements of ReDi compare with existing approaches?\n> \n**A8.** We thank the reviewer for this important question regarding computational cost. While our rectification process requires sequential training, we clarify that each rectification step is much more efficient than distillation for the following reasons. First, as discussed in Section 4.4. and Figure 4(b) in the main paper, the rectification process can be trained with only a small portion of the entire training data (50K images vs. 1M full training data). As a result, it greatly reduces the cost of forwarding pre-trained models and convergence speed of each rectification step. Second, unlike distillation approaches, our method requires only the student model during training, avoiding the cost of operating two models simultaneously.\n\n**Table F** compares the training time of ReDi with the teacher model and other distillation approaches. It shows that the entire rectification process for ReDi$^3$-distill requires **only \\~3% of the time** needed to train the initial teacher model, and faster training than distillation methods up to three rectification steps. We appreciate the reviewers’ comment and will incorporate this result in the paper.\n\n**Table F.**\n||Iter.|GPU Hour / Iter.|Total Training Time|\n|:-|:-|:-|:-|\n|MaskGIT (Teacher model)|1|1800h|1800h|\n|SDTT|3|68h|204h|\n|Di4c|1|50h|50h|\n|ReDi$^2$|2|15h|30h|\n|ReDi$^3$-distill|3|15h|45h|\n\n> **Q9.** Why are different CFG levels used for training ReDi$^1$+ReDi$^2$ v.s. ReDi$^3$-distill?\n> \n**A9.** The CFG levels to define rectified coupling was determined empirically, and our ablation studies show it differs for multi-step rectification versus one-step distillation. Specifically, lower guidance (CFG 1-2) was optimal for direct multi-step sampling (**Table G**), whereas strong guidance (CFG 8) was required to generate the best coupling for the one-step distillation task (**Table H**). This analysis will be added to the appendix for clarity.\n\n**Table G.**\n|ReDi$^1$ (4-step)|FID|IS|\n|:-:|:-:|:-:|\n|CFG 0|9.76|163|\n|CFG 1|7.52|228|\n|CFG 2|7.77|252|\n|CFG 3|8.63|283|\n\n**Table H.**\n|ReDi$^3$-distill (1-step)|FID|IS|\n|:-:|:-:|:-:|\n|CFG 1|14.11|139|\n|CFG 2|13.25|150|\n|CFG 8|11.68|182|"}, {"review_id": "MTBp0FGuYf", "reviewer": "Reviewer_9bDF", "summary": "Authors investigate the discrete flow-based model’s problem of approximation error due to factorization, which causes the inability of such models to generate data in a few-step manner. To overcome this problem, they propose and theoretically justify that iteratively applying rectification of the coupling minimizes conditional Total Correlation (TC) and consequently reduces the approximation error. Furthermore, they empirically demonstrate that approximated conditional TC strictly decreases over rectification iteration. Finally, authors validate the ability of rectified models in a few-shot generation. The results demonstrate noticeable qualitative improvements over distillation-based baselines in one-step generation and comparable improvements in few-step generation.", "questions": "- Why do you directly contrast the proposed method with distillation approaches? However, the approach aims to solve the problem more broadly by solving the factorization problem, which is more similar to methods mentioned in Minor Weaknesses. \n- In checklist question 16, you mention using an LLM for a “proof sketch”. What do you mean by “sketch”? Specifically, is the proof in the Appendix considered a sketch proof?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n- The way authors adapt the rectification procedure to achieve a few-step generation for discrete flow-based models is original and nontrivial.\n- Concise theoretical preliminaries without unnecessary diffusion or flow model training details that seamlessly introduce the factorization problem;\n- Multiple experiments cover the image and text domains;\n- Theoretical (Theorem 1) and experimental (Section 4.4) evidence of decreasing conditional TC. \n\n**Weaknesses**\n- In Section 3.1 you adopt a unified viewpoint on flow-based models. It neatly lies with the perspective of the flow matching models. However, the diffusion models are typically presented using process-reversing formalism. To avoid confusion, it would be helpful (especially having only diffusion model experiments) to add references to a few works on diffusion bridges, such as, [1] or [2]. More importantly, such miss leaves outside an interesting connection of rectified flows to IMF procedure that solves Schrödinger Bridge problem [Appendix A.2, 3] with recent works on discrete space versions of IMF [4, 5].\n- In the text experiments, the authors do not assess diversity of samples. This is crucial, since there is a known pitfall of generative perplexity: it can be artificially lowered through token repetition [6]. Particularly, reviewing samples in appendix it could be noticed that the higher ReDi iteration $k$ the more repetition appears in samples. For example, in Figure 13 line 1 “about about”, line 3 “p.m., p.m.”, line 8 “is is … what what”, etc. All these examples highlight the question: whether low generative perplexity values demonstrate overall quality improvements or are just the result of “hacking” the metric by the model.\n\n[1]\tLiu, Xingchao, Lemeng Wu, and Mao Ye. \"Let us Build Bridges: Understanding and Extending Diffusion Generative Models.\" NeurIPS 2022 Workshop on Score-Based Methods.\n\n[2]\tPeluchetti, Stefano. \"Non-denoising forward-time diffusions.\" arXiv preprint arXiv:2312.14589 (2023).\n\n[3]\tKim, Beomsu, et al. \"Simple reflow: Improved techniques for fast flow models.\"\n\n[4]\tKim, Jun Hyeong, et al. \"Discrete Diffusion Schrödinger Bridge Matching for Graph Transformation.\"\n\n[5]\tKsenofontov, Grigoriy, and Alexander Korotin. \"Categorical Schrödinger Bridge Matching.\"\n\n[6]\tWang, Yequan, et al. \"Perplexity from plm is unreliable for evaluating text quality.\" arXiv preprint arXiv:2210.05892 (2022).\n\n**Minor Weaknesses**\n- Do not mention some important references that cover the problem of the factorization [1, 2];\n- In lines 273-274 it is quite hard to understand how Conditional TC is approximated; please provide some mathematical representation for better understanding;\n- In the main theorem, I suppose there are missing assumptions, such as factorized conditional probability can’t be zero.\n\n[1]\tLiu, Anji, et al. \"Discrete Copula Diffusion.\"\n\n[2]\tXu, Minkai, et al. \"Energy-based diffusion language models for text generation.\"", "was_revised": true, "final_justification": "The method is original, and the provided experimental support is sufficient. However, there are still some questions regarding the proof of the main theorem.", "rebuttal": "We thank the reviewer for your valuable comments and positive assessment for our work. Below, we address the questions and concerns you have raised.\n> **Q1.** In Section 3.1 you adopt a unified viewpoint … such miss leaves outside an interesting connection of rectified flows to IMF procedure that solves SB problem \\[Appendix A.2, 3\\] with recent works on discrete space versions of IMF \\[4, 5\\].\n> \n**A1.** We thank the reviewer for these constructive comments and insightful suggestions. We agree that incorporating the suggested connections will significantly enrich the paper's context and clarify its position within the broader field.\n\nIn the revised manuscript we will clarify the unified viewpoint in discrete generative modeling and discuss the connection to Schrödinger Bridge Problem (SBP) and Iterative Markovian Fitting (IMF). Specifically, we will highlight the procedural similarities between our ReDi method and the IMF algorithm, noting that our iterative training and data generation steps are analogous to the Markovian and reciprocal projections of IMF. We will also briefly introduce recent works that apply the SBP/IMF framework to discrete domains as the reviewer suggested.\nWe believe these additions will provide valuable context and strengthen the paper's theoretical foundation. We thank the reviewer again for this valuable feedback, which has helped us improve our manuscript.\n> **Q2.** In the text experiments, the authors do not assess diversity of samples.This is crucial, since there is a known pitfall of generative perplexity.\n>\n**A2.** We thank the reviewer for the insightful comment. Upon the reviewer’s suggestion, we measured the unigram entropy alongside the Generative Perplexity (PPL), following the methodology in \\[1\\]. Aligned with the reviewer’s observation, we find that the entropy of the generated texts tends to decrease as we iterate more rectification steps. Upon closer inspection, we identified that this is inherited from the limited generative performance of the pre-trained model, perhaps attributed to its limited capacity (GPT-2-small backbone). Specifically, the generated distribution of the pre-trained model (PPL: 42.43 / entropy: 5.22) has a significant gap to the real data (PPL: 14.36, entropy: 5.44). When our rectification process is applied iteratively, this distributional gap can grow with each step, leading to low-diversity characteristics.\n\nWe believe that employing higher-capacity pre-trained models can easily address this problem. To demonstrate this given a limited rebuttal timeframes, we conduct an experiment on a subset of OpenWebText with 1M tokens, ensuring that our teacher model has sufficient capacity to fit the distribution. As shown in **Table A, B**, in this setting, our method achieves significant PPL improvements in the few-step regime (2-8 steps) while simultaneously preserving high entropy. This demonstrates that when the base model is faithfully modeling data distribution, our method effectively improves few-step generation quality without sacrificing diversity.\n\nWe deeply appreciate the reviewer’s comment and will update the experiments in the paper with larger pre-trained backbones and full OpenWebText dataset.\n\n**Table A.**\n|Gen.PPL|2|4|8|16|32|256|1024|\n|:-|:-|:-|:-|:-|:-|:-|:-|\n|Teacher|390.89|60.76|21.80|17.51|16.53|15.37|14.97|\n|ReDi$^1$|169.65|25.73|18.16|17.19|16.80|15.93|15.51|\n|ReDi$^2$|94.66|20.25|17.60|17.14|17.28|16.58|16.38|\n|ReDi$^3$|74.51|20.09|17.50|17.35|17.36|17.35|17.27|\n\n**Table B.**\n|Entropy|2|4|8|16|32|256|1024|\n|:-|:-|:-|:-|:-|:-|:-|:-|\n|Teacher|5.12|5.37|5.46|5.48|5.47|5.45|5.42|\n|ReDi$^1$|5.09|5.44|5.48|5.49|5.49|5.47|5.45|\n|ReDi$^2$|5.37|5.45|5.47|5.48|5.49|5.48|5.49|\n|ReDi$^3$|5.43|5.45|5.47|5.48|5.48|5.50|5.49|\n\n\\[1\\] Dieleman, Sander, et al. “Continuous diffusion for categorical data.”\n\n> **Q3**. Do not mention some important references that cover the problem of the factorization. \\[1, 2\\]; Why do you directly contrast the proposed method with distillation approaches?\n> \n**A3.** We thank the reviewer for suggesting these important references and for the insightful question regarding our choice of comparison.\n\nThe main reason that we choose the distillation approaches as baselines is because they share the same goal and settings with our method: improving the few-step generation performance of the pre-trained discrete diffusion model. While the suggested papers (\\[1, 2\\]) are highly relevant for tackling the same factorization problem, they operate under different assumptions or with different goals, which makes a direct experimental comparison challenging:\n* Discrete Copula Diffusion \\[1\\], like our work, aims to solve the factorization problem for few-step generation. However, it assumes the availability of an additional pretrained autoregressive model for a copula model. Our experimental setup does not include such a component i.e., we aim to directly improve the few-step performance of pre-trained discrete diffusion models without external models\n* EDLM \\[2\\] reduces the approximation error using an energy-based approach. While this can reduce the number of decoding steps, it is not optimized for sampling efficiency. As detailed in their algorithm, each decoding step requires multiple evaluations of the energy-based model. This fundamentally differs from our goal of a truly efficient one- or few-step generative process.\nTherefore, we believe that distillation-based models, which share our specific goal of improving a base model's sampling efficiency without these additional components or computational costs, represent a more direct and appropriate comparison group for our work.\n\nMoreover, we agree that both papers are highly relevant as they address the same core problem of factorization error in discrete diffusion. We will add a detailed discussion of these works and a comparison of their methodologies to our Related Works section to better contextualize our contribution. We thank the reviewer again for these excellent suggestions.\n\n\\[1\\] Liu, Anji, et al. \"Discrete Copula Diffusion.\"\n\\[2\\] Xu, Minkai, et al. \"Energy-based diffusion language models for text generation.\"\n\n> **Q4.** In lines 273-274 it is quite hard to understand how Conditional TC is approximated; please provide some mathematical representation for better understanding.\n> \n**A4.** We thank the reviewer for this question and are happy to clarify our estimation procedure.\n\nWe approximated the Conditional TC via Monte Carlo sampling, as the exact transition probability $p(X_1|X_0=x_0)$ is intractable. Our process involved generating 10 samples of $x_1$ for each of 5,000 initial states $x_0$. From these samples, we estimated the empirical conditional distributions ($\\hat{p}(X_1|X_0)$ and $\\hat{p}(X^i_1|X_0)$) by counting occurrences, which then allowed us to compute the Conditional TC. We will add this detailed description to the appendix.\n\n> **Q5.** In the main theorem, I suppose there are missing assumptions, such as factorized conditional probability can’t be zero.\n> \n**A5.** We thank the reviewer for this precise question regarding the assumptions for our main theorem.\nThe reviewer correctly points out that the KL-divergence $D_{KL}(p||q)$ requires that $q$ must be non-zero on the support of $p$. In the context of the Conditional Total Correlation, $p$ corresponds to the conditional joint distribution $p(X_1|X_0)$ and $q$ corresponds to the factorized product of conditional marginals, $\\prod_i p(X^i_1|X_0)$.\n\nHowever, we would like to clarify that no additional assumption is needed here. This is due to a fundamental property of probability distributions: for any given condition $X_0$, if a point $X_1$ has a non-zero probability under the conditional joint distribution (i.e., $p(X_1|X_0) > 0$), then each of its corresponding conditional marginal probabilities $p(X^i_1|X_0)$ must also be non-zero. Consequently, the factorized term $q = \\prod_i p(X^i_1|X_0)$ is guaranteed to be non-zero on the support of $p(X_1|X_0)$. Therefore, the requirement for the KL-divergence is naturally satisfied without needing an explicit assumption in our theorem. We will add a footnote to the paper to make this point clear.\n\n> **Q6.** In checklist question 16, you mention using an LLM for a “proof sketch”. What do you mean by “sketch”? Specifically, is the proof in the Appendix considered a sketch proof?\n> \n**A6.** We apologize for the ambiguity. To be clear, the proof in the Appendix is a complete and formal proof, not a sketch. Our use of the term \"proof sketch\" in the checklist was intended to describe using the LLM for background research on mathematical tools (e.g., the Data Processing Inequality). The LLM did not write or generate the derivation, and the proof is our own work."}, {"review_id": "ouiJAH31PR", "reviewer": "Reviewer_Ccyk", "summary": "This paper proposes an iterative coupling rectification method for fast sampling in Discrete Flow Models (DFMs). Conventional DFM models factorize high-dimensional distributions by assuming independence across dimensions, which prevents them from capturing inter-dimensional dependencies. As a result, these methods suffer from accumulated approximation errors when using a small number of sampling steps. To address this issue, the authors propose to iteratively rectify the coupling, effectively reducing the Total Correlation (TC). Futhermore, the authors theoretically show that the CTC decreases monotonically with each rectification step. Experimental results demonstrate that the proposed method shows better performance than existing approaches in fewer sampling steps.", "questions": "* See weaknesses\n* What is the computational cost of a single rectification step? A comparison of the overall training time with other baseline methods would help assess the efficiency of the proposed approach.", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "### Strength\n* This paper is well-written and well-organized, with a clear motivation and a logical structure that is easy to follow.\n* The theoretical analysis provides strong support for the proposed methodology. In the ablation study, it is empirically confirmed that CTC decreases on each rectification step.\n* Compared to distillation-based approaches, which often involve complex architectures and loss functions, the proposed method is relatively simple.\n\n### Weakness\n* Details on the implementation of the rectification step (Eq. 6) and the training procedure with the new coupling are insufficiently described. Based on the experimental section, it appears that a distribution is computed by multiple sampling pairs, but this process is not clearly explained.", "was_revised": true, "final_justification": "Since all the questions were raised during the rebuttal period, I will keep the initial rating.", "rebuttal": "We thank the reviewer for your valuable comments and positive assessment for our work. Below, we address the questions and concerns you have raised.\n> **Q1.** Details on the implementation of the rectification step (Eq. 6\\) and the training procedure with the new coupling are insufficiently described. Based on the experimental section, it appears that a distribution is computed by multiple sampling pairs, but this process is not clearly explained.\n> \n**A1.** We appreciate the reviewer’s comment. To obtain the new coupling by Eq.6, we first sample the initial point from the prior distribution by $X_0\\sim p(X_0)$ and apply the iterative denoising using the pre-trained model to obtain a sample from the terminal distribution by $X_1\\sim p_\\theta(X_1|X_0)$. The obtained pair $(X_0, X_1)$ defines the new coupling i.e., $(X_0, X_1)\\sim \\pi_{k+1}(X_0, X_1)$, which derives the new conditional transition probability in Eq.2. The model $p_\\theta$ is then trained with the updated transition probability with the same denoising objective, which corresponds to a rectification step.\n\nWe will add this detailed description to the preliminary section (Section 3.1) and link it to Eq. 6 to ensure clarity and completeness.\n\n> **Q2.** What is the computational cost of a single rectification step? A comparison of the overall training time with other baseline methods would help assess the efficiency of the proposed approach.\n> \n**A2.**  We appreciate the comment. While our rectification process requires sequential training, we clarify that each rectification step is much more efficient than distillation for the following reasons. First, as discussed in Section 4.4. and Figure 4(b) in the main paper, the rectification process can be trained with only a small portion of the entire training data (50K images vs. 1M full training data). As a result, it greatly reduces the cost of forwarding pre-trained models and convergence speed of each rectification step. Second, unlike distillation approaches, our method requires only the student model during training, avoiding the cost of operating two models simultaneously.\n\n**Table A** compares the training time of ReDi with the teacher model and other distillation approaches. It shows that the entire rectification process for ReDi$^3$-distill requires **only \\~3% of the time** needed to train the initial teacher model, and faster training than distillation methods up to three rectification steps. We appreciate the reviewers’ comment and will incorporate this result in the paper.\n\n**Table A.**\n||Iter.|GPU Hour / Iter.|Total Training Time|\n|:-|:-|:-|:-|\n|MaskGIT (Teacher model)|1|1800h|1800h|\n|SDTT|3|68h|204h|\n|Di4c|1|50h|50h|\n|ReDi$^2$|2|15h|30h|\n|ReDi$^3$-distill|3|15h|45h|"}]

Author rebuttal:

Meta-review: This paper introduces ReDi, a rectified discrete flow framework that reduces factorization error via progressive coupling, supported by a theoretical contraction result and empirical validation. Reviewers appreciated the originality of extending rectified flows to the discrete domain, the clarity of exposition, and strong one-step generation performance. Main concerns include unclear efficiency trade-offs compared to distillation, modest gains in multi-step settings, limited evaluation of text diversity, and theoretical assumptions in Theorem 1. The rebuttal provided additional analyses (training cost, FID–IS trade-offs, text diversity metrics, extended proof) that addressed most concerns. Reviewer 9bDF maintained that the proof remains incomplete under current assumptions, but acknowledged that the method is empirically strong and recommended acceptance with the caveat that the theorem be reframed as a proposition if a strict proof cannot be provided. Overall, the work represents a novel and technically solid contribution that advances discrete generative modeling and demonstrates practical promise.
Decision: Accept (poster)