SenseFlow: Scaling Distribution Matching for Flow-based Text-to-Image Distillation
Authors: Xingtong Ge
Xin Zhang
Tongda Xu
Yi Zhang
Xinjie Zhang
Yan Wang
Jun Zhang
Venue: ICLR 2026 Poster (2026)
Keywords: text-to-image generation
diffusion distillation
distribution matching distillation
Primary area: generative models

Abstract: The Distribution Matching Distillation (DMD) has been successfully applied to text-to-image diffusion models such as Stable Diffusion (SD) 1.5. However, vanilla DMD suffers from convergence difficulties on large-scale flow-based text-to-image models, such as SD 3.5 and FLUX. In this paper, we first analyze the issues when applying vanilla DMD on large-scale models. Then, to overcome the scalability challenge, we propose implicit distribution alignment (IDA) to constrain the divergence between the generator and the fake distribution. Furthermore, we propose intra-segment guidance (ISG) to relocate the timestep denoising importance from the teacher model. With IDA alone, DMD converges for SD 3.5; employing both IDA and ISG, DMD converges for SD 3.5 and FLUX.1 dev. Together with a scaled VFM-based discriminator, our final model, dubbed **SenseFlow**, achieves superior performance in distillation for both diffusion based text-to-image models such as SDXL, and flow-matching models such as SD 3.5 Large and FLUX.1 dev. The source code is available at https://github.com/XingtongGe/SenseFlow.

Published as a conference paper at ICLR 2026
SENSEFLOW: SCALING DISTRIBUTION MATCHING FOR FLOW-BASED TEXT-TO-IMAGE DISTILLATION
Xingtong Ge Xin Zhang', Tongda Xu Yi Zhang', Xinjie Zhang', Yan Wang', Jun Zhang!*
The Hong Kong University of Science and Technology, "Sense Time Research, VVivix Al IInttitute for Al Industry Research, Tsinghua University x xingtong. geêgmail com, eejzhang@ust.cc .hk
Sense
einls
1024x1022 samples produced by our 4-step generator distilled from FLUX.I-dev. Please zoom for details.
Figure I:
ABSTRACT
The Distribution Matching Distillation (DMD) has been successfully applied to text-to-image diffusion models such as Stable Diffusion (SD) 1.5. However, vanilla DMD suffers from convergence difficulties on large-scale flow-based text- to-image models, such as SD 3.5 and FLUX. In this paper, we first analyze the issues when applying vanilla DMD on large-scale models. Then, to overcome the scalability challenge,
propose implicit distribution alignment (IDA) constrain the divergence between the generator and the fake distribution. Fur- thermore, we propose intra-segment guidance (ISG) to relocate the timestep denoising importance from the teacher model. With IDA alone, DMD con- verges for SD 3.5; employing both IDA and ISG, DMD converges for 3.5 and FLUX.1 dev. Together with scaled VFM-based discriminator, model, dubbed SenseFlow, achieves superior performance in distillation for both diffusion based text-to-image models such as SDXL,, and flow-matching mod- els such as SD 3.5 Large and FLUX.1 dev. The source code is available at https//ggghbbboommm.iionneeeSeee
final
"Corresponding Author

---

Published as a conference paper at ICLR 2026
INTRODUCTION
Significant advancements have been made on diffusion models (Ho et al. 2020; Rombach et al,
2022 Podell et al. 2024 Esser et al., 2024; Labs 2024) for texi-lo-image generation over recent years. However, these models typically require muluple denoising steps to generate high-quality images. As models continue scale up in terms of the parameter size, the computational cost and inference time for image generation increase substantially, making the process slower and more resource-intensive. To address this issue, various diffusion distillation methods have been developed to distill diffusion model into a few-step generator, including consistency models (Song et al. 2023 Luo et al., 2023a Wang et al. 2024a), progressive distillation (Salimans & Ho 2022 Ren
et al. 2024). adversarial distillation Waagg et al., 2023a Sauer et al. 2024b; [Chadebec et
and score distillation (iin et al. 2024b at Luo et al. (023b). Currently, the Distribution Matching
Distillation (DMD) series (Yin al.] (024a) have demonstrated superior results in distilling standard
diffusion models such as SD 1.5 (Rombach et al. 2022) and SDXL (Podell et al.) 2024).
However, few of these methods have successfully demonstrated effective distillation performance across a broader range of models, particularly in flow-based diffusion models with larger parameter sizes, such as SD3.5 Large (8B) (Esser et al. 2024) and FLUX.1 dev (12B) (Labs] 2024). As models increase in architecture complexity, parameter size, and training complexity, it becomes significantly more challenging to distill these models into efficient few-step generators (e.g.. a 4tep generator).
In this paper, we introduce SenseFlow, which selects the framework of DMD2 Yin et al. 2024a) as a touchstone, and scales up for larger flow-based rext-to-image models, including SD3.5 Large and FLUX.1 dev. Specifically, vanilla DMD2 has difficulty in converging and faces significant training instability on large models, even with the time-consuming two time-scale update rule (TTUR) (Heusel et al.] (017)) applied. Viewing DMD as a min-max game, the inner best response requires the fake distribution model to track and predict the data distribution determined by gener- ator samples effectively, which is brittle and expensive to realize. We therefore introduce implicit distribution alignment (IDA): a lightweight proximal update applied after each generator step that nudges the fake model toward the generator, maintaining ()()2) (x an E-best response. This simple alignment markedly improves stability and enables convergence on large backbones.
Further, DMD2 and most existing diffusion distillation methods still use uniformly of handcrafted sampled timesteps for training and inference. However, due to the complex strategies employed dur- ing training of teacher diffusion models, different timesteps exert varying denoising effects through- out the entire process, which is also discussed in RayFlow (Shao et al. 2025). To avoid the in- efficiency of naive timestep sampling strategy in distillation, we propose to relocate the teacher's timestep-wiss denoising importance into a small set of selected coarse timesteps. For each coarse timestep Ts, we construct an intra-segment guidance (ISG) by sampling an intermediate timestep bmid E (Ti-1, T.), and construct guidance trajectory: the teacher denoises from to lmid, then the generator continues from bmid to Ti-1- We then guide the generator to align its direct prediction from To to T--1 with this trajectory. This guidance mechanism effectively aggregates the teacher's ' fine- grained behavior within each segment, improving the generator's 's ability to approximate complex transitions across fixed sparse timesteps.
For further enhancement, we incorporate a more general and powerful discriminator built upon vision foundation models (e.g., DINOv2 (Oquab et al. 2023), CLIP (Radford et al.. (02II). which operates in the image domain and can provide stronger semantic guidance. The use of pretrained vision backbones introduces rich semantic priors, enabling the discriminator better capture image- level quality and fine-grained structures. By aggregating timestep-aware adversarial signals, this design yields stable and efficient training with superior visual qualities.
To summarize, we dive into the distribution matching distillation (DMD) and scale it up for a wide range of large-size flow-based text-to-image models. Our contributions are as follows:
We discover that vanilla DMD2 suffers from the convergence issue large-scale text-to- image models, even with TTUR applied. To tackle this challenge, we propose implicit dis- tribution alignment to constrain the divergence between the generator and the fake model.
To mitigate the problem of suboptimal sampling in DMD2, we propose ietra-segment guid- ance to relocate the teacher's timestep-wise denoising importance, improving the genera- tor's ability to approximate complex transitions across sparse timesteps.
2

---

Published as a conference paper at ICLR 2026
By incorporating more powerful discriminator built upon vision foundation models with timestep- aware adversarial signals, we achieve stable training with superior performance.
Experimental results show that our final model, dubbed SenseFlow, achieves state-of-the art performance in distilling large-scale now-matching models (e.g., SD 3.5, FLUX.1 dev) and diffusion-based models (e.g., SDXL..
PRELIMINARIES
2.1 DIFFUSION MODELS
Diffusion models are family of generative models, with the forward process perturbing the data Xa ~ ((x)) to Gaussian noise p(Xr) .N(0, 7) with a series distributions p(X,) defined by forward stochastic differential equation (SDE):
dX, ((X,,t)dt + g()d E [0,
where f(X,,t) is drifting parameter, g(t) is diffusion parameter and B, is standard Brownian mo- tion. The diffusion model learns the score function s(,,, Vx, logp(X, (X)) using neural network. And the sampling of diffusion process is to solve the probability flow ordinary differential equation:
dX, (f(X..t t) 2 ( tt) XT ~ (0.
(2)
The two widely adopted diffusion models
toaete namely denoising diffusion probabilistic model (DDPM) and flow matching optimal transport (FM-OT), fit in above framework by setting f(X,, Xe: g(t) = / and f(x;: 1) = I- Xx, gt) = i-t respectively, where 3
+
hyper-parameter of DDPM. The forward SDE of DDPM and FM-OT can be directly solved:
DDPM: ((, Xo) A(ee + fo A.ds Xo. (1 $ A.ds))).
(3)
FM-OT: (X,, Xo) = (((1
- f)Xo, 11).
(4)
However, the backward equation in Eq. 2 is intractable as s(Xx,,) t) neural network. Usually we need cinesccnsuning multi-step solvers. In this paper, we focus on distilling the solution of back- ward equations into another neural network.
2.2 DISTRIBUTION MATCHING DISTILLATION
From now assume pre-trained diffusion model is available, with learned score function s, (X,,)) t) and distribution pr (Xt). The Distribution Matching Distillation (DMD) (Yi et al.
2024b distills the diffusion model by technique named score distillation (Poole et al.) 2022).
More specifically, DMD learns the generator distribution Pg(X, to match the diffusion distribution
(X,):
min DKL (XL.l = -..,,. [log (X,) - log pr (X.))-
(5)
Directly distillation from above target produces suboptimal results. Therefore, DMD introduces
an intermediate fake distribution py (Xe,, t), and optimizes the generator distribution Py and fake distribution py in an interleaved way:
Generator: min [0, [log py (X,) logp)((.))]
Po
Fake: max [,_ [log po (X.))
(6)
P/
In practice, the fake distribution is parameterized as the score function (X,,t) = (Xg
On the other hand, the generator is parameterized with clean image generating network C (0, J) and forward diffusion process q (X,, Xp), such that Py (X,) E (g( Go((s) (c)) To
this end, the DMD updates are achieved by gradient descent and score matching (Vincent 2011
(X,, ax,
Generator: VoL,
Fake: oL, --VVal V.B... Gale))lls Vxx XX)
(7)
3

---

Published as a conference paper at ICLR 2026
3 METHOD: SCALING DISTRIBUTION MATCHING FOR GENERAL DISTILLATION
Implict digpoorr Stee g
Promp"
Prompt
CLIP
h8
La
COOD
Intro Negrore Gussience
St
Sr
DENOVa
ho.
Cec
Figure 2: Left: The generator G receives a text prompt and Ir, to produce one-step output Iga which diffused 2t and processed by 96 and s, for computing DMD gradient. ISG guides G using an sampled intermediate fmid. and IDA aligns G with 84 after generator update. The overall training pipeline shown in Algorithm[] I Right: The discriminator extracts semantic features from generated and real images using CLIP and DINOv2, which are processed by head blocks he, to predict real/fake logits for adversarial training. Trainable modules are shown in pink, while frozen (pretrained) ones are shown in grey.
3.1 BOTTLENECKS IN VANILLA DMD: STABILITY, SAMPLING, AND NAIVE DISCRIMINATOR
While Distribution Matching Distillation (DMD) has shown promising results in aligning genera- tive distributions, its vanilla formulation exhibits several fundamental limitations when applied to large-scale models. First, scalability remains a challenge-tth two time-scale update rule (TTUR), effective SD 1.5 (0.8B) and SDXL (2.6B), fails converge stably when scaled larger models such as SD 3.5 Large (8B) or FLUX (12B). Second, sampling efficiency is limited as the generator does not incorporate the varying importance of timesteps the denoising trajectory, which slows convergence and reduces expressiveness. Third, the discriminator lacks generality, with relatively naive design that struggles to adapt across diverse model scales and architectures. These issues motivate us to propose architectural and algorithmic improvements in this work.
3.2 IMPLICIT DISTRIBUTION ALIGNMENT (IDA) FOR FLOW-BASED MODELS
Recall that the DMD can be viewed as a min-max optimization:
min max V(0, +) A [. [og (o (X.) log (X.)]-
(8)
It obvious that the inner best response attained at (X,) = (X.). Furthermore, following Proposition 2 of GAN (Goodfellow et al.) 2014), if the inner best response is achieved at every round of generator optimization, then generator converges.
However, the inner best response is brittle. DMD2 uses two time-scale update rule (TTUR) (Heusel al. 2017), which increases the update frequency of fake model. However, at large-scale models such as SD 3.5 Large (8B), simply increasing TTUR ratio is expensive and can still oscillate, as shown Fig. 3 To tackle the difficulty achieve inner best response, a proximal alignment step after each generator update, called Implicit Distribution Alignment (IDA):
The IID Score Across Triming llours
introduce
100
XØ (1 A)e,
(9)
15
20
30
where (0, and close to We claim that IDA maintains py (y(X), (X,) py (X,). and thus help DMD con- verges. More formally, we have the following: Proposition 3.1. Under mild assumptions (Assump-
Training
Figure 3: "Training Hours-FID" curves on COCO-5K dataset. When distilling the 8B SD 3.5 Large, IDA improves training sta- bility across TTUR ratios.
tions A.7 and (..). IDA maintains an e-best inner re- sponse. More specifically, after is round of min
4

---

Published as a conference paper at ICLR 2026
5D-35
FLUX.I
8:000
22.000
6.006
0.005
Neralioal
0304
0.000
0.000
0.
02
0.4
0.5
6.5
1.0
limedie..
limsssp:
Figure 4: Left: Normalized reconstruction loss E(I) over timesteps in [0, 1]. Right: Illustration of the Intra-Segment Guidance.
optimization in Eq. g we have
E. (x
& e, ie., p)(()) se (X.) (8-best response).
(10)
In practice, this strategy ensures that the fake distribution remains closely aligned with the genera- tor's distributional trajectory. We observe that combining IDA with even relatively small TTUR (e.g.. 5:1) leads to significantly more stable convergence. An example of this effect is shown in Fig. 3 where compare FID curves under different TTUR settings with and without IDA. As the figure illustrates, IDA consistently reduces FID variance and improves overall performance.
Intuitively, the generator is trained on small set of anchor timesteps (e.g., four fixed t's), while the fake model is trained on a much denser grid. The IDA transfers the generator's 's progress on those anchor f's to the fake model, improving its predictions at those l's. On the other hand, on SD) 3.5 Medium generator, we also observe that DMD training produces generators whose samples are consistent across 4/8/16 Euler steps, indicating that Ge remains close to a smooth continuous-fie integrator. This observation supports using generator parameters to softly anchor the fake model. Proofs, discussion, and visualization results are provided Appendix ŞA
3.3 INTRA-SEGMENT GUIDANCE: RELOCATING TIMESTEP IMPORTANCE
The distillation performance of vanilla DMD2 is constrained by supervision at a few handcrafted, coarse timesteps (e.g., E {249, 499, (99)). This design has two drawbacks: (i) the gener- ator receives signal over the rest of the trajectory, hurting generalization; and (ii) the utility of
each supervised timestep highly position-dependent-aae neighhoring timesteps can differ markedly in prediction error. To quantify this local reliability, visualize the normalized one-step recon- struction loss
E(t) == Ezg, (oi((. [l)o( - xoll2]
(11)
where zo is sampled from the teacher (SD 3.5 FLUX. I-dev) and 2, is obtained by the forward diffusion process in Eq. As shown in Fig. (Left), {(t) is not monotonic and exhibits pronounced local oscillations espeeially for [0.8, 1.0). Hence, treating nearby timesteps as equally informative can anchor training to suboptimal points.
To address this, introduce Intra-Segment Guidance (ISG), which relocates the teacher's denoising importance from within each segment (Ti-1, T3] to its supervised anchor. For every coarse timestep T, sample intermediate timestep émid E (T,-1, T,). The teacher denoises from 7, to mid to produce Traas The generator then continues from biid to Ti-1, yielding the target Furr In paral- lel, the generator directly denoises from т, 7--1 to produce Tr, We optimize an E2 loss that backpropagates only through the generator path:
( Ee, tma _ stop (rad 11
(12)
where stop-grad(-) prevents gradients from flowing through the target. By letting each anchor absorb information from its surrounding segment, ISG makes the anchors more representative of local denoising behavior, improving sample quality and training stability.
3.4 GENERAL AND POWERFUL DISCRIMINATOR BUILT UPON VISION FOUNDATION MODELS
As shown in Fig. 2 the discriminator D consists a frozen Vision Foundation Model (VFM) back- bone fVTM with trainable heads h, enabling a general yet stable adversarial signal across timesteps.
5

---

Published as a conference paper at ICLR 2026
Algorithm SenseFlow Training Algorithm
Require: pretrained teacher model Preah, real dataset Dreat, generator update frequency coarse timestep set S (To, TI T2; T3}
Ensure: trained few-step generator G
1: G t copy Weights((Heal)
> Initialize generator Initialize fake distribution network > Initialize VFM-based discriminator
2: Maake coopyWeights(HHee
3: + innttaaiieeDiccinnnaaooo
4: for iteration
max_iters do
5: 2 ~~ ~~,
6: Sample т; from $
Pick timestep for current iteration With 50% probability, use backward simulation
7: Sample Freal ~ Deal
8:
if random() 0.5 then
9:
+ multiStepSampling(e; - T())
else
11:
end Toz < forwardDiffusion((rran) 74)
12:
13:
+ G(
14:
if iteration mod f = then
15:
LOMD distributionMatchinngg (real, M 2)
(Prea,,
16:
La - - |B [x))
D Eq. 15
Intra-segment guidance (ISG)
17:
midd U(Ts, T3-1)
Fmid (Ta, Ts tmid)
19:
Tur + G (Imid, tmid + T(-1)
1- + G Ti - Ti-1)
21:
LISa + MSE(IT.-T sttoparr..s
22:
La + LDMD + >a LG + >usa LISG
Final loss function for generator
23:
G + update G, La)
Implicit distribution alignment (IDA), as Eq
24:
flke IDA(G, AIDA)
25:
end
Update fake score network (4ake Using logit-normal density, as in (Esser et al. 2024)
26:
LogitNorralSSampinng(... 1)
Ie rrwwardDiffusion(ttogrrddd Cdnoosee + denoisingLooss (T,, t), (xo)grad()))
28:
Mlke
update( Mtake Leenosse)
b Update discriminator D Eq 14
30:
CD - E mmax (0, (# + E[max(0, + D(x)]
31:
D - update( D) LD)
end for
Given an input image and its text prompt, the VFM backbone extracts multi-level semantic fea- tures (v) In addition, we encode the text with CLIP, c fcuP (text), and use VFM features from real images reference, r = real, to inject text-image alignment and realism priors. The discriminator thus
D(a, e, ((vrn(x), e, r).
(13)
These signals allow D judge both realism and semantic consistency. Hinge loss for the discriminator. We adopt the standard hinge loss:
La Ex [max(0. 0, - D(X, c, r)) +E Ex max((, 1 1+ D(Lo, cr))]
(14)
where Piaa the empirical distribution of real images and Py is the generator distribution.
devesarial objective for the generator. During the training process of generator, predictions of
X, at large timesteps can be less reliable. To avoid overpowering the DMD signal under high noise, introduce weighting mechanism. Specifically, scale the adversarial term by the signal

---

Published as a conference paper at ICLR 2026
power of the current timestep. Let the forward process be 2t ag ose (see Sec. ((t) = (1 - o1)², which decreases as noise increases, and optimize
2
We define
Cg = - Exo [ = - E E D(Xo, c.)]
(15)
This weighting emphasizes the DMD gradient at noisy, high-l steps while leveraging GAN feed- back more strongly at cleaner, low-noise steps, improving stability and overall quality in practice. Implementation details of the discriminator are provided in Appendix B..
Overall training pipeline. Algorithm| Saaammmssmmmmmi summarizes the complete SenseFlow optimization loop. At each iteration, we sample coarse timestep т; S and construct 2,, either by backward simulation from noise or by forward diffusion from real data. We update the generator every f iterations (TTUR-style) with the combined objective of DMD, the VFM-based adversarial term, and ISG. After each generator update, we apply IDA to softly anchor the fake model toward the updated generator. Finally, update the fake model and the discriminator with their respective losses.
4 EXPERIMENTAL RESULTS
4.1 EXPERIMENTAL SETUP
Datasets and Benchmarks. Following DMD2 (Yin et al. (022a), our experiments are conducted using filtered set of the LAION-5B [Schuhmann et al. (022) dataset, which provides high-quality image-text pairs for training. We select images with minimum aesthetic score (aes score) of 5.0 and shorter dimension of least 1024 pixels, ensuring the dataset comprises visually appealing, high-resolution images suitable for our model's requirements.
For evaluation, construct validation using the COCO 2017 (Lin al. 2014) validation set, which contains 5,000 images. Each image in this set is paired with the text annotation that yields the highest CLIP Score (ViT-B/32), thus forming a robust text-image validation set. To assess compo- sitional generalization, we further evaluate on GenEval (Ghosh et al. 2023) 2023)-whise wwhich programmati- cally checks object presence, attributes, relations, and counting- and T21- 22-CommBench (Huang al.) 2023 wwhich covers attribute binding, inter-object relations, and complex multi-object composi- tions; we follow the official protocols and report both overall and per-category scores.
Text-to-Image Models. We conduct extensive experiments on three representative large-scale text- to-image models: FLUX.1 dev (12B) (Labs) 2024). Stable Diffusion 3.5 Large (8B) (Esser et al.] 2024), and SDXL (2.6B) (Podell et al., 2024), which span different model sizes and generative paradigms. Results demonstrate the generality and effectiveness of our method across both flow- based and conventional diffusion architectures.
Evaluation Metrics. Following (Wang et al., 2024a Lin et al.) 2024 Yin et al. 2024a), we re- port FID and Patch FID of all baselines and the generated images of original teacher models to assess distillation performance and high-resolution details, dubbed FID-T and Patch FID-T. For se- mantic faithfulness, image quality and human preference,
additionally report CLIP Score (ViT- B/32) (Radford et al. 2021). HPS v2 (Wu et al. 2023) (a human-preference predictor), ImageRe- ward (uu al. (0033) (a learned reward approximating human judgments), and PickScore (Kirstain) et al. 2023) (trained on pairwise human choices), which complement FID) by focusing on perceived quality and semantic alignment.
4.2 TEXT TO IMAGE GENERATION
Comparison Baselines. For SDXL distillation, we compare against LCM (Luo et al.) 2023a
PCM (Wang et al. 2024a), Flash Diffusion (Chadehec et al.). SDXI--Lightinn (Lin et al.
2024).
Hyper-SD (Ren et al. 2024), and DMD2 (Yin et al. 2024a). For SD 3.5 Large, use SD 3.5
Large Turbo as the best baseline (Sauer et al. 2024b). ). For FLUX. I dev, we compare with Hyper-
FLUX (Ren et al. 2024). FLUX.1 schnell (Labs] 2024), and Turo--urbo Team 2024 All baselines are evaluated under their official 4-step configurations.
Quantitative Comparison. We evaluate 4-step models COCO-SK and GenEval (Table I and report T2I- 22--ComBeench results (Table For flow-matching models (SD 3.5 Large and FLUX.1 dev) include both stochastic and deterministic solvers, denoted as "Ours" and "Ours (Fuler)".
7

---

Published as a conference paper at ICLR 2026
Table Quantitative Results on COCO-SK and GenEval Benchmarks. Bold/Underrine: best/second
best in distilling the same teacher. Our method achieves superior performance on 4-step generation.
Method
#NFEL Patch FID-T,
CLIPT HPSv21
Pickt
IR
GenEval*
Stable Diffusion XL Comparison
SDXL (Podell al. 2024
80
0.3293
0.2930
22.67
0.8719
0.5461
LCM-SDXL Luo ct al.
2022a
2023a
30.63
0.3230
0.2824
22.22
0.5693
0,5036
PCM-SDXL Wang et al.
2024a
4
17.77
0.3242
0.2920
22.54
0.6926
0.4944
Flash-SDXL Chadebec al.
4
23.24
0.3216
0.2830
22.17
0,4295
0,4715
SDXL.-Lggtming (Lin al.
2024
4
16.57
0.3214
0.2931
22.80
0.7799
0.5332
Hype (Ren al. 2024
17.49
0.3254
0,3000
22.98
0.9777
0.5398
DMD2-SDXL Yin ct al
2024a)
18.72
0.3277
0,2963
22.98
0,9324
0.5779
Ours-SDXL
4
21.01
0.3248
0.3010
23.17
0.9951
0.5784
Stable Diffusion 3,5 Large Comparison
SD 3.5 (Esser et al. 2024)
0.3310
0.2993
22.98
1.1629
0.7140
SD 3.5 Turbo (Sauer et al. Ours-SD 3.5
2024b
22.88
0.3262
0,2909
22.89
1.0116
0.6877
4
17.48
0.3286
0.3016
23.01
1.1713
0.6955
Ours-SD 3.5 (Euler)
4
20.26
0.3287
0.3008
22.90
1.2062
0.7098
FLUX Comparison
FLUX.1 dev Labs 2024)
50
0.3202
0.3000
23.18
1.1170
0.6699
25
0.3207
0.2986
23.14
1.1063
0.6733
I-UX.I-schnell (Labs
2024)
4
0.3264
0.2962
22.77
1.0755
0.6807
Hyper-FLUUX (Ren al. 2024)
23.47
0.3238
0.2963
23.09
1.0983
0.6193
FLUX-TTubo-AIpha (Team
2024
24.52
0.3218
0.2907
22.89
1.0106
0.4724
Ours- FLU-
4
19.60
0.3167
0.2997
23.13
1.0921
0,6471
Ours-FLUX (Euler)
4
20.29
0.3171
0.3008
23.26
1.1424
0.6420
On COCO-SK & GenEval, across all teachers, our 4-step distillation performs strongly on mod- ern, human-correlated metries. On SD 3.5, "Ours-SD 3.5" and "Ours-SD 3.5 (Euler)" achieve the best and second-bes scores on all metrics, even surpassing the teacher model in HPSv2, PickScore, and ImageReward. The Euler variant achieves the highest GenEval (0.7098), approaching the 80- NFE teacher (0.7140), On SDXL, our distilled model ranks first on most metrics, including HPSv2, PickScore, ImageReward and GenEval, with CLIP close to prior art and competitive Patch FID-T. On FLUXX.1 dev, our models again deliver best and second- best performance across five of six met- rics. The Euler variant further surpasses the teacher model in HPSv2, PickScore, and ImageReward. As for T2I-CompBench, our SD 3.5 (Euler) model is best in five of six categories and second on "Spatial" category, establishing overall state-of-the -hee--a--e-rr performance. For SDXL, our model is best second-best in all six categories, giving the strongest overall SDXL distillation this benchmark. For FLUX, the Euler variant best or second three of six categories, achieving the overall second best performance. Detailed results of GenEval and T21-CompBench are shown in Appendix B.4
Overall results indicate that our method preserves fidelity while improving semantic alignment and compositional correctness, as reflected by GenEval and T21-CompBench across diverse models.
Qualitative Comparison. Fig 5 presents qualitative comparisons across a set of prompts. Our method generates images with sharper details, better limb structure, and more coherent lighting dynamics, compared to teacher models and baselines. Notably, "Ours-SD3.5" and "Ours-FLUX" produce more faithful and photorealistic generations under challenging prompts involving fine tex- tures, human faces, and scene lighting. Additional visualization results are provided in the appendix.
4.3 ABLATION STUDIES
Effectiveness of Implicit Distribution Alignment. To assess the effectiveness of our proposed IDA, we conduct experiments SD 3.5 Large with various TTUR ratios. As shown in Fig. 3 we compare FID curves across different settings, both with and without IDA. Without IDA, the curves corresponding to "TTUR(5)". "TTUR(10)". and "TTUR(20)" exhibit severe oscillations, indicating unstable training dynamics and unreliable optimization of the fake distribution- even at high ratio of 20:1. This instability leads to inaccurate DMD gradients and poor convergence. In contrast, the
8

---

Published as a conference paper at ICLR 2026
wise old man with white heard in the enchanted and magical forest
high-rcsolution photo of orange Porsche under sunshine
onl perches quietly tristed branch deep within ancient forest. sharp yellom keen and watchful
closc-up
narrallive nostalgia
Qinen aa
iinls
SD3.5
SD3.5 Turbo
Ouss-SD3.5
FLUX.1 schnell
Ouis-FLUX
Qualitative comparisons challenging prompts across methods. Our method shows superior fidelity, especially in rendering human faces, scene composition, and fine-grained textures.
Figure 5:
settings that incorporate IDA (i.e., "IDA+TTUR(5)" and "IDA+TTUR(1))" demonstrate signifi- cantly smoother and more stable FID reductions, highlighting IDA's ability to stabilize training and improve convergence, even a relatively small TTUR ratio (5:1).
In addition to the FID analysis, we report quantitative comparisons in Tab. between "w/o ISG" and "w/o ISG, w/o IDA" using five metries: FID-T, HPSv2, PickScore, ImageReward, and AESv2. Across all metrics, adding IDA leads to consistent improvements, further confirming that IDA plays a key role in enhancing training stability and distillation quality. More observation results in Ap- pendix A.5 also serve evidences for IDA as the proposition of s-best inner response.
Table 2: Ablation Study Results of IDA, ISG,
and VFM Discriminator.
Method
FID-T, HPSv2 Pickt IRt
AESv21
Stable Diffusion 3.5 Large
Ours
13.38
0.3015
23.03 1.1713
5,482
w/o ISG
17.00
0.2971
22.75 1.0186
5.453
w/o ISG, w/o IDA
43.84
0.2555
20.60 0.3828
5.102
Stable Diffusion XL
DMD2-SDXL
15.04
0.2964
22.98 0.9324
5.530
Figure
The ISG improves training consis- tency, especially in early stage of training.
DMD2 VFM
18.55
0.2995
23.00 0.9744 5.625
Effectiveness of Intra-Segment Guidance. To evaluate the effectiveness of the Intra-Segment Guidance (ISG) module during distillation, we conduct ablation study Stable Diffusion 3.5 Large. As shown in Tab. 2 we compare our model with and without ISG (denoted as "Ours" and "w/o ISG", respectively) the COCO-SK dataset. The results indicate that incorporating ISG leads

---

Published as a conference paper at ICLR 2026
to significant improvements across all aspects, including image quality, text-image alignment, and human preference quality.
Fig. 6 shows a qualitative snapshot at 3k training iterations (generator updated for 300 steps with a 10:1 TTUR). With ISG, generations are visibly more consistent and semantically accurate, while the model without ISG exhibits color shifts and degraded details. This supports our interpretation of ISG as a segment-aware supervision that stabilizes early optimization and accelerates convergence.
Training-time overhead. We evaluate the running time of our introduced IDA and ISG. In our TTUR setting (f=5), ISG and IDA run only on generator steps, amortizing their cost. Across three backbones, enabling ISG increases per-iteration time by +4.44% (SDXL), +3.23% (FLUX), and +6.16% (SD 3.5). Adding IDA on top of an ISG-free baseline adds only +3.97% % (FLUX) and +0.57% (SD 3.5). These overheads are modest compared the gains in convergence stability and sample quality; full timing statistics are Appendix. B.3 Tab.
VFM-based Discriminator, We ablate the discriminator the SDXL backhone by replacing the diffusion-baase discriminator in DMD2-SDXL with our VFM-based design ("DMD2 w VFM"), As reported in Tab. [ the VFM discriminator improves human-centrie metries and aesthetics-HPSv2. PickScore. ImageReward, and AESv2-will showing trade-off on FID-T. These results suggest that leveraging VFM features provides stronger semantic and stylistic priors for adversarial feed- back, demonstrating better generalization and alignment with human preferences.
FID-T vs. human-preference trade-off. Adding the VFM discriminator raises FID-T, as shown in Tab. [ However, We suggest that cuman--reeeeence metrics (HPSv2, PickScore, ett) mainly capture perceived quality and semantic faithfulness, whereas FID-T is sensitive to both quality and diversity relative to the teacher. The VFM discriminator imposes a semantic prior, nudging the generator toward VFM-preferred modes: this improves alignment with human-preferred semantics but may reduce sample variance of less-favored regions, hence the modest rise in FID-T. T. In few- step distillation context, we prioritize high-quality, semantically meaningful modes over exhaustive distribution coverage; We view this diversyy trade-off acceptable in practical use.
5 RELATED WORK
Diffusion Distillation methods mainly fall into two categories: trajectory-based and distribution-
based approaches. Trajectory-based methods, such as Direct Distillation (Luhman & Luhman [02T)
and Progressive Distillation (Salimans & Ho 2022, Ren al. 2024 Lin et al., [2024] Chadebec
et al.). learn to replicate the denoising trajectory, while Consistency Models (Song et al.) 2023]
Luo et al., 2023a Kim et al. 2024 Wang et al. [024aa] Lu Song 2024, Chen al.] 2025) en- force self-consistency across steps. Distribution-bast
methods aim to match the generative distri- bution, including GAN-based distillation (Wang et al. 2023a 2024b Luo et al.) 2024) and VSD variants (Wang et al. 2023b Yin et al.) 2024b a). ADD (Sauer et al., (024b) and LADD (Sauer et al. 2024a) explored distilling diffusion models using adversarial training with pretrained feature extractors. RayFlow (Shao et al.] (025) explored sampling important timesteps for better distilla- tion. Among these, DMD2 (Yin et al. (022a)) has shown strong results on standard diffusion models (e.g., SDXL.), (D.g., but its stability degrades on large-scale models. Our work builds upon DMD2 and ad- dresses these limitations by introducing SenseFlow, which scales distribution matching distillation SD 3.5 and FLUX.1 dev through improved alignment and regularization strategies.
6 DISCUSSION & CONCLUSION
We scale up distribution- distribution-matching distillation for large flow-based models via implicit distribution alignment and intra-segment guidance; together with a VFM-based discriminator, these yield our SenseFlow, which achieves stable and effective few-step generation across both diffusion and flow- matching backbones. Across three SDXLL SD 3.5 Large, and FLUX. dev-Senselloog attains superior overall 4-step results on modern numan--refereee and compositional benchmarks. Looking ahead, we aim to push to more aggressive sampling regimes (2-step and 1-step) and to study alternative vision backbones for the discriminator/guidann modules (Oquab et al. 2003 Ravi
et al. 2025 Ranzinger al. 2024 He et al. 2022).
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENT
This work was supported by the Hong Kong Research Grants Council under the Areas of Ex- cellence scheme grant AoE/E-601/22-R and NSFC/RGC Collaborative Research Scheme grant CRS.HKUST603//2.
REFERENCES
Clement Chadebec, Onur Tasar, Eyal Benaroche, and Benjamin Aubin. Flash diffusion: Accelerat- ing any conditional diffusion model for few steps image generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 15686-15695.
Junsong Chen, Shuchen Xue, Yuyang Zhao, Jincheng Yu, Sayak Paul, Junyu Chen, Han Cai, Enze Xie, and Song Han. Sana-sprint: One-step diffusion with continuous-time consistency distillation. arXiv preprint arXiv:2503.0966 2025.
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference on machine learning. 2024.
Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. Geneval: An object-focused framework for evaluating text-to-image alignment. Advances Neural Information Processing Systems, 36: 2023.
lan J Goodfellow,
Jean Pouget-Abrr Abadde,, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross B. Girshick. Masked autoencoders are scalable vision learners. In CVPR, pp. 15979-15988. IEEE, 2022.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge local nash equilibrium. In NIPS, pp- 6626-66377 2017.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 1166840-6511. 2020.
Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. T2i-compbench: A compre- hensive benchmark for open-world compositional text-to-image generation. Advances in Neural Information Processing Systems, 36:78723-78747, 2023.
Dongjun Kim, Chich-Hsin Lai, Wei-Hsiang Liao, Naoki Murata, Yuhta Takida, Toshimitsu Uesaka, Yutong He, Yuki Mitsufuji, and Stefano Ermon. Consistency trajectory models: Learning proba- bility flow ODE trajectory of diffusion. In ICLR. OpenReview.net, 2024.
Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. Pick- a-pic: An open dataset of user preferences for text-to-image generation. NeurlPS, 2023.
Black Forest Labs. Flux.
htt.:///cooollc---ooooees---aaao forest- 2024.
Shanchuan Lin, Anran Wang, and Xiao Yang. Sdxl-lightning: Progressive adversarial diffusion
distillation. arXiv preprini arXiv:2402. 13929. 2024.
Tsung-Y Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European Conference on Computer Vision, pp. 740-755. Springer, 2014.
I Loshchilov.. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05100 2017.
Cheng Lu and Yang Song. Simplifying, stabilizing and scaling continuous-timg consistency models. arXiv preprint ar 1081, 2024.
11

---

Published as a conference paper at ICLR 2026
Eric Luhman and Troy Luhman. Knowledge distillation in iterative generative models for improved sampling speed. arXiv preprint arXiv:21010025: 2021.
Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao. Latent consistency models: Synthe- sizing heshuuooolttion images with few-step inference. arXiv preprint arXiv:2310.04375. 2023a.
Weijian Luo, Tianyang Hu, Shifeng Zhang, Jiacheng Sun, Zhenguo Li, and Zhihua Zhang. Diff- instruct: A universal approach for transferring knowledge from pre-trained diffusion models. In NewrIPS, 2023b.
Yihong Luo, Xiaolong Chen, Xinghua Qu, Tianyang Hu, and Jing Tang. You only sample once: Taming one-step text-to-image synthesis by self-cooperative diffusion gans. arXiv preprint arXiv:2403.12931, 2024.
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniee, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193 20233
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. SDXL: improving latent diffusion models for high-resolution image synthesis. In ICLR. OpenReview.net, 2024.
Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209. 14988 2022.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp- 7748-87653 PmLR, 2021.
Mike Ranzinger, Greg Heinrich, Jan Kautz, and Pavlo Molchanov. AM-RADIO: agglomerative vision foundation model reduce all domains into one. In CVPR, pp. 12490-12500. IEEE, 2024.
Nikhila Ravi, Valentin Gabeur, Suan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloé Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Va- sudev Alwala, Nicolas Carion, Chao-Yuan Wu, Ross B. Girshick, Piotr Dollár, and Christoph Feichtenhofer. SAM 2: Segment anything in images and videos. In ICLR. OpenReview.net, 2025.
Yuxi Ren, Xin Xia, Yanzuo Lu, Jiacheng Zhang, Jie Wu, Pan Xie, Xing Wang, and Xuefeng Xiao. Hyper-sd: Trajectory segmented consistency model for efficient image synthesis. arXiv preprint 1rXiv:2404.13686 2024.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björ Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Tim Salimans and Jonathan Ho.
Progressive distillation for fast sampling of diffusion models. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. URL htt.https:////tt https:://ppeeeeiee..... wet.:wwwwwwwweeeeee.eoo..e..ee forum?? ?id- TIdIXIpzhoI
Axel Sauer, Frederic Boesel, Tim Dockhorn, Andreas Blattmann, Patrick Esser, and Robin Rom- bach. Fast high-resolution image synthesis with latent adversarial diffusion distillation. In SIG- GRAPH Asia 2024 Conference Pupers, pp. 2024a.
2024a.
Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion dis- tillation. In European Conference on Computer Vision, pp. 87-103. Springer, 2024b.
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems, 2522297--5294, 2022.
12

---

Published as a conference paper at ICLR 2026
Huiyang Shao, Xin Xia, Yuhong Yang. Yuxi Ren, Xing Wang, and Xuefeng Xiao. Rayflow: aware diffusion acceleration via adaptive flow trajectories. arXiv preprint arXiv:2503.07699, 2025.
Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. In International Conference on Machine Learning, pp. 32211-32252. PMLR, 2023.
Alimama-Creative al
Team.
mlu.I-turbb-aapha.
https: https::/huggingfaan .c/ 2024. Accessed: 2025-05-15.
Pascal Vincent. connection between score matching and denoising autoencoders. Neural compu- tation, 23(7):1661-1674, 2011.
Fu- Yun Wang. Zhaoyang Huang, Alexander Bergman, Dazhong Shen, Peng Gao, Michael Lingel- bach, Keqiang Sun, Weikang Bian, Guanglu Song, Yu Liu, et al. Phased consistency models. Advances in neural information processing systems, 37:83951-84000 2024a.
Yutong Wang, Jiajie Teng, Jiajiong Cao, Yuming Li, Chenguang Ma, Hongteng Xu, and Dixin Luo. Efficient video face enhancement with enhanced spatial-temporal consistency. arXiv preprint arXiv:2411.16668, 2024b.
Zhendong Wang, Huangjie Zheng, Pengcheng He, Weizhu Chen, and Mingyuan Zhou. Diffusion- gan: Training gans with diffusion. In The Eleventh International Conference Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023a. URL net/ Forum?id=HHa 7UbpNHuA
https:://penrest
Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Pro- lifiedreamer: High-fidelity and diverse text-to-3d generation with variational score distillation. Advances in Neural Information Processing Systems, 36:8406-8441, 2023b.
Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to- image synthesis. arXiv preprint arXiv:2306. 09341, 2023.
Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: Learning and evaluating human preferences for text-to-image generation. In NeurlPS, 2023.
Tianwei Yin, Michaël Gharbi, Taesung Park, Richard Zhang, Eli Shechtman, Fredo Durand, and Bill Freeman. Improved distribution matching distillation for fast image synthesis. Advances in neural information processing systems, 37:47455 2024a.
Tianwei Yin, Michaël Gharbi, Richard Zhang. Eli Shechtman, Fredo Durand, William T Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 6613-6623, 2024b.
13

---

Published as a conference paper at ICLR 2026
A THEORY PROOFS, DISCUSSION, AND ADDITIONAL RESULTS FOR IDA
Symbol Definitions. We provide explicit symbol definitions and clarifications for our IDA method, which is primarily applied to flow matching-based text-to-image models such SD 3.5 Medium/Large and FLUX.1 I dev. Since these models operate within the flow matching framework, formalize the relevant notation used throughout the analysis and proof.
Let the generator be Gg. which maps a random noise ~ (O0, 7) to generated sample GG()). The fake/teacher models predict velocity fields vs(x, t) and (z(2, t), corresponding the fake model and the teacher model, respectively.
Xo) as described in Eq the generator induces t-marginals: Given the forward kernel q(x,
pg Xx) = (a Go(t))],
(16)
where Py (X₂) is the distribution induced by the generator samples at timestep t. Similarly, the teacher and fake models define pr (X,) and (X)), respectively.
Generator-induced FM target field and tracking objective (inner loop of Eq. 8). Let
X1, denote the random FM path induced by the forward kernel q(x, Xo). where Xo Py and X1 (oO T) is the Gaussian noise endpoint. We define the generator-induced FM target field as
(0(2, () E[Xi Xo X. x]]
(17)
Using this target, we define the tracking error at the k-th round of min-max optimization:
d2 = Ex,x, 11 (Xe,t) 60, (X., f)ll-
(18)
This is the quantity directly optimized by the inner loop: a smaller d, indicates that the fake model (and thus py) better tracks the generator-induced path distribution Pg-
In our setup, the generator shares the same network architecture as the teacher model and is also implemented a conditoonl velocity network, with output defined for any t [0, 1]. Here, define the generator-purametrriied velocity field na (2,, () as the output of the generator's network timestep
(19) Notation. We abuse notation slightly and write vo (xx,,) to denote evaluating the deterministic field -)(:,)) at the random input (X,,t). t). In addition, during DMD training, supervision is applied only a sparse set of anchor timesteps for the generator, making a few-step generator that can produce clean samples in small number of steps (e.g.. 4). However, this sparsity does not restrict the definition of un (I. t) to the anchors: since the generator is time-conditional, we can still evaluate its output for any t [0, 1}; anchors only specify where gradients applied during training.
vo(2,t) = fe(z, t),
t [0,
Self-consistent error.
We further define the self-consistent error at round k as
B E,,x, [|, (X,,t) 60, (X,, t)]l-
(20)
The self-consistent error quantifies how well the generator's own velocity prediction agrees with its pathwise FM target along the generator sample-induee induced trajectory. Our analysis assumes sup, B < which is supported empirically by step consistency results in Appendix A.5
Assumption A.1 (Local regularity). There exists a neighborhood 24 C Rd of the parameter trajec- tory and constants L, C.,, C7 > such that:
(Param->te -field Lipschitz)
Be,xirpa ll owr (X,, uxe, L 1|W - -wll, Va,w' U (21)
(Generator smoothness)
Ex,x Vo (X,, - vo, (X,, 1) S Cv 1.110++1 - Oell
(Target smoothness) Exx.,rp, (X,,, ūo, (X,,,) 1) < Co |10+11 0x
In particular, the param-ffell Lipschitz condition applies to both the fake and generator parameters, e.g., to the pair (w', w) = (0x+1, 0x). We also assume the max optimization steps satisfy ||0++1 0xll 0, since the generator will gradually converge the training process.
14

---

Published as a conference paper at ICLR 2026
A.1
DECOMPOSITION AND INNER BEST RESPONSE
Proposition A.2 (Cross-entropy decomposition and best response). With V(0, Φ) defined in Eq. for fixed 0,
V(0, .) = E, DKL(pg(X)) (l (x.)) - E, Dku(P)(X)) (l (X.)). Hence maxç V(0, IE, (Pall))))p) attained Pf (Xt) - P)(()) for a.e. a.e.
(22)
t.
Proof. Use the cross- entsopy identity Ep, cancels. Non-negativity of KL. yields the claim.
= - and the fact that H (Pg)
A.2 BOUNDING THE TRACKING ERROR UNDER IDA UPDATE
Firstly, we define the parameter tracking error Cj := 1 11ll and the field gap A Ex,xx |,x, () vo, (X,, 4)|l Meanwhile, as defined in Eq. 18 and Eq. 20 we have the tar- get field tracking and self-consistent errors dę and 3x- In this subsection, we use the IDA update 0k+1 XΦx + (1 x)0x+1 (Eq. 9 to control the parameter tracking error and the field gap.
What we prove. With the IDA update, we (i) bound the next-step field gap in terms of the parameter tracking error e and the outer step ||0 0xIl. (ii) derive coupled one-step recursion for (ex, dg), and (iii) obtain the asymptotic bound lim ou de S B..
Lemma A.3 (Field-gap bound under IDA using e&). Under Assumption A.1
A &
LAex +
[I(1 - X) + C.) ||0x+1 - 0&
(23)
Proof. Insert and subtract vs, and apply generator smoothness and the param-*field Lipschitz:
A++1 3 E| Uo2+1 U0 |l + E|| voo 10
$ Ø1 - 0% + C2 - 0xll-
Using the IDA update, 0x+1 0- = \(0x - 0%) + (1- A)(0 - Og), so by the triangle inequality
||œæ+1 0& A1|øx 0x
A)]10K+1 OKll = dex + A)] 0&
Plug this into the previous display to obtain Eq. 23
Lemma A.4 (Coupled one-step recursion for tracking). Under Assumption A.1 and Eq. the fol- lowing coupled recursions hold:
Ek+1 S Aek A110k+1 0x
(24)
det1 Ldex
+ 3x + L(1 A) + 2C,, + Cτ] |10 Bxll
(25)
=
Proof. For Eq. 24 from Φx+1 = Aox + (1 A)0841 we have ex+1 |A(dx - 0c+1)l S dea + A||0x41 0zl
25 For Eq. by the triangle inequality,
dx+1 E
0o
E |Udzer
00
El
U0241
0o
At-1
Be
Apply Lemma A.3 to ARti and the generator/target smoothness to Be+1:
Be+1 S 3& + (C. + Ca)l 0x
Combining the bounds and using DR+1 < LŽe; + [L(1 - A) + C.C]llR44 - 0xl gives Eq. 25
Proposition A.5 (Asymptotic bound for the tracking error). Ifsup; 3e S for some finite B, < 0o and [0x++1 - 0zl + 0, then
lim de k-+00
< B..
Proof. From Eq. 24 (1 X) lim sup e A lim sup || 0x 0, lim sup ej 0. Taking
lim sup in Eq. 25 and using lim limsup ||0 - 0el = O together with sup Br < 3, yields the
claim.
15

---

Published as a conference paper at ICLR 2026
A.3 FROM VELOCITY FIELD ERROR TO KL GAP ALONG THE FM PATH
In this subsection, given pairs of velocity fields (Cox ) at round k, we relate their discrepancy to the pathwise KL divergence between Pa (X+) and (Xx). Assumption A.6 (Fisher-KKL control and score-velocity relation). For each E [0, 1J. there exists a constant C such that
(x(leg(x)llpxxxx)) < C El (X.,) 1|2
(26)
where (.,() V Po 1) denotes the score. Moreover, along the Gauesian) flow- matching path, the score and velocity differences are related by scalar factor a(t) > 0 that depends only the time schedule:
sy(X,,t) (x,, a(t) (vo(X,,)) wo vx(,)tt))
(27)
Assume t + Ct and t + a(t) are measurable and that C E.[C, a(t)²] oc.
(28)
The constants C₁ and (t)) are independent of (Ø) 0).
Immediate consequence. Combining Eq. 26 Eq. 27
and averaging over yields
E. DRI(PA (Xt) (X.)) S C|| (o((,, (X.,))))
(29)
Proposition A.7 (From field error to e-best response). Under Assumptions A.7 and A.6 define the T2 versions of the tracking and self-consistent errors
du = (EllvOL (Xe, t) 60g (X,, 1)1)) 1/2
BR = (Ellvo, (X2,t) 00 (X,, (e.t)13 1)) 1/2
Then for each iterate l there exists
Ex = 20 + (2)
such that
E. DRI(pg( (X.) (x.)) S Ex- It controls the inner KL gap, and thus induces an E ta-best-responss bound.
Proof. By Assumption A.6
E, DrL. < C E 2y | Decompose 0g = (uo (0) +
(vg 00) and apply ||u + 2
< 2| 2|
Bllup-
v
S 2 2/
11 + 2
6ell2 2(c + -3)
Combine with A.6 obtain the claim.
A.4 COMBINING THEM TOGETHER
Combining the IDA-specific bound on the field tracking error (Proposition A.5) with the KI. control in Proposition A.7 we obtain an e-best-response guarantee for the inner loop under IDA. Finally, invoking the decomposition in Proposition A.2 we have
V(f, Φ) E, DKL (Pg I - E, D.D ll)))
With E, (KI(py)lss) S from Proposition A.7 we obtain
E, DKu(PyllP.) - < (IDA) < E, DKa(PyllPr).
(30)
In words, under IDA the inner loop delivers an c-best response, so the outer loop approxi- mately minimizes E. DRI. (py (1p)) within e.
A.5 OBSERVATIONS OF STEP CONSISTENCY AND EMPIRICAL BOUNDEDNESS OF THE SELF-CONSISTENC ERROR
This subsection provides empirical evidence that the student generator exhibits step consistency (similar outputs under 4/8/16 FM integration steps), which in turn indicates that the self-consiaee celf-consistent error 3, E, Ex Uo, [X,, t) En (X,, t) remains bounded during
the b-best-
response analysis in 3.2 and A.4
16

---

Published as a conference paper at ICLR 2026
Figure 7: Step consistency (DMD + VFM, TTUR 5:1). Each column shares prompt/seed; rows use 4/8/16 Euler steps. Despite being trained four anchor timesteps. the 4-step generator remains visually close when re-evaluated under 8 or 16 steps, suggesting a smooth underlying velocity field along the FM path.
Figure 8: Effect of IDA. Adding IDA improves global fidelity and reduces artifacts across all step counts while preserving step consistency: sharper textures, cleaner edges, and more coherent object geometry observed at 4/8/16 steps.
Setup. We distill SD 3.5 Medium (2B) teacher into 4-step student Go. Unless otherwise noted, we use Euler integration and keep prompts and latent seeds fixed across step counts. For each column show three rows: top 4 steps. middle 8 steps, bottom 16 steps. Fig. repports DMD VFM discriminator at TTUR 5:1; Fig. g reports DMD + IDA VFM discriminator at the same TTUR.
17

---

Published as a conference paper at ICLR 2026
Observation A (Step consistency bounded self-consistent error). Across all columns in Fig. 7 and Fig. 8, the 4/8/16-step samples remain close. This indicates that the distilled student fol- lows a smooth velocity field along the FM schedule. In our notation (Eq. 20), this is consistent with (and provides empirical support for) the generator's 's self-consistent error Pk being bounded through- training (i.e., the learned field does not exhibit unstable behavior between anchor timesteps). As a sanity check, if the learned field were highly inconsistent between anchors (e.g., large/drifting mismatch to its pathwise FM target), then refining the Euler discretization (4-116) would typically induce noticeable trajectory drift and different endpoints, which is not observed here. Consequently, the non-divergent in Proposition A.7 empirically supported.
Observation (IDA improves image quality while retaining consistency). Comparing Fig. (no IDA) with Fig. (with IDA), IDA consistently enhances image quality at all step counts counts-fewer artifacts, more faithful textures, and more coherent structures- maintaining step consistency. This matches the theory: IDA reduces inner-loop drift by keeping the fake model close to the generator (thereby stabilizing tracking, i.c., reducing dæ), and together with bounded self-consistency (i.e., controlled 3x supported above), yields a stable s-best-response guarantee. Substituting E, DKL (p)) ) I E into Proposition A.2 yields the sandwich bound in Eq. 30 imply- ing that the outer loop approximately minimizes IE, (L (ppr) within
same level of
B IMPLEMENTATION DETAILS AND MORE EXPERIMENTAL RESULTS
B.1 IMPLEMENTATION DETAILS
Our entire framework is implemented in PyTorch with CUDA acceleration and trained using A100 GPUs with total batch size of 8. We adopt the AdamW optimizer (Loshchilov) (017)) with hyperparameters 31 = 0.9 and B2 = 0.999. The learning rate is set to le 6 for the distillation of SDXL and SD 3.5 Large, and 1e 5 for FLUX.1 dev. To efficiently support large-scale model train- ing, we utilize Fully Sharded Data Parallel (FSDP), which enables memory memory-efficitn efficient and scalable training of large models.
Timestep settings. We adopt different coarse timestep schedules depending on the model architecture. For SDXL.,
Reef
Text
follow the 1000-step discrete DDPM schedule used in DMD2 (Yin al. 2024a). selecting step indices 2249, 499, 749,999 9999. For SD 3.5 Large, we switch continuous timestep values (0.25, 0.5, 0.75, 1.0}. which are more suitable for flow-based models. In the case of FLUX.1 dev, which adopts shifted a inference strat- egy, we directly use the corresponding sigmas {0.512, 0.759, 0.904, 1.0} as coarse anchors.
Prompt
CLIP
Augment
Transformer
Roy
Transformer
Training details. We set the default TTUR (Two Time-Scale Update Rule) ratio to 5 in our main experiments SDXI., SD 3.5 Large, and FLUX.1 dev. For large flow-based models such SD 3.5 Large and FLUX.1 dev, we apply all proposed improvements, including Implicit Distribution Alignment (IDA). Intra-Segment Guidance (ISG), and the VFM-based Discriminator. For the diffusion- based SDXL model, we em- ploy ISG and the VFM-based Discriminator while omitting IDA.
Transformer
h6.
DINOv2
fed
Figure 9: Design of the VFM-based discriminator.
B.2 DETAILED VFM-BASER DISCRIMINATOR DESIGN
As shown Fig. 9 the discriminator integrates pretrained vision (DINOv2) and language (CLIP) encoders to provide semantically rich and spatially aligned supervision. Given input image
18

---

Published as a conference paper at ICLR 2026
apply normalization (from [-1,1] 1] to [0, 1)) and differentiable data augmentation (including color jitter, translation, and cutout). The augmented image is processed by a frozen DINOv2 vision trans- former extract multi-level semantic features. Each selected layer output is reshaped into 2D spatial map (e.g.. [B, C, H, W) and passed through a lightweight convolutional head composed of spectral-normalized residual blocks.
reference image Tee is processed through the same DINOv2 pathway (without augmentation) to extract corresponding semantic features. Meanwhile, the text prompt is encoded by CLIP (ViT- L/14) text encoder into a condition feature c, which is projected to a spatial map. Each discriminator head fuses the image feature, reference feature, and prompt condition via element-wiss multipli- cation and spatial summation to compute the final logits. (Note: In Section 3.4, we described the reference features r as extracted by the CL.IP encoder. In practice, r = (VFM((Tt)) is obtained using the same DINOv2 backbone as the input image. The Fig. 2 should also be corrected.)
B.3 TRAINING-TIME OVERHEAD OF ISG AND IDA
Under TTUR with frequency f=5, both components are applied only on generator updates (every iterations), which amortizes their cost. We measure wall-clock time (minutes) at multiple check- points and report the average per-iteration time (seconds/iter). All runs use Ax0000. Total training steps and times: SDXL 18k (32 hours), SD 3.5 Large 27k (56.3 hours), FLUX.1 dev 12k (23.4 hours). We report relative overhead as
twith
touse 100%.
%Overhead
basc
Table 3: Training- time overhead of ISG and IDA.
Method Time (min)
3k
6k
9k
12k
15k
Avg, time (s)/iter
% Overhead
Ours SDXL.
313
638
968
1284
1600
6.400
+4.44 %
Ours SDXL w/o ISG
310
621
925
1228
1835
6.128
Ours FLUX
352
705
1056
1407
1757
7.028
+3.23
Ours FLUX w/o ISG
337
680
1022
1364
1702
6.808
+3.97%
Ours FLUX w/o ISG, w/o IDA
323
648
975
1306
1637
6.548
Ours SD 3.5
375
750
1126
1503
1878
7.512
+6.16%
Ours SD 3.5 w/o ISG
355
720
1064
1417
1769
7.076
+0.57%
Ours SD 3.5 w/o ISG, w/o IDA
354
706
1059
1410
1759
7.036
Across backbones, cnabling ISG increases per-iteration time by +4.44% on SDXL, +3.23% on FLUX.1 dev, and +6.16% on SD 3.5 Large. Adding IDA top of an ISG-free baseline adds +3.97% on FLUX and +0.57% on SD 3.5. These overheads ISG (3-6%) and IDA (0.6-4%) are minor and manageable in practice, especially because both executed only generator steps under TTUR. In returm, they bring consistent gains in convergence stability and sample quality.
B.4 DETAILED RESULTS AND DISCUSSION OF GENEVAL AND T21-COMPBENCH
Tables 4 and report detailed compositional results. On SD 3.5, our 4-step models (same distilled weights. different samplers) achieve the strongest overall performance: the Euler variant attains highest GenEval among 4-step methods (Tab. 4 and best on five of the six T21-CompBench categories while ranking second on Spatial (Tab. 5). This shows that our distillation preserves fidelity while improving ane-grained attribute binding and multi-object reasoning.
On SDXL, distilled model leads GenEval among SDXL distillation methods and is best Color/ShapeTexxtre in T21-CompBench, while remaining competitive on Spatial, Non-spatial and Complex-3-ine These gains, together with strong human-preference metrics in the main text, indi- cate better semantic faithfulness at the same 4-step budget.
On FLUX, using the same distilled weights, the two solvers present complementary trade-off: the stochastic sampler attains the highest GenEval among 4-step baselines, whereas the Euler variant best Shape and second Texture/ppaiil/Cooplees333 in T21-CompBench. Overall, the
19

---

Published as a conference paper at ICLR 2026
two compositional suites corroborate COCO-5K findings: the proposed distillation improves semantic alignment and compositional correctness across diverse teachers without increasing the sampling cost.
Table 4: GenEval benchmark results. We report overall score and per-attribute pass rates (%). Bold/Underline: best/second best in distilling the same teacher.
Method
#NFE GenEvalf Single Obj. Two Obj. Counting Colors Position Color-Attr.
SDXL Distillation
LCM-SDXL
8
0,5036
99.06%
55.56%
39.69%
85.37%
7.00%
15.50%
PCM-SDXL
4
0.4944
97.50%
56.06%
39.69%
81.65%
7.50%
14.25%
SDXL-Lighhning
4
0,5332
98.44%
60.35%
45.31%
84.57%
10.50%
20.75%
Hyper-SDXL SDXL
4
0.5398
98.44%
65.40%
38.75%
88.83%
12.50%
20.00%
DMD2-SDXL
0.5779
99.69%
75.76%
47.81%
87.50%
10.50%
25.50%
Ours-SDXL
4
0,5784
99.69%
73.74%
47.81%
88.83%
10.00%
27.00%
SD 3.5 Large Distillation
SD Large Turbo
0.6877
99.06%
88.89%
68.75%
77.93%
23.00%
55.00%
Ours SD 3.5 Ours SD 3.5 (Euler)
4
0.6955
99.06%
92.93%
63.44%
81.12%
22.00%
58.75%
4
0.7098
100.00%
91.67%
67.81%
81.38%
24.50%
60.50%
FLUX Distillation
FLUX.. I schnell
0.6807
99.38%
89.39%
60.00%
77.93%
29.00%
52.75%
Hyper-FLUX
4
0,6193
98.12%
69.95%
67.50%
75.53%
16.75%
43.75%
BLUX--Tubo-Alppe
4
0,4724
88.12%
44.70%
52.50%
64.63%
13.25%
20.25%
Ours-FLUX
4
0,6471
98.75% %
70.71%
82.50%
80.05%
14.00%
42.25%
Ours FLUX (Euler)
0.6420
99.06%
71.91%
80.00%
78.72%
16.50%
40.50%
Teachers
SDXL
80
0.5461
96.88%
69.70%
41.88%
87.23%
10.25%
21.75%
SD 3.5 Large
80
0.7140
100.00%
90.66%
69.38%
81.38%
26.50%
60.50%
FLUX.1 Dev
0.6689
99.38%
82.83%
74.06%
77.66%
22.00%
46.00%
FLUX.1 Dev
25
0.6733
99.69%
84.34%
75.31%
81.12%
20.75%
42.75%
B.5 ANALYSIS OF DIVERSITY
Figure 10: Qualitative diversity under the VFM diseriminator. Left: DMD2-SDXLL Right: Ours- SDXL.
Setup. To quantify the impact of the VFM discriminator on sample diversity, we construct prompt validation set and, for each prompt, generate 16 samples per model. On this set, compute two diversity metrics across all image pairs: (i) LPIPS PPIPS-diversin, defined as the average pairwise LPIPS distance between samples; and (ii) -iveriivv obtained by feeding all images through pre-
20

---

Published as a conference paper at ICLR 2026
Table 4-Step Results on T21-CompBench. Bold/Underline: best/second best in distilling the same teacher. Our distilled SD 3.5 model approaches overall wfae-of-the-arr distillation performance.
Method
Color
Shape
Texture
Spatial
Non-spatial
Complex-3-in-1
LCM-SDXL
0.5997
0.4015
0.4958
0.1672
0,3010
0.3364
SDXL-Lightning
0,5758
0,4492
0.5154
0.2124
0,3098
0.3517
Hyper-SDXX
0,6435
0,4732
0,5581
0.2213
0.3104
0.3301
PCM-SDXL
0,5591
0.4142
0.4693
0.2013
0.3099
0.3234
DMD2-SDXL
0.6531
0.4816
0.5967
0.2231
0,3100
0.3597
Ours-SDXL
0.6867
0.4828
0.5989
0.2224
0,3100
0.3594
SD 3.5 Large Turbo
0.7050
0,5443
0.6512
0,2839
0.3130
0.3520
Ours-SD 3.5
0.7657
0.6069
0.7427
0.2970
0.3177
0.3916
Ours-SD 3.5 (Euler)
0.7711
0.6149
0.7543
0.2857
0.3182
0.3968
FLUX.1 schnell
0.7317
0.5649
0.6919
0.2626
0,3122
0.3669
Hyper-FLUX
0.7465
0,5023
0.6153
0.2945
0,3116
0.3766
FLUX-Turbo-Alpee
0.7406
0,4873
0.6024
0.2501
0.3094
0.3688
Ours-FIUUX
0.7284
0.5055
0.6031
0.2451
0.3028
0.3652
Ours-FLUX (Fulcr)
0.7363
0.5120
0.6112
0.2521
0.3028
0.3697
Table LPIPS-diversity (average pairwise LPIPS distance) and CLIP-diversity (variance in CLIP
image embeddings) on 23 -3-proppt validation set with 16 samples per prompt. Higher is better.
Method
LPIPS-Diversityy t CLIP-Diversity 1
DMD2-SDX
0,5960
0.0985
Ours-SDXL
0.6002
0.0802
SD 3.5 Large Turbo
0.5659
0.0879
Ours SD 3.5 Large
0.5664
0.0900
trained CLIP image encoder, fz-normalizing the features, and averaging - -iiinessimllaryyy over all pairs. Both metries averaged over prompts, and higher values indicate higher sample diversity.
Results. The quantitative results are shown in Tab. For SDXI., Ours-SDXLL achieves I.PIS-- diversity 0.6002 vs. 0.5960 for the DMD2- SDXL baseline, and CLiP-diversity 0.0802 vs. 0.0985. For SD 3.5 Large, our method obtains LPIPS-diversity 0.5664 vs. 0.5659 for the SD 3.5-Turbo baseline and CLIP-diversity 0.0900 vs. 0.0879, Overall, LPIPS-based diversity remains essentially unchanged (all methods within about 1 1-2%), while CLIP- changes mildly: it decreases slightly for SDXL -0.0985-000802). This supports our original claim that the VFM discrimina- tor slightly reshapes the distilled generator toward human-preferred semantic modes; and diversity change small compared to the gains human- peeaaan-rreeeeee-aliinnnen metrics.
Qualitative visualization. To further illustrate this effect, in Fig. TO we visualize 16 samples for the prompt "a teddy bear on a skateboard in Times Square" for DMD2-SDXL (left) and Ours-SDXL (right). Both models exhibit clear variation in pose, background, and apparel. DMD2-SDXL shows slightly more variation in viewpoint (e.g., more side-angle shots of the bear), while our model tends to produce more centered, compositionally consistent bears-matee masching the mild CLIP -iIP-diversity dif- ference and the improved numan--prffeeee performance reported in Tab.
In summary, the VFM discriminator gently reshapes the distilled generator toward human- preferree semantically meaningful modes, while inducing only a mild change in standard diversity measures.
B.6 1-2-STEP GENERATION
To assess the behavior of SenseFlow beyond the 4-step regime, we further evaluate 2-step and 1-step generation across SDXL, SD 3.5 Large, and FLUX. The results are summarized in Tab.
For 2-step sampling, SenseFlow naturally extends without any architectural modification and con- sistently outperforms strong baselines under the same setup and evaluation protocol. On SDXL, the 2-step model surpasses DMD2-SDXL. on all numan--referenee metrics (HPSv2, AESv2, PickScore, ImageReward). On SD 3.5 Large, the improvement is particularly pronounced: 2 steps, the model
21

---

Published as a conference paper at ICLR 2026
Table 7: SenseFlow in the low-NFE regime (2-step and 1-step). We compare the distilled students with corresponding baselines on SDXL, SD 3.5, and FLUX.
Method
#NFE
CLIP
HPSv2 f
AESv2
PickScore f
ImageReward 1
DMD2-SDXL
0.3295
0.2813
5.432
22.57
0.8666
Ours-SDXL
0.3263
0.2827
5,710
22.89
0.9192
Ours-SDXL
0.3298
0.2818
5.584
22.24
0.8570
SD3.5-Large-Turbo
0.3248
0.2745
5.394
22.16
0.7188
Ours-SD3.5 Large
0.3326
0.2889
5,537
22.88
1,2022
Ours-SD3.5 Large
0.3332
0.2803
5.421
22.32
1.0651
Hyper-FLUUX
2
0.3176
0,2682
5.541
21.63
0.3636
Ours-FLUX
0.3207
0.2866
5.926
22.38
0.9296
Sense Flow
Figure 1I: 1-step 1024x1024 samples produced by our SenseFlow on SDXL (Left) and SD 3.5 Large (Right).
achieves an ImageReward of 1.2022 and a PickScore of 22.88, clearly exceeding SD L.r-L Turbo. For FLUX, the 2-step SenseFlow student also improves over Hyper-FLUX on metrics.
For I-step generation, we start from the pretrained 4-step SenseFlow students and apply a short fine- tuning schedule (6000 iterations) for the single-step setting. This yields competitive performance with only modest drop relative to the 2-step case; for example, the 1-step SD 3.5 Large model attains an ImageReward of 1.0651 and a CL.IP score 0.3332. Qualitative comparisons for 2-step and 1-step sampling are provided in the Fig. 15 and Fig. II In particular, for SD 3.5 Large, the visual degradation from 4 steps to 2 steps is very small (from columns to Fig. 15 indicating that SenseFlow remains robust even in aggressive low-NFE regimes.
Overall, these results demonstrate that SenseFlow delivers strong generation quality at 2 steps with- out architectural changes and exhibits promising single-step performance after light tuning. This highlights its potential to further advance 1-2-step distillation for large (8B+) text-to-image models, setting where existing open-source solutions remain limited.
22

---

Published as a conference paper at ICLR 2026
Table 8: Effect of ISG on training dynamics (SD 3.5 Large). FID-T on COCO-Sk at different training
iterations for the full method (with ISG) and an ablation without ISG. Lower is better.
Method
1.5k
3k
4.5k
6k
9k
12k
Ours (w/ ISG)
14.48
22.32
17.65
18.11
16.18
15.20
Ours w/o ISG
138.2
24.99
19.43
19.12
18.07
19.06
FID-T Score Actos Traising lasations
FID Score Actoss Traising Berations
140
Ocn
120
100
FID-T
80-
04-
40
60-
6000
8000
2050
6000
15000
12500
Tuining Remations
Thining Berraions
Figure 12: FID-T and FID scores across training iterations on COCO-5k for SD 3.5 Large, compar- ing the full method with ISG and an ablation without ISG.
B.7 IMPACT OF ISG ON TRAINING DYNAMICS
We study the effect of megment Guidance (ISG) training stability and convergence speed for SD 3.5 Large. Fig. 12 visualizes the evolution of FID and FID-T on COCO-5k as a function of train- ing iterations for the full method (with ISG) and ablation without ISG, while the corresponding FID-T values at selected checkpoints are summarized in Tab.
Stability and convergence speed. ISG mainly improves the stability and convergence speed of training. At 1.5k iterations, the model trained without ISG still exhibits extremely poor alignment to the teacher (FID-T z 138), whereas the model with ISG has already reached reasonable regime (FID-T x 14.48). As training proceeds, both variants become more stable, but the ISG model consistently attains lower FID-T throughout training (e.g., 16.18 vs. 18.07 at 9k iterations, and 15.20 vs. 19.06 at 12k iterations). A similar gap is observed on the FID curves in Fig. [2] These results indicate that ISG effectively redistributes timestep importance within each segment, making the student updates less sensitive to under-trained timesteps and leading to faster and more stable convergence.
B.8 COMPARISON TO DMD2 AND DETAILED COMPONENT ABLATIONS
On DMD2 for SD 3.5 Large. For SDXL,, we directly use the official DMD2-SDXL model as a strong baseline in the main table. For SD 3.5 Large, however, there is currently no public DMD2 implementation. We therefore implement a faithful DMD2-style latent discriminator on top of the DiT backbone and attempt train a "DMD2-SD 3.5 Large" student under the same settings. De- spite extensive tuming (e.g.. increasing the TTUR up to 20), this model fails to converge in practice and collapses to almost all-black samples. Only after replacing the vanilla discriminator with the proposed VFM-based discriminator does training start to produce meaningful images, albeit still with noticeable instability. Adding IDA and ISG on top VFM then gradually improves the quality and stability, eventually yielding the performance reported for SenseFlow.
Because the DMD2-SD 3.5 Large variant does not reach a meaningful operating regime (i.e., collapses), we do not include it as baseline in Tab. [ Instead, the we provide qualitative vi- sualizations of this progression in Fig. 13 from DMD2 (black images) to DMD2+VFM, then DMD2+VFM+ID. (Ours w/o ISG). Ours (full), and DMD2+VFM+ISC (Ours w/o IDA). This vi- sual sequence illustrates why a direct DMD2 bascline is difficult to scale to SD 3.5 Large and how the proposed modifications mitigate its failure modes.
23

---

Published as a conference paper at ICLR 2026
More comprehensive ablations. To analyze the contribution of individual components, include an expanded ablation in Tab. For SD 3.5 Large, the table reports: (a) the full SenseFlow model, (b) variant without ISG (w/o ISG), (c) variant without IDA (w/o IDA), and (d) variant without both ISG and IDA (w/o ISG, w/o IDA). Performance degrades steadily along this path: removing ISG already causes noticeable drop across FID-T and human-pres creman-preference metries, removing IDA leads to further degradation, and removing both ISG and IDA results in a clear collapse (FID-T increases from 13.38 43.84, with large drops in HPSv2, PickScore, and ImageReward), This pattern confirms that both IDA and ISG contribute substantially to the final performance and that they interact synergistically.
For SDXL, the same table compares DMD2-SDXL, DMD2 equipped with the VFM discriminator (DMD2 w VFM), and the full Ours-SDXL. model. This isolates the effect of the VFM discrim- inator (DMD2 vs. DMD2 VFM) and then the additional gains obtained by integrating ISG on top (DMD2 w VFM vs. Ours-SDXL).. particular, equipping DMD2 with VFM improves most human-preferencee metrics, and adding the full SenseFlow pipeline yields further consistent gains.
Overall, Tab. and the accompanying qualitative grid Fig. 13 together provide a step-by-step view of how the three components-VFM discriminator, IDA, and ISG-each improve the model and how their combination yields the strongest overall performance, while also clarifying why direct DMD2 baseline is not informative for SD 3.5 Large.
Table 9: Expanded ablation study of IDA, ISG, and the VFM discriminator,
Method
FID-T!
HPSv21
Pickt
ImageReward
AESv21
Stable Diffusion 3.5 Large
Ours
13.38
0.3015
23.03
1.1713
5.482
w/o ISG
17.00
0.2971
22.75
1.0186
5.453
w/o IDA
17.83
0,2800
22.47
0.9365
5.407
w/o ISG, w/o IDA
43.84
0.2555
20.60
0.3828
5.102
Stable Diffusion XL
DMD2-SDXL DMD2 w VFM
15.04
0.2964
22.98
0,9324
5.530
18.55
0.2995
23.00
0.9744
5.625
Ours SDXL
17.76
0.3010
23.17
0.9951
5.703
Table 10: Quantitative results of different adversarial loss weights.
Method
FID-T
CLIP Score f
HPSv2
PickScore f
ImageReward
Hyper-SDXL
13.71
0.3254
0.3000
22.98
0.9777
Ours (Ag 0.25)
17.53
0.3234
0.3003
23.15
0.9326
Ours (A = 0.5)
17.76
0.3248
0.3010
23.17
0.9951
Table 11: Quantitative results of different backbone scales.
Method
FID-T +
CLIP Score
HPSv2
PickScore ?
ImageReward
Ours w VIT-S
17.26
0.3262
0.2983
23.12
0.9635
Ours w VIT-B
16.58
0.3234
0.2991
23.07
0.9218
Ours w VIT-I
17.53
0.3239
0.3003
23.15
0.9326
B.9 QUALITATIVE COMPARISONS
We further provide qualitative side-by-side comparisons for SDXL, SD 3.5 Large, and FLUX. For each model family, show samples from SenseFlow and strong baselines under the same prompts. As illustrated in Figures [6] SenseFlow tends to produce images with sharper details, cleaner structure, and more faithful overall quality.
24

---

Published as a conference paper at ICLR 2026
owl perches quietly twisted branch deep within ancient forest. sharp yellow eyes keen und wutchful
high-resolution orange Porsehe under sunshine
A girl with hairband performing a song with her guitar on warm evening at local market, children's story book
shiha inu wearing heret and black turtleneck
DMD2
DMD
DMD VFM+IDA (Ours w/o ISG)
Ours
DMD VFM+ (Ours IDA)
Figure 13: Qualitative ablation SD 3.5 Large under shared prompts. Plain DMD2 collapses to all-black samples; adding VFM recovers basic structure, while introducing IDA and ISG progres- sively improves fidelity and image alignment, with the full SenseFlow model yielding the most visually pleasing and semantically faithful results..
B.10 MORE ABLATION STUDY RESULTS AND VISUALIZATION SAMPLES
Effeet of Different Adversarial Loss Weights. In our main experiments, the hyperparameter Ag in Algorithm|] [1. Line 22, is set to 0.5. 0.1, and 2.0 for SDXL, SD 3.5 Large, and FLUX.1 dev, respectively. To further investigate the impact this hyperparameter, we conduct an ablation study using SDXL as example, decreasing AG to 0.25. The results are presented in Tab. 10 We observe that setting λo = 0.5 leads to improved performance across most metrics, including CLIP Score, HPSv2, PickScore, and IewaeReward. Notably, this configuration achieves the best scores on HPSv2, PickScore, and ImageReward among all methods Tab, I These results highlight the strong semantic and visual supervision capabilities of our VFM-based discriminator.
25

---

Published as a conference paper at ICLR 2026
Results of Different Backbone Scales. We evaluate the impact of different VFM backbone scales (ViT-S, B, and L) in the discriminator on SDXL distillation. Interestingly, the results (Tab do not follow a monotonic trend with respect to model size. ViT-B achieves the best FID-T, while ViT-S yields higher CLIP Score and ImageReward. ViT-L. slightly outperforms others on HPSv2 and PickScore. These findings suggest that different backhone scales offer different trade-offs in semantic alignment versus visual fidelity, and that larger backbones do not necessarily guarantee consistent improvements across all metries. This observation is partially consistent with findings in the ADD(Sauer et al. 2024b) paper, which also noted diminishing returns when scaling the discrim- inator. In our main paper, we adopt ViT-L as the default backbone for the WFM-based discriminator.
More visualization results of our methods on SD 3.5 Large and SDXL. As shown in Fig. 17 and Fig. 18 we present more samples produced by out 4-step generator distilled from SD 3.5 Large and SDXL., separately. The prompts of these samples are listed later in our appendix.
B.11 PROMPTS FOR FIG.
FIG. 17
AND FIG. 18
We use the following prompts for Fig. From left to right, top to bottom:
A red fox standing alert in snow-covered pine forest
A girl with a hairband performing song with her guitar on a warm evening at a local market, children's story book
Astronaut on camel on mars
A cat sleeping on a windowsill with white curtains fluttering in the breeze
A stylized digital art poster with the word "SenseFlow" written in flowing smoke from stage spotlight
A surreal landscape inspired by The Dark Side of the Moon, with floating clocks and rain- bow beams
a hot air balloon in shape of heart. Grand Canyon
A young man with leather jacket and messy hair playing cherry-red electric guitar on rooftop sunset
A young woman wearing a denim jacket and headphones, walking past a graffiti wall
A photographer holding camera, squatting by lake, capturing the reflection of the moun- tains in an early morning
a young girl playing piano
A close-up of a woman's face, lit by the soft glow a neon sign in a dimly lit, retro diner,
hinting at narrative of longing and nostalgia
Besides, we use the following prompts for Fig. 17 From left to right, top bottom:
A quiet room with Oasis album covers framed on the wall, acoustic guitar resting on a stool
An astronaut lying in the middle of white ROSES, in the style of Unsplash photography.
cartoon dog sits at a table, coffee mug on hand, as a room goes up in flames. "Help" the dog is yelling
Art illustration, sports minimalism style, fuzzy form, black cat and white cat, solid color background, close-up, pure flat illustration, extreme high-definition picture, cat's 's eyes de- pict clear and meticulous, high aesthetic feeling, graphic, fuzzy, felt, minimalism, blank space, artistic conception, advanced, masterpiece, minimalism, fuzzy fur texture.
Close-up of the top peak of Aconcagua, a snow-covered mountain in the Himalayas at sunrise during the golden hour. Award-winning photography, shot Canon EOS R5 in the style of Ansel Adams.
A curvy timber house near a sea, designed by Zaha Hadid, represents the image of cold, modern architecture, night, white lighting, highly detailed
teddy bear skateboard in times square
26

---

Published as a conference paper at ICLR 2026
black and white picture of woman looking through the window, in the style of Duffy Sheridan, Anna Razumovskaya, smooth and shiny, wavy, Patrick Demarchelier, album cov- ers, lush and detailed
As for Fig. 18 we use following prompts from left to right, top to bottom:
Astronaut in jungle, cold color palette, muted colors, detailed, 8k
bookshelf filled with colorful books, potted plant, and small table lamp
A dreamy beachside bar at dusk serving mojitos and old fashioneds, guitars hanging on the wall
A portrait of a human growing colorful flowers from her hair. Hyperrealistic oil painting. Intricate details.
Peach-faced lovebird with slick pompadour.
stunning and luxurious bedroom carved into a rocky mountainside seamlessly blending nature with modern design with a plush earth-toned bed textured stone walls circular fire- place massive uniquely shaped window framing snow-capped mountains dense forests
An acoustic jam session in a small café, handwritten setlist on the wall, cocktails on every table
blue Porsche 356 parked in front of yellow brick wall.
27

---

Published as a conference paper at ICLR 2026
towering jellyfish queen glides gracefully through kingdom, translucent fendrils trailing behind elegant gown. Bieluminescent patterns ripple ucross her ethereal body, pulsing in syne with the deep ocean currents. Tiny fish mesmerizing formations around soft, hypnotic glow
every movement.
Digital of young girl reading an enchanted forest, inspired by Arthur Rackham's whimsical style. Warm sualight filters through lush greenery (soft greens, browns, golden yellows). 35mm medium shot centers her, book in hand, surrounded by fuiries, glowing fireflles, and curious foxes. Delicate glitter accents add magic perfect for a child's dreamy escape.
stunning greek fantasy paluce the center of sprawling city full of librarles, shops, and bustling activity, background rolling mountain range, large fluffy clouds the sky Use ethereal lighting and light colors to create an idyllic atmosphere, Epic sky. Otherworidly, Hyper realistic
spires, large windows airy quality palace
Bokeh, Electric Colors, Accent Lighting, Lightning, Inferno, insanely detailed and intricute, hypermaximllitt elegant, ormate. realistic, super detailed, phoenix with wings flume,front view Hiew.magieul atmosphere.
page clothing designs,
SOREELL CARIIIII top corner
the heart ancient cathedral, Excalibur upon altar of marble, encased in shimmering, ethereal light The stained-glass windows cast multiccloree beams across blade, illuminatinn intricate carved quiet reverence the chamber for legends say that only the true king its hilt without being turned to dust.
SDXL
LCM-SDXL
Hyper-SDXL
DMD2-SDXL
Ours-SDXL
Figure 14: Qualitative SDXL comparisons under shared prompts. Each corresponds one prompt, showing SenseFlow and strong SDXL baselines side by side.
28

---

Published as a conference paper at ICLR 2026
A mischierous minion transformed intu dark side warrior, inspired by Darth Vader, stands menacingls dimly chamber. Its yellow, cyllnnrrical body is painted matte biack. with glossy red accents glowing faintty, wears Bowing black: cape, custom helmet with sharp edges menoring poggl-eye glowing red. hand. tiny yet powerfull havec still being adorabit chaotic. illominated by faint red and blue lights, evoking the eminous atmosphere of Sich lair.
lightsaler hums energy. The minion's expression
asual playful
ready
background
giant, entirely gingerbread deep. rumbling dough cazy.
kitchen. His icing-swirt eychrows lift in delight as he pulls tray of enchunted pustries from the oven each one shuped like tiny, dancing creature. scent sugar fills candy-button eyes pride.
Japanese sty tunn sushi restaurant cartoon with soft and funny contours with 3d with white background
A dramatic black-and-white portrait of retired footballer holding his boots in one hand and guitar in the other
baker polling fresh croissants brick rustic bakery
A stylized digital with the word "SenseFlow" written flowing smoke from stage spotlight
Serseo
SSense Flow
Sense low
Sense Flow
Sense FenseFlow
Fly
SD 3.5
SD Turbo (4)
Ours-SD 3.5 (4)
SD 3.5 I. Turbo (2)
Ours-SD 3.5 (2)
Figure 15: Qualitative SD 3.5 Large comparisons under shared prompts. SenseFlow produces visu- ally appealing images with improved composition and text-image consistency compared to SD 3.5-
Large-Turbo and SD L.reLa
29

---

Published as a conference paper at ICLR 2026
yellon, sylindrical body is painted matte black, with glossy red accents glowing faintly. It wears flowing black cape. custom hetmet with sharp edges and single menaring pogzle-eye glowing red. In hand, thny yet powerful red lightsaber homs with energy. The mision's expression is playful mischief,
transformed
warrior, inspired by
menacingly dimly
while still being adorably chaotic. dark background determinntion minated Faint red and blue lights, evaking ominoos atmosphere Sith
Inside the FC Barcelona locker room, No.10 jersey with the name "Sensel low" hangs under the spottight, surrounded by red
and blue walls and team pear
SENSEFLOW 10
MAVE
10
10
10
10
A celestial hard with Bowing, star-spoekled robes strums erystallime harp that hums with the music of the cosmos. Their silver hair caught their eves shine like twin galasies. they play, glowing comstellations weaving stories of forgatten legends. The air vibratte with an ethereal melody, bending reality itself to their song.
pulling
bakery
A stylish bartendor preparing "Catalan Sunset" cocktail, with small Barça etched on the glass
surreal landscupe inspired by The Dark Side the Moon, with Boating clocks and rainbow beams
FLUX
FLUX-Schnell
FuUX-Turb-Alphh
Hyper-FLUU
Ours-FLUX
Figure 16: Qualitative FLUX comparisons under shared prompts. SenseFlow provides clearer struc- tures and better alignment with the textual descriptions than Hyper-FLUX and FLUX-Turbo-Ale
30

---

Published as a conference paper at ICLR 2026
Help
Qeets
Figure 17: 1024x1024 samples produced by our 4-step generator distilled from SD 3.5 Large
31

---

Published as a conference paper at ICLR 2026
Figure 18: 1024x1024 samples produced by our 4-step generator distilled from SDXL.
32


Reviews:
[{"review_id": "nBAmLlSFjl", "reviewer": "Reviewer_qo4N", "summary": "This paper proposes an improved version of DMD (VSD). The idea is to set the parameters of the fake score model as a linear combination of the fake model and the student model. The paper also proposes to reallocate timestep importance to help convergence.", "questions": "Please refer to the weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper studies the problems of DMD and proposes a fix. The proposed method achieves good results for finetuning Flux and SD3.5. \n\n2. The paper proposes good ablation experiments to help readers understand the contribution of each component. \n\n3. The proposed method is faster compared with DMD.", "weaknesses": "1. Although I recognize IDA as a meaningful contribution, the proposed method IDA only works when the student model and fake model have the same network structure. For example, VSD (DMD) is originally designed for text-to-3D. Or more broadly speaking, the distribution matching does not require the student model to be same as the teacher model (real model here). In these general settings, the IDA does not work. \n\n2.  The images in Fig5, Ours-SD3.5 seem too bright. Same thing for Fig 10. What could be the cause?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0JMA90c2x2", "reviewer": "Reviewer_xdRf", "summary": "The paper addresses instability in Distribution Matching Distillation (DMD/DMD2) when scaling to large flow-based text-to-image (T2I) teachers (e.g., SD-3.5 Large, FLUX.1-dev), identifying poor fake-network tracking and sparse timestep supervision as key issues. It introduces $\\textbf{Implicit Distribution Alignment (IDA)}$, which stabilizes training by progressively aligning the fake network with the generator ($\\phi \\leftarrow \\lambda \\phi + (1-\\lambda)\\theta$), and $\\textbf{Intra-Segment Guidance (ISG)}$, which samples intermediate denoising steps to better cover under-trained timesteps. A VFM-based discriminator (frozen DINOv2 + CLIP) provides semantic guidance. On SDXL, SD-3.5 Large, and FLUX.1-dev, SenseFlow trains 4-step generators that nearly match 80-step teacher quality (e.g., GenEval 0.7098 vs. 0.7140) with minimal overhead ($\\approx$3--6\\% for ISG, 0.6--4\\% for IDA).", "questions": "+ Please add benchmarks about diversity as I mentioned above\n+ Does IDA hurt/help in that small-scale models setting (SD1.5 and SDXL)? Also, can you discuss how anchor selection (Appendix B.1) affects results?\n+ Can SenseFlow handle 1-2 step(s) sampling ? If yes then please show comparison and benchmarks", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "+ IDA directly addresses the brittle inner loop in DMD via an explicit, cheap parameter interpolation; Appendix A formalizes that this bounds the generator–fake field gap. The “Training‑hours vs. FID” plot (Fig. 3) shows markedly smoother convergence with IDA on SD‑3.5 Large.\n\n+ Experiments span three strong teachers (SDXL 2.6B, SD‑3.5 Large 8B, FLUX.1‑dev 12B) with contemporary baselines (LCM/PCM/Lightning/Hyper, SD‑3.5‑Turbo, FLUX‑Turbo/‑schnell). Results cover COCO‑5K, GenEval, and T2I‑CompBench; for SD‑3.5 the 4‑step Euler variant is best or second‑best across metrics and best in 5/6 CompBench categories.\n\n+ Tables/figures ablate the contributions of IDA, ISG, and VFM‑disc (e.g., removing both tanks SD‑3.5 quality from FID‑T 13.38 → 43.84), and they quantify overheads (ISG ≈3–6%; IDA ≈0.6–4%). Appendix A.5’s step‑consistency visuals support the assumption behind IDA.", "weaknesses": "+ The paper notes that adding the VFM discriminator improves human‑preference proxies while slightly raising FID‑T (interpreted as reduced diversity). However, there is no explicit diversity study (e.g., user study, precision–recall curves over seeds). Given the important of this trade‑off in distillation, a quantitative diversity analysis would strengthen the claims.\n\n+ The authors seem to omit IDA on smaller models SDXL but claims generality. A small ablation whether IDA helps small-scale models or not can clarify further the importance of IDA. Timestep settings differ per model (Appendix B.1) but the impact of these choices isn’t studied.\n\n+ All the results are 4‑step; but the prposed method should be able to handle 1-2 step(s) sampling. Showing at least 2‑step results or failure modes would calibrate how far SenseFlow goes in the low‑NFE regime.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1iVkRVwKyV", "reviewer": "Reviewer_j3tb", "summary": "This paper addresses the limitations of Distribution Matching Distillation (DMD and DMD2)—a state-of-the-art timestep distillation framework for diffusion models—when applied to large-scale flow-based text-to-image models such as SD3.5 and FLUX. The authors first analyze the performance of vanilla DMD on these large models and identify key challenges in scaling. To overcome them, they propose three improvements: Implicit Distribution Alignment (IDA), which enforces consistency between the weights of the fake model and the student model; Intra-Segment Guidance (ISG), which redistributes timestep denoising importance learned from the teacher model; and a VFM-based discriminator, which leverages the rich prior knowledge encoded in large foundation models.", "questions": "1. If possible, please provide additional visualizations illustrating the effect of ISG on the training process.\n2. The use of strong foundation models such as DINOv2 and CLIP for the discriminator is reasonable. However, the incorporation of the reference image feature as an additional conditioning input deviates from the vanilla DMD design. Could the authors clarify the motivation for this choice and discuss its impact on image quality and diversity?\n3. How many images were used during training?\n4. In Table 1, it is strongly recommended to include DMD2 results for SD3.5 and FLUX to demonstrate the proposed method’s effectiveness over DMD2 when applied to flow-based models, as claimed.\n5. Table 2 is not comprehensive and should include more ablation variants that isolate or combine each proposed component.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The paper is well-written and well-structured, presenting a clear and compelling motivation for the study.\n2. The proposed approach is thoughtfully designed, addressing the common challenges of stability and scalability in distribution-matching distillation.\n3. The experiments are comprehensive, supported by detailed analysis and clear explanations.", "weaknesses": "1. The effect of ISG is not very clear. Although Table 2 shows slight improvements in quality and human-preference metrics, no qualitative comparisons are provided for this component. Including an additional visualization—similar to Figure 3—or a reconstruction loss curve could better illustrate ISG’s impact on training stability and convergence speed.\n2. The design of the VFM discriminator also requires further clarification, particularly regarding the rationale for incorporating another reference image feature as a conditioning input. Moreover, additional visual comparisons would strengthen the paper, especially since you claim a trade-off between distribution coverage (FID-T) and human-preference scores when using this discriminator.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Dt9bpl0OyU", "reviewer": "Reviewer_iTJr", "summary": "This paper improves Distribution Matching Distillation (DMD) for large text to image models like SD 3.5 and FLUX by introducing Implicit Distribution Alignment (IDA) to handle divergence issues, Intra Segment Guidance (ISG) to improve timestep wise denoising and VFM-based discriminator for effective training.", "questions": "(i) In Figure 3, I noticed that TTUR(10) appears to result in more unstable behavior compared to TTUR(5). Intuitively, a larger update ratio should allow for better optimization of the inner problem, potentially leading to more stable training. Maybe authors have some intuitions about that and can provide  an intuitive explanation for this behavior?\n\n(ii) There seems to be a typo in line 726: \"Eq. equation\". Please correct it.\n\n(iii) In the proof of Lemma A.3, could the authors explain why the inequality $\\mathbb{E}||v_{\\phi_{k+1}} - v_{\\theta_k}|| \\leq L ||\\phi_{k+1} - \\theta_k||$ holds under Assumptions A.1? \n\n(iv) While I recognize that most reported metrics reflect human preferences to some extent, human evaluation remains an important standard in large-scale generative model assessments. Could the authors clarify why no human evaluation was included, and whether such comparisons might be provided in future work?", "limitations": "", "rating": 4, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "(i) This paper makes contributions by scaling DMD to large flow-based models, such as SD3.5 and FLUX\n\n(ii) The authors propose Implicit Distribution Alignment (IDA) is a novel method that helps stabilize training\n\n(iii) The approach also shows strong empirical performance across multiple challenging benchmarks.", "weaknesses": "(i) The theoretical justification provided in Proposition 3.1 lacks clarity. Specifically, the definition of the conditional drift term $\\hat{v}_{\\theta}(X_t, t) = X_1 - X_0$ in line 674 is problematic. Since this quantity is conditioned on $X_1$, it should be treated as a random variable dependent on $X_1$. However, the authors appear to use it deterministically in the derivation of error bounds, which raises concerns about the correctness of the results.\n\n(ii) The error term $\\Delta_k = \\mathbb{E}||v_{\\phi_k} - v_{\\theta_k}||$ is not well justified. According to the authors’ own note in line 667 about definition of $v_{\\theta_k}$: “where the velocity field is the output of the generator network at each timestep t”, $v_{\\theta}$ is only defined at anchor timesteps. In contrast, $v_{\\phi_k}$ is defined at all timesteps. Since the expectation is taken over $t$, this mismatch makes the definition of $\\Delta_k$ unclear.\n\n(iii) My main theoretical concern is with Proposition A.7. As far as I understand, the proof does not include any updates related to IDA, since there are no updates over $k$ shown in the argument. Despite this, the authors make conclusions about the behavior of the procedure. It is possible that I may be missing some detail, but this seems to be mainly due to unclear definitions discussed earlier.\n\n(iv) The proposed Intra-Segment Guidance (ISG) loss (Equation 12) appears closely related to the loss function used in Consistency Trajectory Models (CTM) [1]. If we set $s = \\tau_{i-1}, u = t_{\\text{mid}}, \\tau_i = t$, ISG becomes a specific case of the CTM loss. If this connection is correct, the novelty of ISG should be further clarified and better distinguished from prior work.\n\n(v) Equation 4 seems to contain a conceptual error. At $t = 0$, the forward process should converge to a delta distribution centered at $X_0$, yet the formula implies convergence to a standard Gaussian. This contradicts the standard behavior of the forward process. Moreover, in the appendix, the authors denote $X_1$ as the data variable, which adds further confusion regarding the consistency of notation.\n\n[1] https://arxiv.org/abs/2310.02279 Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)