Align Your Flow: Scaling Continuous-Time Flow Map Distillation
Authors: Amirmojtaba Sabour
Sanja Fidler
Karsten Kreis
Venue: NeurIPS 2025 poster ()
Keywords: Generative models
diffusion models
consistency models
flow models
Primary area: deep_learning
TL;DR: We develop flow map methods for state-of-the-art few-step generation, generalizing flow, diffusion, and consistency models.

Abstract: Diffusion- and flow-based models have emerged as state-of-the-art generative modeling approaches, but they require many sampling steps. 
Consistency models can distill these models into efficient one-step generators; however, unlike flow- and diffusion-based methods, their performance inevitably degrades when increasing the number of steps, which we show both analytically and empirically.
Flow maps generalize these approaches by connecting any two noise levels in a single step and remain effective across all step counts. 
In this paper, we introduce two new continuous-time objectives for training flow maps, along with additional novel training techniques, generalizing existing consistency and flow matching objectives. We further demonstrate that autoguidance can improve performance, using a low-quality model for guidance during distillation, and an additional boost can be achieved by adversarial finetuning, with minimal loss in sample diversity.
We extensively validate our flow map models, called *Align Your Flow*, on challenging image generation benchmarks and achieve state-of-the-art few-step generation performance on both ImageNet 64x64 and 512x512, using small and efficient neural networks. 
Finally, we show text-to-image flow map models that outperform all existing non-adversarially trained few-step samplers in text-conditioned synthesis.

Align Your Flow: Scaling Continuous-Time Flow Map Distillation
Amirmojtaba S 12,3
Sanja Fidler 1,23
Karsten Kreis*
'VVDDI
University of Toronto
Vector Institute
Project Page: http://wwwaaaaaaa..co.oommmooooe
Figure 1: Four-step samples by distilled text-conditioned flow map model (prompts in App. I).
Abstract
Diffusion- and flow-based models have emerged as state-of-the-art generative modeling approaches, but they require many sampling steps. Consistency models can distill these models into efficient one-step generators; however, unlike flow- and baatsidon-based methods, their performance inevitably degrades when increasing the number steps, which we show both analytically and empirically. Flow maps generalize these approaches by connecting any two noise levels in a single step and remain effective across all step counts. In this paper, we introduce two new continuous- time objectives for training flow maps, along with additional novel training techniques, generalizing existing consistency and flow matching objectives. We further demonstrate that autoguidance can improve performance, using low- quality model for guidance during distillation, and an additional boost can be achieved by adversarial finetuning. with minimal loss in sample diversity. We extensively validate our flow map models, called Align Your Flow, on challenging image generation benchmarks and achieve state-of-the-arr few-step generation performance on both ImageNet 64x64 and 312x512, using small and efficient neural networks. Finally, we show text-to-image flow map models that outperform all existing non-ad advessriialy trained few-step samplers in next-conditioned synthesis.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Method
Consistency Model
Flow Map
Flow Matching
== 0
kXx,1)
Objective
9,5, RVX. A6-/3,11
ReF, signt vi (.)) Ov.
Figure 2: Overview of Flow Maps. Flow maps generalize both consistency models and flow matching by connecting any two noise levels (s, in a single step. When s = 0, flow maps reduce to consistency models; when - they'r re equivalent to standard flow matching models. Our proposed EMD objective (see Theorem 3.2) similarly generalizes the continuous time consistency and flow matching losses. For detailed derivations, please see the Appendix.
Introduction
Diffusion and flow-based generative models have revolutionized generative modeling [25, 70, 45, 62, 14, 8), but they rely on slow iterative sampling. This has led to the development of approaches to accelerate generation. Advanced, higher-order samplers [68, 50, 51, 12, 89, 34, 61] help, but cannot produce high quality outputs with <10 steps. Distillation techniques [63, 67, 87, 72], in contrast, can successfully distill models into few-step generators. In particular, consistency models [7 69, 49] and variety of related techniques [38, 79, 80, 42, 94, 17, 22] have gained much attention recently. Con- sistency models learn to transfer samples that lie on teacher-defined deterministic noise-to-data paths to the same, consistent clean outputs in single prediction. These approaches excel in few step gen- eration, but have been empirically shown degrade performance as the number of steps increases.
In this work, we analytically show that consistency models are inherently incompatible with multi-step sampling. Specifically, we show that their objective of strictly predicting clean outputs inevitably leads to error accumulation over multiple denoising steps. Motivated by this limitation, we turn to the flow map formulation as a unifying and more robust alternative. The flow map framework also known as Consistency Trajectory Models was introduced in [38, 5] and encompasses diffusion and flow- based models [45], consistency models [71, 69, 49], and other distillation variants [80, 17, 94, 95] within single coherent formulation. Flow maps allow connecting any two noise levels in a single step, enabling efficient few-step sampling as well as flexible multi-step sampling. As flow maps, figuratively speaking, learn a mapping that "aligns the teacher flow" into few-step sampler, we call our approach Align Your Flow (AYF). We propose two new continuous-time training objectives, which can be interpreted as AYF's versions of the Eulerian and Lagrangian losses described by Boffi et al. [5]. The new objectives use consistency condition at either the beginning or the end of denoising interval. Notably, the first of our objectives generalizes both the continuous-timm consistency loss [71, 49] and the flow matching loss [45]. While regular consistency models only perform well for single- or two-step generation and degrade for multi-step sampling, e.g. for 4 steps or more, flow map models such as AYF produce high-quality outputs in this multi-step setting, too.
To scale AYF to high performance, we leverage the recently proposed autoguidance [35], where a low-quality guidance model checkpoint is used together with the regular model to produce model with enhanced quality. Specifically, we propose to distill an autoguided teacher model into an AYF student and introduce several practical techniques that stabilize flow map training and push performance further. Moreover, unlike prior distillation approaches that rely on adversarial training to boost quality at the expense of sample diversity [67, 66, 87, 86, 38], we show that a short finetuning of pretrained AYF model with combination of our proposed flow map objective and adversarial loss is sufficient to yield significantly sharper images with minimal impact on diversity.
We validate AYF on popular image generation benchmarks and achieve of-h--of- performance among few-step generators on both ImageNet 64x64 and 512x512, while using only small and efficient neural networks (Fig. 4). For instance, 4-step sampling of AYF's ImageNet models is as fast or faster than previous works' single step generation. Additionally, our adversarially finetuned AYF also achieves significantly higher diversity compared to other adversarial training approaches. We further distill the popular FLUX..1 1 model [41] and obtain text-to-image AYF flow map models that significantly outperform all existing non-adversarially trained few-step generators in text-conditioned

---

LCM
TCD
AYF (ours)
FLUX.1 [schnell]
LCM
TCD
AYP (ours)
FLUX.1 (achnell
Figure 3: Samples (4 steps): LCM [54], TCD 194]. FLUX.1 [schnell] [41], AYF (view zoomed in).
synthesis (Fig. 1). For these experiments. we use an efficient LoRA [27] framework, avoiding the overhead of many previous sext-to-image distillation approaches.
Contributions. (i) We prove that consistency models inherently suffer from error accumulation in multi-step sampling. (ii) We propose Align Your Flow, a high-performance few-step flow map model with new theoretical insights. (iii) We introduce two training objectives and stabilization techniques for flow map learning. (iv) We apply autoguidance for distillation for the first time and show that adversarial finetuning further boosts performance with minimal loss in diversity. (v) We achieve hate-of-thhe-ar few-step generation performance on ImageNet, and we also show fast high-resolution text-to-image generation, outperforming all non-adversarial methods in this task.
Background
Diffusion Models and Flow Matching. Diffusion models are probabilistic generative models that inject noise into the data with a forward diffusion process and generate samples by learning and simulating a time-reversed backward diffusion process, initialized by pure Gaussian noise. Flow matching [45, 48, 2, 1, 39] is a generalization of these methods that eliminates the requirement of the noise being Gaussian and allows learning a continuous flow between any two distributions po. P1 that converts samples from one to the other.
Denote the data distribution by Xo Pdaa and the noise distribution by X1 Pmoise. Let X =1 t) Xg + f. X1 indicate the noisy samples of the data for time E [0, 1]. corresponding to the rectified flow [48] conditional optimal transport [45] formulation. The flow matching training objective is then given by Exp.x,. [vo())] - (x1 xx) w(t) is a weighting function and Væ is a neural network parametrized by 0. The standard sampling procedure starts l by sampling X1 Puoise- Then the probability flow ODE (PF-ODE), defined by dx. vo(x,, t)dt, is simulated from = 1 to = 0 to obtain the final outputs. We will assume to be in the flow matching framework from this point on of the paper.
de
Consistency Models. Consistency models (CM) [71] train a neural network fx(x,,t) t) to map noisy inputs x₁ directly to their corresponding clean samples xo. following the PF-ODE. Consequently, fx(x,,)) 1) must satisfy the boundary condition fo fx 0) = x, which is typically enforced by parame- terizing fx(x,,)) !) (f)x. + Coun (f)FF)()) (xe, with (0) = 1, Cout (0) = 0. CMs are trained to have consistent outputs between adjacent timesteps. They can be trained from scratch or distilled from given diffusion flow models. In this work, are focusing on distillation. Depending how
time is dealt with, CMs can be split into two categories:
Discrete-time CMs. The training objective is defined between adjacent timesteps
Ex,, r (w(t) t), fo (x--an, At))]
where 8- denotes (f)) w=t)) is a weighting function, At > 0 is the distance between adjacent timesteps, and .(...) is a distance function. Common choices include E2 loss d(x, y) = ||x y l Eseudo-Huber loss d(x,y) y) /||x y11 + c² e [69], and LPIPS loss [90]. Discrete-time CMs are sensitive to the choice of AI, and require manually designed annealing schedules [71, 18). The noisy sample Xt-At at the preceding timestep Át often obtained from x, by numerically solving the PF-ODE, which can cause additional discretization errors.
3

---

Continuous-time CMs. When using d(x = ||x - --lla and taking the limit At - 0, Song et al. [71] show that the gradient of Eq. (1) with respect to converges to
dfg- (xx, ()
Vo
w(t)f) (x,,)) t)
dt
where df. (X..1) dt Xx, fg (Xe, 1) dx +G,f.- (xr, 1) is the tangent of fa- at (xx, t) along the trajectory of the PF-ODE dr This means continuous-timn CMss do not need to rely numerical ODE solvers which avoids discretization errors and offers better supervision signals during training. Recently, Lu and Song [49] successfully stabilized and scaled continuous-time CMs and achieved significantly better results compared the discrete-time approach.
dxs
Continuous-TT Time Flow Map Distillation
Flow maps generalize diffusion, flow-based and consistency models within a single uni- fied framework by training a neural network (,) to map noisy inputs x, directly to any point x, along the PF-ODE in a single step. Unlike consistency models, which only perform well for single- or two-step generation but de- grade in multi-step sampling, flow maps remain effective all step counts.
In Sec. 3.1, we first show that standard consistency models are incompatible with multi-step sampling, leading to inevitable performance degradation beyond certain step count. Next, in Sec. 3.2, we introduce two novel continuous-time objectives for distilling flow maps from pretrained flow model. Finally, in Sec. 3.3, we explain how we leverage autoguidance to sharpen the flow map. Sec. 3.4 addresses implementation details. The detailed training algorithm for AYF is provided in the Appendix.
Figure 4: Two-step AYF samples on ImageNet512.
3.1 Consistency Models are Flawed Multi-Step Generators
CMs are a powerful approach to turn flow-based models into one-step generators. To allow CMs to trade compute for sample quality, a multi-step sampling procedure was introduced by Song et al. [71]. This process sequentially denoises noisy x, by first removing all noise to estimate the clean data and then reintroducing smaller amounts noise. However, in practice, this sampling procedure performs poorly as the number of steps increases and most prior works only demonstrate 1- or 2-step results.
To understand this behavior, we analyze simple case where the initial distribution is isotropic Gaussian with standard deviation c, i.e. Paata (x) .V c² I). The following theorem shows that regardless of how accurate (non-optimal) CM is, increasing the number of sampling steps beyond certain point will lead to worse performance due to error accumulation in that setting.
Theorem 3.1 (Proof in Appendix). Let (x) = N0 e² I) be the data distribution, and let f* (xx,, t) denote the optimal consistency model. For any 8 > 0, there exists suboptimal consistency model f(x;, t) such that
Expxpix [I)((x), (x,, < for all t E [0,
1)12]
and there is some integer N for which increasing the number of sampling steps beyond N increases the Wasserstein-2 distance of the generated samples to the ground truth distribution (i.e. worse approximation of the ground truth).
This suggests that CMs, by design, not suited for multi-step generation. Interestingly, when = 0.5-a common choice in diffusion model training, where the data is often normalized to this std. dev. [34] m34]-multi-step CM sampling with non-optimal CM produces the best samples at two steps (Fig. 5). This is in line with common observations in the literature [49]. This behavior is the opposite of standard diffusion models, which improve as the number of steps increases. Prior works have attempted to address this issue (see Sec. 4), and they all ultimately reduce to special cases of flow maps.

---

3.2
Learning Flow Maps
Flow maps are neural networks fx(,,, t, 8) that general- ize CMs by mapping noisy input directly to any other point x, by following the PF-ODE from time 0, they reduce to standard CMs.
2ocesed Viea
to When
When performing many small steps, they become equiv- alent to regular flow diffusion model sampling with the PF-ODE and Euler integration. A valid flow map fo,,,, s) must satisfy the general boundary condition fx(,,,t, = X2 for all f. As is done in prior work, this enforced in practice by parameterizing the model fo(xe,6, 8, = Cakip(,, s)x; + Coun 8) 8) Fx( 8) where (Lip(1, and Cour(,, t) 0 for all L. In this work, we set Cakip (f, s) = 1 and Cout (t, s) = (s t) for simplicity and to align with an Euler ODE solver.
Wassersteln
0073
20t
Number 50 sampling steps
Figure 5: Wasserstein-2 distance be- tween multi-step consistency samples and data distribution (c=0.5).
Unlike CMs, which perform poorly in multi-step sampling, flow maps are designed to excel in this scenario. Additionally, their ability to fully traverse the PF-ODE enables them to accelerate tasks such as image inversion and editing by directly mapping images to noise [43].
As are interested in distilling diffusion or flow matching model, we assume access pretrained velocity model (x( (). The flow map model is trained by aligning its single-step predictions with the trajectories generated by the teacher's s PF-ODE, ie. dx, V (X,, t). We propose two primary methods for training flow maps. The first training objective aims to ensure that for fixed s, the output of the flow map remains constant as we move (x,, t) along the PF-ODE. Let 6- = stopgrad (0). The theorem below summarizes the approach. We call this loss AYF-Eulerian Map Distillation (AYF- EMD), as it can also be interpreted as a variant of the Eulerian loss of Boffi et al. [5]. The AYF-EMD loss naturally generalizes the loss used train continuous-time consistency models [71, 49], as it reduces to the same objective when = O. Interestingly, it also generalizes the standard flow matching loss, to which it reduces in the limit
de
I. See Appendix for details.
Theorem 3.2 (Proof in Appendix). Let tt be the flow map. Consider the loss function defined between two adjacent starting timesteps and = c(s t) for small
Ex,t, [w(t, ()) t,,) - (xv, iet i, s))]
where Xy is obtained by applying a I- 1-step Euler solver to the PF-ODE from t to t'. In the limit as c-> 0, the gradient of this objective with respect to 9 converges to:
df. (xe, t, s)
VØEx. w s)sign(t -s) 8) -fg (x),t,s).
dt
where w st) = w(t, s) s/.
The second approach ensures consistency at timestep s instead. This method tries to ensure that for a fixed t),,t, t), the trajectory fo (x,, f, is aligned with that points' PF-ODE. We call this loss AYF-Lagrangian Map Distillation (AYF-LMD), as is related to the Lagrangian loss of Boffi et al. [5]. The theorem below formalizes this approach.
Theorem 3.3 (Proof in Appendix). Let fo(x,,t, t, s) be the flow map. Consider the loss function defined between two adjacent ending timesteps and s' = + c(t for small >> 0,
Exi, [w[(, (x),))()),1,s)) - ODE. (x,x 1))1)
where ODF refers to running a 1-step Euler solver on the PF-ODE starting from x at timestep t to timestep In the limit as e -> 0, the gradient of this objective with respect to O converges to:
s) (dfy) ds
VoEx w' (1, () sign(s - 1) fg (,),
vo (xi, (),,
where w st) = wit, 8) x - s).
In our 2D toy experiments, comparing the two objectives above, we found the AYF-LMD objective to be more stable. However, when applied to image datasets, it leads to overly smoothened samples that drastically reduce the output quality (see Appendix for detailed ablation studies).
5

---

3.3 Sharpening the Distribution with Autoguidance
The training objective of diffusion- and flow-based models strongly encourages the model to cover the entire data distribution. Yet it lacks enough data to learn how to generate good samples in the tails of the distribution. The issue is even worse in distilled models which use fewer sampling steps. As result, many prior distillation methods rely on adversarial objectives to achieve peak performance, often sacrificing diversity and ignoring low-probability regions altogether. The most commonly used technique to partially address this in conditional diffusion and flow-based models is classifier-free guidance (CFG) [24]. CFG trains a flow or diffusion model for both conditional and unconditional generation and steers samples away from the unconditional regions during sampling. Prior works [57, 49] have explored distilling CFG with great success. However, CFG struggles with overshooting the conditional distribution large guidance scales, which leads to overly simplistie samples [40].
Recently, Karras al. [35] introduced autoguidance as better alternative for CFG. Unlike CFG, this technique works for unconditional generation well. Autoguidance uses smaller, less trained version of the main model for guidance, essentially steering samples away from low-quality sample regions in the probability distribution, where the weaker guidance model performs particularly poorly, We found that distilling autoguided teacher models can significantly improve performance compared to standard CFG. To the best of our knowledge, autoguided teachers. Specifically, during flow map distillation we define the guidance scale À and use the autoguided teacher velocity guided
the first demonstrate the distillation of
X t) = Xv t) + (1 A)v weak (xx, t),
where va represents the weaker guidance model. summary, we use autoguidance in the teacher mechanism to "sharpen" the distilled flow map model. See Appendix for a visual comparison between autoguidance and CFG on a 2D toy distribution.
3.4 Training Tricks
Training continuous-time CMs has historically been unstable [69, 18]. Recently, sCM [49] addressed this issue by introducing techniques focused on parameterization, network architectures, and mod- ifications to the training objective. Following their approach, we stabilize time embeddings and apply tangent normalization, while also introducing few additional techniques further improve stability.
Our image models are trained with the AYF-EMD objective in Theorem 3.2, which relies on the tangent function d (x.(.a) Under our paramctrization, this tangent function is computed by
dfg (x,,, ( [xx: - -x, + (s t) dt
dF, tt dt
(4)
dt
where dx, = (x,, t) represents the direction given by the pretrained diffusion or flow model along the PF-ODE. We find that most terms in this formulation are relatively stable, except for (F.(...(.)) = Vx, Fe (xe, s) dx, + +x,Fo(x,,t dr
dt
s). tt). Among these, the instability originates mainly from F, 8), which can be decomposed into
t,
de
0.F 4,8) Cenose (t)
demb( (roise) OF,
Øt
0Cyoise
Jemb'
where emb(.) refers to the time embeddings. most commonly in the form of positional embed- dings [25, 78] or Fourier embeddings [70, 73]. SCM [49] proposes several techniques to stabilize this term including tangent normalization, adaptive weighting, and tangent warmup.
We use tangent normalization [49], i.e. dfg-
/ -f
c) with c 0.1, as find it to be critical for stable training. However, in our experiments, adaptive weighting had no meaningful impact and can be removed. We make a few tweaks to the time embeddings and tangent warmup to ensure compatibility with flow matching and better training dynamics which we describe below.
Stabilizing the Time Embeddings The time embedding layers are one of the causes for the instability of O F, f, s). As noted in [49], the Cnoise parameterization used in most CMs is based on the EDM [34] framework, where the noise level is defined as Croise (a) = log(o). In the flow matching framework, which we use, the noise level for timestep t is given by d, م-z- which can lead to
instabilities when passing through log operation as 0 or t + 1. To address this, we modify the

---

time parameterization by setting Cuoise = t, ensuring stable partial derivatives. To utilize pretrained teacher model checkpoints trained with different time parameterizations, we first finetune the student's time embedding module to align with the outputs of the original checkpoints. For example, if we want to adapt EDM2 checkpoints, which use a = 1-1- we minimize the following objective:
Be)p(t) |lemb (t) embariginal (log (ot))1
This approach enables us to re- re-pupose nearly any checkpoint, making it compatible with our flow matching framework with minimal finetuning. rather than training new models from scratch.
Regularized Tangent Warmup We initialize the student model with pretrained flow matching or diffusion model weights, following prior work to speed up training [49, 71J. Lu and Song [49]
proposed a gradual warmup procedure for the second term in Eq. (4), i.e.. dfg- (xx,,f.s) dr Specifically, they introduced a coefficient r' that linearly increases from 0 to over the first 10k training iterations, gradually incorporating the term. This warmup has clear intuitive motivation. When considering only the first term in Eq. (4) (i.e., the r = O case), the objective simplifies to regularization term that encourages flow maps to remain close straight lines (please see the Appendix for the derivation):
V sign(t s) 5) s)) x dxx F tx),t
x V Fx t, s) VV 1) (5)
dt
Therefore, for r < 1, the warmed-up loss with coefficient r is equivalent to a weighted sum of the actual loss and this regularization term:
dxx dt
Fo (x,,t, r(s dF (x,, t)
Vo sign(t 5) tx),t,s
dt
r sign(t f)f t, s) dfp (x) + +
||t s|-1| |Foo t,, vo(xe, 1)
ds
In our experiments, training these models for too long after the warmup phase can cause destabiliza- tion. A simple fix is to clamp found rmax = 0.99 to be effective in all cases.
value smaller than 1, ensuring some regularization remains. We
Timestep scheduling As in standard diffusion, flow-based, and consistency models, selecting an effective sampling schedule for (f) 8) during training is crucial. Similar to standard consistency mod- over training, flow map models propagate information from small intervals Is I = O to large ones |s - 4 1. For details on our practical implementation of the schedules, as well as complete training algorithms, please see the Appendix.
els, where information must propagate from
to
Related Work
Consistency Models. Flow Map Models generalize the seminal CMs. introduced by Song et al. [71]. Early CMs were challenging to train and several subsequent works improved their stability and performance, using new objectives [69]. weighting functions [18] variance reduction techniques [79], among other tricks. Truncated CMs [42] proposed a second training stage, focusing exclusively on the noisier time interval, and Lu and Song [49] successfully implemented continuous-time CMs for the first time.
Flow Map Models. Consistency Trajectory Models (CTM) [38] can be considered the first flow map-like models. They combine the approach with adversarial training. Trajectory Consistency Dis- tillation [94] extends CTMs to text-to-image generation, and Bidirectional CMs [43] train additionally on timestep pairs with (<8,, also accelerating inversion and tasks such as inpainting and blind image restoration. Kim et al. [37] trained CTMs connecting arbitrary distributions. Multistep CMs [22] split the denoising interval into sub-intervals and train CMs within each onc, enabling impressive gener- ation quality using 2-8 steps. Phased CMs [80] use similar anterval-splitiing strategy combined with an adversarial objective. These methods can be seen as learning flow maps by training on (t, s) pairs, where is the start of the sub-interval containing t. Flow Map Matching [5] provides rigorous analy- sis of the continuous- time flow map formulation and proposes several continuous-tim losses. Shortcut models [17] adopt similar flow map framework, but these two works struggle to produce high-quality images in contrast to our novel AYF, the first nigh-performance continuous timm flow map model.
Accelerating Diffusion Models. Farly diffusion distillation approaches are knowledge distillation [52] and progressive distillation [63, 57]. Other methods include adversarial distil- lation [67, 66), variational score distillation (VSD) [87, 86), operator learning [93] and further
7

---

Table 1: Sample quality on class- conditional ImageNet 64x64, Recall met- ric is also included.
Table 2: Sample quality on conditioal ImageNet 512x512. For additional baselines, which AYS all out- performs, please see the Appendix.
Metioal
SPR.G FID Beaal(l)
Method
NFE (
FID (
4Params
Gllups
Time (x)
Teacher Diffacion Model
KIN
EDM2-S [26]
63x2
2.23
280M
12852
$.31
1.22
EDM2-XXL [36]
63x2
1.81
1.58
69552
31.50
KDN (loan) (4))
244
EDME S AAlloguid [35]
63x2
1.34
280M
12852
8,31
IEME + Autepalaee (25]
43
141
0.99
EDM2-XXL Autoguid. [35]
63x2
1.25
1.5B
69552
Adverartal Juiet Transing
Adversarial & Johnt Trahning
Soyerio XI (65)
1.53
SIDA-S [97] thest adv. baseline)
1.69
280M
102
0,06
AYF-S adv, loss (ours)
1.92
280M
102
0.06
DMB02
11400 (87)
2,47
1.81
280M
204
0.12
1.52
1.64
260M
0.24
CTM
1922
Consitencc Distillation
Soonced Maccnng (14]
3001
sCD&S (49)
3.07
280M
CLIEE-1 1921
1.6
1000
2
2.50
280M
204
0.13
AYT aait bast (Aar))
1.30
SCD-M [49]
2.75
498M
181
0.10
1:00
sCD-L [49]
2
2.55
2.26
498M
776M
362
282
0.14
0.20
IMilveies Destibation withool Advevenaa (Reechro
2,04
776M
564
0:53
051
sCD-XL [19]
1.15
FED [14]
2
1.93
1.1B
812
0.38
740
sCD XXL [49]
1
2.28
1.5B
552
0.25
1633
2
1.88
1.5B
1101
0.50
(akapl Fro (20)
6.70
AYF-S (owrx)
1
3.32
260M
102
0.06
CD [71]
6.3
1.87
280M
1.70
280M
JCD (49)
2466
Consistency Training
AYF S4ary
2406
045
SCT-S [49]
10.13
280M
102
0.06
115
2
9.85
280M
204
0.13
Comsiteecy Trideias
sCT-M [49)
2
5,53
5.84
496M
498M
362
181
0.20
0.10
ICT
402
SCT-- (49)
77HM
ECT
4.65
776M
(47(19]
SCT-XL [49]
4,33
1.18
0.19
2.00
2
3.73
1.18
812
0.38
2,0
SCT-XXI. [49]
2
1
4.29
3.76
1.58
1.58
1104
552
0.25
0.50
techniques [55, 20, 4, 74, 85, 48, 82, 46 98, 96], many of them relying on adversarial losses, too. However, although popular, adversarial methods introduce training complexities due to their GAN-like objectives. VSD exhibits similar properties and does not work well high guidance levels. Moreover, these methods can produce samples with limited diversity. For these reasons we avoid such objectives and instead rely on autoguidance to achieve crisp high-quality outputs. Finally, many training-free methods efficiently solve diffusion models' generative differential equa- tions (50, 32, 34, 12, 89, 61], but they are unable perform well when using < <10 generation steps.
5 Experiments
We train AYF flow maps on ImageNet [10] at resolutions 64 x 64 and 512 x 512, measuring sample quality using Fréchet Inception Distance (FID) [23], as previous works. We also use our AYF frame- work to distill FLUX.1 [dev] [41], the best text-to-image diffusion model, using efficient LoRA [27] framework and reduce sampling steps to just 4. Experiment details explained in the Appendix.
ImageNet Flow Maps. We adopt the EDM2 [36] framework, using their small "S" models, and modify network parametrization and time embedding layer as detailed in Sec. 3. Pretrained checkpoints available online are used both as teacher network and as flow map initialization. We incorporate autoguidance into the flow map model by introducing additional input, X, corresponding to the guidance scale [49, 57]. During training, is uniformly sampled from [1,3] and applied to the teacher model via autoguidance. At inference, we leverage the y-sampling algorithm from [38] for stochastic multistep sampling of flow map models. Results are reported using the opti- mal y and À values. For ImageNet 512 x 512, the teacher and distilled models are in latent space [60].
208 Jauno
3.0
2.5
2.0
15
025
015 0.20 0.23 0.30 0.35
(5)
(a) ImageNet-64
(b) ImageNet-511
Figure 6: FID + as function of wall clock time.
In Tab. we show ImageNet 64 x 64 results, reporting FID and recall scores along with number of neural function evaluations (NFE). Our flow maps achieve the best sample quality among all
8

---

non-adversarial few-step methods, given only 2 sampling steps by sacrificing optimal 1-step quality. This is because learning a flow map is a more challenging task compared to only consistency model. In Tab. 2 we compare AYF against the state-of-the-an consistency model sCD/sCT [49] on ImageNet 512 x 512, also reporting total wall sampling clock time, Gflops, and #parameters. We show that although our small-sized model achieves slightly worse one step sample quality, it is on par with the best sCD model at only two steps while using only 18% of the larger models' compute. Increasing the sampling steps to four improves the quality even further while still being over twice as fast as the large I-step sCM model (wallclock time). We further analyze the performance vs. sampling speed trade-off in Fig. 6, showing that AYF is much more efficient than sCD/sCT (also see Appendix for additional comparison). Autoguidance allows AYF to use a small network and still achieve strong performance and the efficient network results in 2-step or 4-step synthesis still being lightning fast.
Ad Adversarial finetuning of AYE. Given a pretrained AYF flow map model, we found that a short finetuning stage using combination of the EMD objective and an adversarial loss can significantly boost the performance across the board, especially for 1-step generation, with a minimal impact to sample diversity measured by recall scores. Using this approach, we achieve state-of-the-ar performance on few-step generation on ImageNet64 (see Tab. 1). For implementation details, please see Appendix. Additional GAN and diffusion model baselines on ImageNet 512 x 512 can be found in the Appendix; AYF outperforms all of them.
Text-to-Image Flow Maps. We apply AYF to distill the open- text-to-image model FLUX.1 [dev] [41] into few-step generator, finetuning a FLUXX.1 1 base model into a flow map model using LoRA [27] with the objective in Theorem 3.2 for 10,000 steps. This distillation process took approx. four hours on 8 NVIDIA A100
Text Alignment
0.7
0.525
0.35
0.75
GPUs, which is highly efficient, in contrast to several previous large-scale text-to-image distillation methods,
ICM
TCD
AYF (rum)
Ti:
Samples from the model are shown in Fig. 1. We compare LCM [53, 54] and TCD [94]. two consistency-distile LoRAs trained on top of SDXL [59] without adversarial objectives. To evaluate quality we ran a user study. The results (Fig. 7) show clear preference for our method. We also provide qualitative comparisons in Fig. 3. Compared to LCM and TCD, our images are more aesthetically pleasing with finer details. We also included FLUX.1 [schnell] [41]. commercially distilled model trained with Latent Adversarial Diffusion Distillation [66]. Our method achieves comparable image quality to the [schnell] model, while requiring only four sampling steps and 32 GPU hours without the use adversarial losses. In conclusion, AYF achieves state-of-the few- step text-to-image generation performance among non-ad von-aarersarial methods. Detailed ablation studies different components of AYF (EMD vs. LMD; autoguidance vs. CFG, AYF vs. Shorteut) are presented in the Appendix. Additional qualitative examples of images generated by AYF are shown in the Appendix, too.
Visual Fidelity
0.775
AYF
Figure 7: User preferences com- paring LoRA-based consistency and flow map models (4-step sam- ples). LCM and TCD use SDXL and AYF uses FLUX.1 I [dev] as base model, respectively.
6 Conclusions
We have presented Align Your Flow (AYF), a novel continuous-time distillation method for training flow maps, which generalizes flow-matching and consistency- -based models. Importantly, flow maps remain effective generators across all denoising step counts, unlike standard consistency models; a fact we prove analytically for the first time.
In addition, we use autoguidance to enhance the quality of the teacher model, resulting in an improved distilled student, and an additional boost can be achieved by adversarial finetuning, with minimal impact in sample diversity. We achieve of-the art performance among aoversreraliaal distillation methods on both ImageNet64 and ImageNet512 generation. Since AYF requires only relatively small neural networks, which further reduces the computational burden and boosts sampling efficiency, even 2-step or 4-step sampling from AYF is as fast or faster than previous single-step generators. We also distill FLUUX.1 1 using an efficient LoRA framework, resulting in thce----the-rtt text-to-image generation performance among non-adversarial distillation approaches.

---

Future work could explore applying AYF to video model distillation or in other domains, for instance in drug discovery for efficient molecule or protein modeling.
References
[1] Michael S. Albergo, Nicholas M. Boffi, and Eric Vanden Eijnen. Stochastic interpolants: unifying framework for flows and diffusions. arXiv preprint arXi::2303.0879)) 2023.
[2] Michael Samuel Albergo and Eric Vanden-Eijnden. Building normalizing flows with stochastic interpolants. In The Eleventh International Conference on Learning Representations (ICLR),
2023.
[3] Fan Bao, Shen Nie, Kaiwen Xue, Yue Cao, Chongxuan Li, Hang Su, and Jun Zhu, All are worth words: A vit backbone for diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 22669-22679, 2023.
[4] David Berthelot, Arnaud Autef, Jierui Lin, Dian Ang Yap. Shuangfei Zhai, Siyuan Hu, Daniel Zheng, Walter Talbott, and Eric Gu. Tract: Denoising diffusion models with transitive closure time-distillation. arXiv preprint arXiv:2303.0444 04444444 2023.
[5] Nicholas M Boffi, Michael S Albergo, and Eric Vanden- Eijeen-Eijnden. Flow map matching. arXiv preprint arXiv:2406.07507, 2024.
[6] Nicholas M Boffi, Michael S Albergo, and Eric Vanden- How build a consistency model: Learning flow maps via self-distillation. arXiv preprint arXiv:2.005. 18825, 2025.
[7] Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale gan training for high fidelity natural image synthesis, 2019.
[8] Tim Brooks, Bill Peebles, Connor Homes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Wing Yin Ng, Ricky Wang, and Aditya Ramesh. Video generation models as world simulators. 2024. URL https: https://openas com com/ res//ereeacch// video cadeo--gneeationn mooeels-mooo----oo as-world-w-d-------------------- simulators.
[9] Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T Freeman. Maskgit: Masked generative image transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13315-11325. 2022.
[10] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248-255, 2009. doi: 10. (DW.CCCCRR.000050666666
[11] Prafulla Dhariwal and Alexander Quinn Nichol. Diffusion Models Beat GANs on Image Synthesis. In Advances in Neural Information Processing Systems, 2021.
[12] Tim Dockhorn, Arash Vahdat, and Karsten Kreis. GENIE: Higher- Denoising Diffusion Solvers. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
[13] Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages
2021.
[14] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, Kyle Lacey, Alex Goodwin, Yannik Marek, and Robin Rombach. Scaling rectified flow transformers for high-resolution image synthesis. In International Conference on Machine Learning (ICML), 2024.
[15] Zhengcong Fei, Mingyuan Fan, Changqian Yu, and Junshi Huang, Scalable diffusion models with state space backbone. arXiv preprint arXiv:2402.05608 2024.
[16] Zhengcong Fei, Mingyuan Fan, Changqian Yu, Debang Li,, and Junshi Huang. Diffusion-rwkv: Scaling rwky-like architectures for diffusion models. arXiv preprint MxXix:2404.04478 2024.
10

---

[17] Kevin Frans, Danijar Hafner, Sergey Levine, and Pieter Abbeel. One step diffusion via shortcut models. ArXiv, abs/2410.12557, 2024.
[18] Zhengyang Geng, Ashwini Pokle, William Luo, Justin Lin, and J Zico Kolter. Consistency models made easy. arXiv preprint arXiv:2406. 14044488..4448, 2024.
[19] Zhengyang Geng, Mingyang Deng, Xingjian Bai, Zico Kolter, and Kaiming He. Mean flows for one-step generative modeling. arXiv preprint arXiv:2505.13447, 2025.
[20] Jiatao Gu, Shuangfei Zhai, Yizhe Zhang, Lingjie Liu, and Joshua M Susskind. Boot: Data-free distillation of denoising diffusion models with bootstrapping. In ICML 2023 Workshop on Structured Probabilistic Inference & Generative Modeling. 2023.
[21] Ali Hatamizadeh, Jiaming Song, Guilin Liu, Jan Kautz, and Arash Vahdat. Diffit: Diffusion vision transformers for image generation. In European Conference on Computer Vision, pages 37-55. Springer, 2024.
[22] Jonathan Heek, Emiel Hoogeboom, and Tim Salimans. Multistep consistency models. arXiv preprint arXiv:2403.06887,, 2024.
[23] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Guyon, U. Von Luxburg. S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
[24] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint 125552 2022.
[25] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
[26] Emiel Hoogeboom, Jonathan Heek, and Tim Salimans. simple diffusion: End-to-end diffusion for high resolution images. In International Conference Machine Learning, pages 13213- 13232. PMLR, 2023.
[27] J. Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference Learning Representations (ICLR), 2022.
[28] Nick Huang, Aaron Gokaslan, Volodymyr Kuleshov, and James Tompkin. The gan dead; long live the gan! modern gan baseline. Advances in Neural Information Processing Systems, 44215,, 2024.
[29] A. Jabri, David J. Fleet, and Ting Chen. Scalable adaptive computation for iterative generation. In International Conference Machine Learning, 2022.
[30] Jackyhate. fext-to-image-2m. 2025. text-to-image 1o0-immaee220 2M.
[31] Alexia Jolicoeur-Maatnn Martineau. The relativistic discriminator: a key element missing from standard gan. arXiv preprint arXiv:1807.00734 2018.
[32] Alexia Jolicoeur-Martinn Ke Li, Rémi Piché-Taillefés Tal Kachman, and loannis Mitliagkas. Gotta go fast when generating data with score-based models. arXiv preprint arXiv:2 /4080, 2021.
[33] Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of stylegan. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8110-8118 8199, 2020.
[34] Tero Karras, Miia Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. Advances in neural information processing systems, 35: 26565-26577, 2022.
11

---

[35] Tero Karras, Mika Aittala, Tuomas Kynkäääniemii, Jaakko Lehtinen, Timo Aila, and Samuli Laine. Guiding diffusion model with bad version of itself. arXiv preprint arXiv:2406. 02507. 2024.
[36] Tero Karras, Miika Aittala, Jaakko Lehtinen, Janne Hellsten, Timo Aila, and Samuli Laine. Analyzing and improving the training dynamics of diffusion models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 24174-24184, 2024.
[37] Beomsu Kim, Jaemin Kim, Jeongsol Kim, and Jong Chul Ye. Generalized consistency trajectory models for image manipulation. arXiv preprint arXiv:2403. /2500, 2024.
[38] Dongjun Kim, Chieh- Lai, Wei-Hsiang Liao, Naoki Murata, Yuhta Takida, Toshimitsu Uesaka, Yutong He, Yuki Mitsufuji, and Stefano Ermon. Consistency trajectory models: Learning probability flow ode trajectory of diffusion. ArXiv, abs/2310.02279, 2023.
[39] Diederik P Kingma and Ruiqi Gao. Understanding diffusion objectives as the ELBO with simple data augmentation. In Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS), 2023.
[40] Tuomas Kynkäänniemi, Miika Aittala, Tero Karras, Samuli Laine, Timo Aila, and Jaakko Lehtinen. Applying guidance in limited interval improves sample and distribution quality in diffusion models. arXiv preprint arXiv:2404.07722, 2024.
[41] Black Forest Labs. Flux, https:///ithi: http://ithuu..cmmmmal//.l http://llack-foset--eeeeeee waa--aaassfffaa /flux 2023.
[42] Sangyun Lee, Yilun Xu, Tomas Geffner, Giulia Fanti, Karsten Kreis, Arash Vahdat, and Weili 14895, Nie. Truncated consistency models. arXiv preprint arXiv:2410. (4.8955, 2024.
[43] Liangchen Li and Jiajun He. Bidirectional consistency models. arXiv preprint arXiv:2403.18035, 2024.
[44] Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. Autoregressive image generation without vector quantization. arXiv preprint arXiv:2406. 1838, 2024.
[45] Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747, 2022.
[46] Hongjian Liu, Qingsong Xie, Zhijie Deng, Chen Chen, Shixiang Tang, Fueyang Fu, Zheng jun Zha, and Haonan Lu. Scott: Accelerating diffusion models with stochastic consistency distillation. arXiv preprint arXiv:2243.01 505, 2024.
[47] Qihao Liu, Zhanpeng Zeng, He, Qihang Yu, Xiaohui Shen, and Liang-Chiel Chich Chen. Alle- viating distortion in image generation via multi-resolution diffusion models. arXiv preprint arXiv:2406.00 09416, 2024.
[48] Xingchao Liu, Chengyue Gong, and qiang liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In The Eleventh International Conference on Learning Representations (ICLR), 2023.
[49] Cheng Lu and Yang Song. Simplifying, stabilizing and scaling continuous-time consistency models. arXiv preprint arXiv:2410. 1081, 2024.
[50] Cheng Lu, Yuhao Zhou, Fan Bao. Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems, 35:5775-5787, 2022.
[51] Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm- solver++: Fast solver for guided sampling diffusion probabilistic models. arXiv preprint arXiv:2211.01095 2022.
[52] Eric Luhman and Troy Luhman. Knowledge distillation in iterative generative models for improved sampling speed. arXiv preprint arXiv:2101.02388 2021.
[53] Simian Luo, Yigin Tan, Longbo Huang, Jian Li, and Hang Zhao. Latent consistency models: Synthesizing high-resolution images with few-step inference. ArXiv, abs/23 10.0438, 2023.
12

---

[54] Simian Luo, Yiqin Tan, Suraj Patil, Daniel Gu, Patrick von Platen, Apolin' ario Passos, Longbo Huang, Jian Li, and Hang Zhao. Lem-lora: A universal stable-diffusion acceleration module. ArXiv, abs/2311 .05556, 2023.
[55] Weijian Luo, Tianyang Hu, Shifeng Zhang, Jiacheng Sun, Zhenguo Li, and Zhihua Zhang, Diff- instruct: A universal approach for transferring knowledge from pre-trained diffusion models. In seirth-e Conference on Neural Information Processing Systems, 2023.
[56] Nanye Ma, Mark Goldstein, Michael S Albergo, Nicholas M Boffi, Eric Vanden-Eijnden, and Saining Xie. Sit: Exploring flow and diffusion-based generative models with scalable interpolant transformers. arXiv preprint arXiv:2401.085: .08740, 2024.
[57] Chenlin Meng, Robin Rombach, Ruiqi Gao, Diederik Kingma, Stefano Ermon, Jonathan Ho, and Tim Salimans. On distillation of guided diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14297-14306, 2023.
[58] William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4195-4205, 2023.
[59] Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01995 2023.
[60] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- Resolution Image Synthesis with Latent Diffusion Models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), 2022.
[61] Amirmojtaba Sabour, Sanja Fidler, and Karsten Kreis. Align your steps: Optimizing sampling schedules in diffusion models. arXiv preprint arXiv:2404. 14507, 2024.
[62] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L. Denton, Seyed Kamyar Seyed Ghasemipour, Bureu Karagol Ayan, Seyedeh Sara Mahdavi, Raphael Gontijo Lopes, Tim Salimans, Jonathan Ho, David J. Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusion models with deep language understanding. ArXiv, abs/2205.11487, 2022.
[63] Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. arXiv preprint arXiv:2202.005112 2022.
[64] Tim Salimans, Thomas Mensink, Jonathan Heek, and Emiel Hoogeboom. Multistep distillation of diffusion models via moment matching. The Thirty--ighth Annual Conference on Neural Information Processing Systems, 2024.
[65] Axel Sauer, Katja Schwarz, and Andreas Geiger. Stylegan-xl: Scaling stylegan to large diverse datasets. In ACM SIGGRAPH 2022 conference proceedings, pages 1-10, 2022.
[66] Axel Sauer, Frederic Boesel, Tim Dockhorn, Andreas Blattmann, Patrick Esser, and Robin Rombach. Fast high-resolution image synthesis with latent adversarial diffusion distillation. In SIGGRAPH Asia 2024 Conference Papers, pages 1-11, 2024.
[67] Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion distillation. In European Conference on Computer Vision, pages 87-103. Springer, 2025.
[68] Jimming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXi::2010.025002 2020.
[69] Yang Song and Prafulla Dhariwal. Improved techniques for training consistency models. arXiv preprint arXiv:2310. 10.,119,,, 2023.
[70] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-Based Generative Modeling through Stochastic Differential Equations. In International Conference on Learning Representations (ICLR), 2021.
[71] Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models, arXiv preprint 2303002300 2023.
13

---

[72] Yanke Song, Jonathan Lorraine, Weili Nie, Karsten Kreis, and James Lucas. Multi-student diffusion distillation for better one-step generators. arXiv preprint arXhv:2410.23274 2024.
[73] Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Fridovich-Kein Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimensional domains. Advances neural information processing systems, 33:7537 2020.
[74] Joshua Tian Jin Tee, Kang Zhang, Hee Suk Yoon, Dhananjaya Nagaraja Gowda, Chanwoo Kim, and Chang D. Yoo. Physics informed distillation for diffusion models. arXiv preprint arXiv:241 1.08378, 2024.
[75] Yao Teng, Yue Wu, Han Shi, Xuefei Ning, Guohao Dai, Yu Wang, Zhenguo Li, and Xihui Liu. Dim: Diffusion mamba for efficient high-resolution image synthesis. arXiv preprint arXiv:2405.14224, 2024.
[76] Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. Visual autoregressive mod- eling: Scalable image generation via next-scale prediction. arXiv preprint arXiv:2404.02905, 2024.
[77] Yuchuan Tian, Zhijun Tu, Hanting Chen, Jie Hu, Chao Xu, and Yunhe Wang. U-dits: Down- sample tokens in u-shaped diffusion transformers. arXiv preprint arXiv:2405.027555 2024.
[78] A Vaswani. 2017.
Attention is all you need. Advances in Neural Information Processing Systems,
[79] Fu- Yun Wang, Zhengyang Geng, and Hongsheng Li. Stable consistency tuning: Understanding and improving consistency models. arXiv preprint arXiv:2410. 18958, 2024.
[80] Fu- Yun Wang. Zhaoyang Huang, Alexander William Bergman, Dazhong Shen, Peng Gao, Michael Lingelbach, Keqiang Sun, Weikang Bian, Guanglu Song. Yu Liu, Xiaogang Wang, and Hongsheng Li. Phased consistency models. arXiv preprint arXiv:2405. 18407, 2024.
[81] Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Pro- lifiedreamer: High-fidelity and diverse text-to-3d generation with variational score distillation.
Advances in Neural Information Processing Systems, 36:8406-8444 8441, 2023.
[82] Sirui Xie, Zhisheng Xiao, Diederik Kingma, Tingbo Hou, Ying Nian Wu, Kevin P Murphy, Tim Salimans, Ben Poole, and Ruiqi Gao. Em distillation for one-step diffusion models. Advances in Neural Information Processing Systems, 37:45073-45104, 2024.
[83] Yilun Xu, Gabriele Corso, Tommi Jaakkola, Arash Vahdat, and Karsten Kreis. Disco-diff: Enhancing continuous diffusion models with discrete latents. In International Conference on Machine Learning, 2024.
[84] Jing Nathan Yan, Jiatao Gu, and Alexander M Rush. Diffusion models without attention. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8229, 2024.
[85] Ling Yang. Zixiang Zhang, Zhilong Zhang, Xingchao Liu, Minkai Xu, Wentao Zhang, Chenlin Meng, Stefano Ermon, and Bin Cui. Consistency flow matching: Defining straight flows with velocity consistency. arXiv preprint 2mXiv:2407.023998 2024.
[86] Tianwei Yin, Michaël Gharbi, Taesung Park, Richard Zhang, Eli Shechtman, Fredo Durand, and William T Freeman. Improved distribution matching distillation for fast image synthesis. arXiv preprint arXiv:2405.1 0.0000/./466. 4467, 2024.
[87] Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Fredo Durand, William Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6613 6623, 2024.
[88] Lijun Yu, José Lezama, Nitesh B Gundavarapu, Luca Versari, Kihyuk Sohn, David Minnen, Yong Cheng, Vighnesh Birodkar, Agrim Gupta, Xiuye Gu, et al. Language model beats diffusion-tokemizer is key to visual generation. arXiv preprint 271002231. 2023.
14

---

[89] Qinsheng Zhang and Yongxin Chen. Fast sampling of diffusion models with exponential integrator. arXiv preprint arXiv:220..13002 2022.
[90] Richard Zhang, Phillip Isola, Alexei Efros, Eli Shechtman, and Oliver Wang. The unrea- sonable effectiveness of deep features perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586-595, 2018.
[91] Wenliang Zhao, Lujia Bai, Yongming Rao, Jie Zhou, and Jiwen Lu. Unipe: A unified predictor- corrector framework for fast sampling of diffusion models. Advances in Neural Information Processing Systems, 36:49842-49869, 2023.
[92] Bowen Zheng and Tianming Yang. Diffusion models are innate one-step generators. arXiv preprint arXiv:2405.20770, 2024.
[93] Hongkai Zheng, Weili Nie, Arash Vahdat, Kamyar Arizzadenesheli, and Anima Anandkumar. Fast sampling of diffusion models via operator learning. In International conference machine learning, pages 42490-42402. PMLR, 2023.
[94] Jianbin Zheng, Minghui Hu, Zhongyi Fan, Chaoyue Wang, Changxing Ding, Dacheng Tao, and Tat-Jen Cham. Trajectory consistency distillation, 2024.
[95] Linqi Zhou, Stefano Ermon, and Jiaming Song. Inductive moment matching. arXiv preprini arXiv:2503.07565, 2025.
[96] Mingyuan Zhou, Zhendong Wang, Huangjie Zheng, and Hai Huang. Long and short guidance score identity distillation for one-step text-to-image generation. ArXiv 2406.01561, 2024.
[97] Mingyuan Zhou, Huangjie Zheng, Yi Gu, Zhendong Wang, and Hai Huang. Adversarial score identity distillation: Rapidly surpassing the teacher in one step. arXiv preprint 1rxXiv:4400.49111 2024.
[98] Mingyuan Zhou, Huangjie Zheng, Zhendong Wang. Mingzhang Yin, and Hai Huang. Score identity distillation: Exponentially fast distillation of pretrained diffusion models for one-step generation. In International Conference on Machine Learning, 2024.
15

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We summarize the contributions our work in the introduction and the abstract. We conduct comprehensive experiments and analytical derivations support all claims and proofs.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We discuss the limitations of our approach in both the experiments section and the conclusion, and include dedicated section this in the Appendix.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full of assumptions and complete (and correct) proof?
16

---

Answer: [Yes]
Justification:
We give complete proofs for our theoretical results in our appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We share all implementation details needed to replicate our results in the method section and the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
17

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We plan publicly release code upon publication. Together with the implementation details given in the paper, our results can be reproduced.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm
irriiisssiooPolii onPolicy) for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / //tppp.ccpppliiiiimmmm/ wwtppccpppc///c/ct.ccccccccccclt wwtpc//gg///eeeeeeceeoeee
wwwiibmmsssioPP.. ssiinsolicyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We describe all training and sampling details, including all hyperparameters, in our method and experiments sections as well as the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: We follow the standard practice of the field of reporting FID and recall scores in a table format which do not involve error bars.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoo bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
18

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provide our hardware specifications and report the number of GPU hours in our experiments section and the Appendix.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that http:ses//errpp...ccbbbbee/ttttt hctppppbll..ttiiccctueeelellllll
didn't 't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https httpps://nnnt http:://eeuupppeeeeee
Answer: [Yes]
Justification: Our work fully confirms with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. if there special consid- eration due to laws or regulations in their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We include discussion broader impact in the Appendix.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
19

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: We will only be publicly releasing our small-scale ImageNet codebase which does not pose any safety risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We cite all research works be build on and use the code of others according to their licenses. We list all licenses in the Appendix.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
20

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi thcode. com comddatasess has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors encouraged to reach out the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We plan release code for ImageNet experiments after publication.
This will include a complete README with detailed instructions and license.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [Yes]
Justification: To measure the performance of our distilled text-to-image model, perform a user study, the details of which are presented in the Appendix.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: We de not perform any research on human subjects that may have any potential risks.
Guidelines:
21

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification:
We do make use of LLMs for developing our core methods and results.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy fnttps://neurb:: hte.rrpp..ccCCoffeeeeeeccc. wttpccc///ooooiaaeeee / for what should or should not be described.
22

---

Appendix
A Broader Impact
24
B Limitations
24
Proofs, Derivations and Thcoretical Details
25
C.I Theorem: Consistency Models Flawed Multi-step Generators
25
C.2 Theorems: Flow Map Objectives
28
C.2.1 AYF-EMD
28
C.2.2 AYF-LMD
29
C.3 Derivation: Tangent Warmup as Linearity Regularization
29
Connections to Existing Methods
D.1 Flow Matching
30
D.2 Continuous- Time Consistency Models
31
D.3 Consistency Trajectory Models
31
D.4 Flow Map Matching
31
D.4.1 EMD
31
D.4.2 LMD
32
MeanFlow Models
E
Flow Maps in Prior and Concurrent Works
33
E.1
Discrete-time Flow Maps
................................
33
E.2
Continuous-tima Flow Maps
................................
34
E.3
Concurrent Works
34
F
Experiment Details
35
F.I
ImageNet Experiments
................................
35
F2
Adversarial Finctuning
................................
37
F3
Training Algorithms
37
F4
Text-to-lmage Experiments
37
F55
User Study Details..
37
G Ablation Studies
38
Loss Function Guidance Ablation
38
Clamped Gradient Warmup and Stable Time Embeddings Ablations
G.3 LoRA Rank Ablations
40
H Additional Samples
41
H.l Text-to-Image
41
H.2
ImageNet-512
41
H.3
ImageNet-64.
41
Text Prompts Generated Images
Licenses
54
23

---

A Broader Impact
Generative models have recently major advances and and highly aesthetic images, videos and other modalities. Align Your Flow represents a universal approach to significantly accelerate model sampling from diffusion and flow-based generative models through distillation. Real-time generation can unlock new interactive applications and help artistic and creative expression enabled by generative modeling tools. Moreover, accelerated generation also makes inference workloads more computationally efficient, thereby reducing generative models' energy footprint. Although evaluated Align Your Flow on image generation benchmark, our proposed methodology is in principle broadly applicable. For instance, one could also imagine future applications drug discovery, where fast generative models can propose novel molecules rapidly and enable efficient in-silico drug candidate screening. However, generative models like diffusion and flow models, and their distilled versions, can also be used for malicious purposes and, for instance, produce deceptive imagery. Hence, they generally need to be applied with an abundance of caution.
routinely synthesize realistic
B Limitations
As shown in ablation studies (App. G; also Fig. 8), our AYF models stabilize performance across multi-step sampling. However, this comes at the cost of slightly degraded one-step performance compared methods focused solely one-step generation (e.g. sCD [49] SiDA [97]). Note that adversarial finetuning can mitigate this and improve performance across the board, as shown Fig. 8 and with minimal loss in diversity. We believe that it would be possible further improve upon that with more carefully tuned post-training stage, possibly leveraging recent variational score distillation techniques [87, 81]. Additionally, small gap remains between the AYF model and its multi-step teacher flow- bas- model, regardless of the number of sampling steps used. This is expected, given that both models have roughly the same number of parameters, yet AYF is trained on a more challenging task and preserves the same toise-to-data mapping. Scaling up model capacity may help bridge this performance gap. Finally, our work focuses on distillation and assumes access to pretrained flow-based teacher. Prior works [49] suggest that direct consistency training can outperform distillation in certain scenarios. Since our AYF-EMD loss is directly compatible with this paradigm, exploring flow map training without distillation is promising direction for future work.
Consistency Models
Map
Map Adv. finetuning
3-
2-
4
8
16
24
32
48
64
Number of Sampling Steps
Figure 8: FID versus number of sampling steps ImageNet 512x512 (lower is better). Diffusion models require dozens of steps to reach good quality, and consistency models deteriorate after only a few, whereas our AYF flow maps maintain low FID across the board. AYF is slightly weaker at single step generation, but a brief adversarial fine-tuning stage closes this gap and improves quality for all numbers of sampling steps.
24

---

C Proofs, Derivations and Theoretical Details
C.1 Theorem: Consistency Models are Flawed Multi-step Generators
Theorem C.1 (Restated from Theorem 3.1). Let Pdato (x) = c²² be the data distribution. Then
the optimal consistency model f" (xx,, 1) is given by
Xx
f* (x,,
vp + (1-41-17)) 2)
For any > consider the following non-optimal consistency model
fe(x,,, 1) (c+1xe)x,
+1-
Then, there exists an integer N for which increasing the number of sampling steps during multistep CM sampling beyond N increases the Wasserstein-2 distance of the generated samples the ground truth distribution.
Proof. In this Gaussian setting, all intermediate noisy distributions p(x:), where x, = = X1 (1 1) x Xo, Pdata: X1 ~ (0, (0 I), and / E [0, 1]. remain Gaussian. As result, the error of the
non-optimal consistency model is:
Ex (xx,(())))) f* (x 4
Ex 112
X₁
12
Vt²
(1
t)
(6)
F2
1 (1 2) Exo:x1 ||| x1 + +(1 (1- xo = = s
and satisfies the error bound in Theorem 3.1 by choosing a small enough that satisfies e² < 8.
Due to the Gaussian setting, both the optimal velocity from flow matching, v vx(x,, t), and the optimal denoiser, D'(x,, t), have closed-form solutions.
We first derive the denoiser. Following the framework of [34], let p(x; o) denote the distribution obtained by adding independent Gaussian noise with standard deviation the data. Then,
²²
p(x; o) (O. (c² + +)) Vx logg(xx
Dx(xx))= c²
+ 0²
where the final step follows from the identity
Vx log polg = (D(x; - x)/o²
from [34]. Using this, the optimal velocity from flow matching, v* (xe, t), can be computed as
vx,,, El[x
Xo
x,
E
Xx
Xo
x,
Xt
E[xo xx]-
t
t
Substituting D xx 1-7: 1-1 for E(xo xz). we obtain
("(x,, = X -D-
1-)
Using the closed-form expression for D' (x, o)
D'
1
c
v (xxxxt)=
2 1-t
I-1
Simplifying further,
Xt (1-
(1 (11-))
t- 2 (1- x
v"(,,, = - c²(1 1)2 1
=
12 (1 1)2c2
25

---

Integrating this velocity along the PF-ODE,
from t to 0 yields the optimal consistency model: dx, v" (xt,
(x),,
C.Xt
vp + 2)
Consider the non-optimal consistency model
(c+txe)x,
f. (xt, ()
+1-
Note that () = x satisfies the boundary condition and is a valid consistency model.
Let us analyze a single multistep transition from timestep to timestep s. This process consists of two steps: 1. The noise is removed by predicting the clean data using the consistency model, yielding x6 - (x, l). The estimated clean data x is then noised back timestep s using
x
x z + (1 s) x X z (0,I).
Assuming that x NV 12 x, will also follow an isotropic gero-mean Gaussian distribution and is obtained as follows:
(c + te)x,
x x z (1 8) x
v (1-
Therefore, the variance of v, is given by
Var(x,) 5² (1 -1- (c tt)²
f2 + (1-1) t)2c2 (1-2)26 Var(x;).
(8)
Using this recurrence relation, compute the variance of the distribution obtained by running multistep CM sampling on a uniform n-step schedule:
00
n
Since both the ground truth distribution and the distribution of xo derived by n-step sampling are isotropic zero-mean Gaussians, the Wasserstein-2 distance between them has the following closed form solution:
W (p(p(x)): Pdata Pdata (x)) (VVar(Xo) c)²
Let Var(s) = Var(x,) for convenience. We will show that as n - oc, the variance Var(0) when computed via the recurrence defined in Eq. (8) on the uniform n step schedule diverges, i.e. Var(0) - co. This means performing multi-step sampling with the consistency model will result in accumulated errors beyond a certain point.
Define
Var(s)
h(s) = 82 + + (1- 8)
Plugging this into Eq. (8) gives:
h(8) 2² (1 (c 2((1))
32 + 8)222
We know h(1) = 1 and ((0)- = Var(0)/e2. So, it's enough to show that h(0) + 00 as + co.
It's easy by induction that h(t) 2 1. Define g(t) h(t) measure how much it grows. Then:
g(8) (1 (1- 2 + te ( +) (1 5)
+ (1 - s)2
(1 5)²
(2ele + 222 + (c+ + (e) (e))
(10)
22 + (1 - 2
8)²
(2te + e² gg)())
32 + (1 - 2)
26

---

All terms are positive, so let's lower bound g(t) by defining simpler sequence:
s]=
g' (8)
8² + (1 - s) (2ctc + (t))
(11)
Clearly g(s) 2 g'(s), so it's enough to show g() + 00 as - 00.
Now define:
r(s) g'(s)
2CE
Then the recurrence becomes:
(1 9)-
r(s) = 82 + (1 - 8)2C2 (t + err())),
(12)
with r(1) = 0 and r(0) = of (2ce, So we just need to show r(0) -
To analyze this, fix and and consider the function:
(1 s)²
fa,(()) == 82 + (1 8)2 (t 2x).
(13)
This is an affine map with unique fixed point:
o(s, t) 1(1 8)2
(14)
5²
Subtracting os8 1) from both sides, we get:
f(x) o(s,t) = M(s((x - t)s
(15)
where:
A(8) (1-5)e + 1.
So (x) pulls every point toward its fixed point o(8, t), with a pull factor of X(s). As + 0, since t) > (1-s)² the fixed point oss,t) - co, and the pull factor A(s) approaches 1, meaning the pull gets weaker.
This means the recurrence in Eq. (12) is applying sequence of weaker and weaker pulls toward bigger and bigger targets. To prove r(0) - oo, we now proceed by contradiction. Assume there exists some 8 > 0 and infinite sequence nı < such that r(0) < 5 when using the
schedule [0, n. ...,.. 1).
Since o(s, t) 2 (1-a)2 and the function (1=w}² all E [0,t*]
00 as x + O, we can pick t" E [0, 1] such that for
a
(1 t² =8
28 = o(s,t) (e))t)+ 2 (1-s)² 2 28.
(16)
So every fixed point in [0, t*] is at least 26.
Now split the problem into two cases:
Case I:
There exists some & E [0, [*] where r(s) 2 Since all future pulls are toward values 25,
r(.) will stay above 8, so r(0) 2 8-a contradiction.
Case 2: For all s t0. we have r(s) 8. Pick any E (0,t*. Letl A(s*)
1, which is
the maximum pull factor (corresponding to the weakest pull) on [t*,"]]. Then for s* S < t
28 - r (o(s, t) r(s))
+ (25 o(s,t))
(o(s,t)-fa.(f) - fs,(r(()) + (25 d(s,t))
M(s)(o(s, - (r(()) + +(25 ()s,())
S {(ss)((o(s,t) - -(())) + (25 o(s,t)
=e(o(s,t) r(t))
+ +(25 o(s, t))
-E(25- r(1)
+ (1 - () o(s,))
E(25 r(t)).
27

---

Applying this inequality M times (where M is the number of steps between s' and t* on the schedule).
we get:
8 - 28 - r(s") - 25
(17)
If . is large enough so that 28 < 6, we get a contradiction. This happens when n > M/(t" s*)
Since both cases lead to contradiction, we conclude that r(0) - oo as - oo, completing the proof.
Intuition: When doing multi-step sampling with consistency models we first perform denoising by removing all noise from a noisy image to obtain a clean one, and then re-add a smaller amount of noise. But because the model is not perfect, the denoised image drifts slightly off the true data manifold. When noise is added back in, the resulting image is now slightly off the noisy manifold the model was trained on. This mismatch compounds: each denoising step starts from slightly worse input, pushing the sample further off-manifold over time. As a result, errors accumulate with more sampling steps, leading degrading image quality beyond certain point.
C.2 Theorems: Flow Map Objectives
C.2.1
AYF-EMD
Theorem C.2 (Restated from Theorem 3.2). Let (x), be the flow map and (x(x,, t) denote the pretrained flow matching model. Define 8- stopgrad(()) (0). Let and = c(s denote two adjacent starting timesteps for a small e > O. Note that E [t. s) is derived by taking a small step from t towards Consider the following consistency loss function defined
(O) = Ex,,t, [w(t, s) ()),(t)),,, fg (x-(xe,t' (',,t'ss) (1)22
(18)
where Xy is obtained by using 1-step euler solver on the PF-ODE, i.e. dx; = (x( t)dt, from to t'. In the limit as e + 0, the gradient of this objective with respect to 1 converges to:
df, (x,, L, 8)
lim (1/ VoL c-0
V VOlx,tt
wo(t stt, - s)f, (x,, t,
(19)
dt
Proof. For this proof, we follow a similar logic as the proof of Theorem in [71]. We start by computing the gradient of Eq. (18) with respect to and simplifying the result to obtain
1 Vo Ex [vo(t, 8) f - fo- (xe, 1', x)
VoL VoLEMD(()
c
Exi,i. [w(t, Vofo 8) (xo, t,
fo (xv, t,, s)]]
Öfg
= Bxiiis ([(xx)Vo6((x,11.3) (o(xx)))))
f (x 8)
0x (xy x1]+
Ofg
Ot (t' -t) t) + O(e2)
w(t, Vol(,))V)l((xxtt.s) (-
Ofg. (E(s - t) vo(xe, t)) + Hfg Ol
(e(s - t)) )} + +
-xx,ts
Ox
Öfg
δx vo(xi,, + 0fg-
)1
Ex,,,e w(t,s)(t 5) Vof t, s)
+ O(c)
Ot
(dgg- dlg (X1,
= Ex,,,,
st s) (x,,,t) t. ST
+ O(E),
dt
Taking the limit of both sides as e + completes the proof.
Corollary C.3. Theorem 3.2 assumes that the step size is proportional to the interval length, i.e. I-- t'l x It - sl, leading to the introduction of a (t - s) term in the weighting function. This can be eliminated by using t sign(s t) x e and leads to the following objective:
dfp (x,, t.
VolExatee w(t,, s)sign(t 5) (xe, t, t)
(20)
dt
28

---

C.2.2 AYF-LMD
Theorem C.4 (Restated from Theorem 3.3). Let fo(x;, t.8) be the flow map, Vo(xi,t) be the pretrained flow matching model, and define 0 stopgrad(0) Let s and =s c(t 8) denote two adjacent ending timesteps for small > O. Note that s' E [t, is obtained by taking small step from s towards Consider the following consistency loss function defined
(0) = Ex,,,, [w[t, 8) -)) -x (Xt, (11)
-
(21)
where 4D refers to running a 1-step euler solver on the PF-ODE. i.e. dx, vo(x,, t)dt, starting from z at timestep s' to timestep s. In the limit as + O, the gradient of this objective with respect to converges to:
(deo (xe, t))
lim (ee = VolEx, ((t,s)(s t)fo (x),
vol (x,, (x,,
ds
(22)
Proof. We start by computing the gradient of Eq. (21) with respect to 0 and simplifying the results to obtain
(LLLMD() = 1 - Ex [wo(t, s) (x,, t, ODE (x,, stt,s'))) 1
Ex w 8) Vofo 5) (Fo, t, ODE, (x,, ))]]
(x,,
Bxisti ww(t Vof(xx))) (f0 (fo (x1,t, + (s - s') volfo (x,,t,,',,8')1) Ot. (x,, ss) O(e²)
wit, ((tt:))off(xx,tts)) (x(x (x,, t,
Os
+(8 8') (V (x,,, 8), +())))
Bx
w(t, Vol.
(-(0R)
(Dfo (x),, (5' (s' .)+ -lo)vo(lo-
+ +
Os
Expts (o) Opg (x), 8)
vis (xL; (x) + O(e)
Ös
Taking the limit of both sides + completes the proof.
Corollary C.5. Theorem 3.3 assumes that the step size is proportional to the interval length, i.e. Is- s'1 x || - s), leading the introduction of (t-s) s) term in the weighting function. This can be eliminated by using s' = + sign(t - s) x which leads to the following objective:
dfg (x,,t,) t,
Volxist ((tgg) t),y (x)
vo (x)
(23)
ds
C.3 Derivation: Tangent Warmup as Linearity Regularization
Here, we derive Eq. (5). This equation shows the equivalence between the tangent warmup technique and regularization term on flow maps that encourages linearity.
sign(t E) (x,, s)
dxx dt
Fo t.)
Vo [sign(t s) + (s Fo t, s)) x (vo t) Fg (x,,t,s))
Ve |-It x (x, (vo(x,, t) Fo- t,,,t,s))
V F (x,, t, x (F (xL, t, s) (x t))]
(24)
V a Xx, 1, 5)) x (F g- (xt, 8) - volxe, 1))
(V (xt, L, 5)) (Fo(Xe, 8, vx 1))
[o t. vo(x,, t))) x (Fo(xe, t, vo(xe, t))
0.5 V Xx, t, (xe, t, 8) s)1
29

---

where (i) is because we discard It - s| 2 O, which doesn't t change the gradient direction, (ii) is because Fg Fa- and (iii) is because VoVa (x,,, 0.
D Connections to Existing Methods
In this section, we highlight the connections between AYF and existing methods. We show how AYF generalizes several prior approaches and discuss its relationship recent concurrent works. See
Fig. 9 for a schematic overview of these connections. the following subsections, we will derive these relationships in detail.
MeanFlow Models
fo (xx, (, = X; + (s - 6) x (,(x..4,s)
Consistency Trajectory Models
Consistency Models
Align Your Flow
Discretize time
No StopGrad
Flow Matching
Flow Map Matching
Figure 9: AYF can be seen generalization of many prior works such as Flow Matching [45], Continuous-time Consistency Models [49], Flow Map Matching [5], Consistency Trajectory Distilla- tion [94], and the concurrent MeanFlow Models [19].
D.1 Flow Matching
We show that the AYF-EMD objective introduced Theorem 3.2 generalizes the standard flow matching objective. In particular, we prove that the gradient of the AYF-EMD objective reduces to the flow matching gradient in the limit as s + I, up to constant factor.
Recall the flow map parameterization:
fo(X;, t, = (s Fo Fx t,
(25)
Substituting into the AYF-EMD objective gives:
Folie w'(t, 8) sign(t 8) fg (x),,,)) dfg (x,,t,s)
dt
dx;
cF,
)
VolExint
w'(t, 8) sign(t 8) (x: (s - ())FF))
-Fo + (s
- 1)
dl
dl
(26)
dx:
dFF
]
- w0 (1, - F
F + (s 1
dt
dt.
dx,
dF 0- ]
x VolEx,i
FJ Fo
+ (t s)
dt
dt
Taking the limit as s + gives:
dx; ) V Ex,,
Fo(xx, t,t) dx,
12
V VoEx..
Fy t. Fa- (x
(27)
df
dt
which is exactly the standard flow matching loss.
30

---

D.2 Continuous- Time Consistency Models
We show that the AYF-EMD objective introduced Theorem 3.2 also generalizes the continuous- time consistency model objective. Specifically, we prove that the AYF-EMD objective reduces to the continuous CM objective when O, up to constant factor.
Consider the AYF-EMD objective:
VolEx,, ti w' st, sign(t fg' (x,,t, t, dfg (x,, t,
(28)
dt
Setting 0 gives:
Vo w/ (), (x,, dfg (Xe; 4,0)
(29)
dt
since sign(t) for t E [0, 1]. Noting that (x,, t. 0) corresponds to the CM prediction noise level f, this recovers the standard continuous-tin cantuuuu--mmeeeeee CM objective [49].
D.3 Consistency Trajectory Models
In this part, we discuss the connection to Consistency Trajectory Models (CTMs) [38, 94]. Recall from the derivation of the AYF-EMD objective in App. C.2.1 that its gradient corresponds to the continuous-timg time limit of a discrete consistency loss. Interestingly, TCD uses this exact same discrete consistency loss train its flow map, with fixed discretization schedule. Therefore, the TCD objective can be seen as a discrete approximation of the AYF-EMD loss.
D.4 Flow Map Matching
In this part, we highlight the similarities and differences between our AYF objectives and the losses proposed by Boffi al. [5].
D.4.1 EMD
Here, we will show the connection between our AYF-EMD objective and the EMD loss proposed by Boffi et al. [5]. Recall the EMD loss from their work:
cxc
2
dfa (x
8)
VolEx t. wo(t, s) 0,66 (, V (x,,
V xl
w(t, s)
2
2
(30)
This loss be derived in similar fashion AYF-EMD loss by introducing modification to Eq. (18). Specifically, by replacing the second term f-- (xr,t's f', s) with fx(x,, s) and allowing gradients to flow through both terms, recover the EMD loss from Boffi et al. [5].
Concretely, we can show that in the limit as 0, the gradient of this objective with respect to converges to:
dfg (x,, t, s)
lim Vo (x) fx s) c
Velxx,
w' st)
dl
(31)
where w' wt,,,) w(t, s) |t s/2.
The proof is as follows:
VoExatts
[wtt,, s) [fl t) - fo (xx, s))
c(( dfo t,
27
V Ex.,,
w(t, 8)
+ O
dl
(t. dla (x1,,)))
Vo
w(l, x)
+ O(e))
dt
dfa df t, 8) 27
VoExiti.
w' s'(t,s)
+ O(c),
dt
31

---

which converges to the gradient of the EMD loss + O. Note that when deriving AYF-EMD, taking the limit of the discrete consistency loss when + 0 only required dividing by c, since the relevant term decays linearly in 6. In contrast, the gradient in this case decays quadratically with respect to e, which is why we divide by e² before taking the limit.
The small difference in the AYF-EMD derivation, namely applying a stop-gradien operation on the second term in Eq. (18), has significant effect on training dynamics. Without it, one must backpropagate through the Jacobian- vector product (JVP) used to compute df.((x,f.s)) This often dt introduces instability and slows down training. However, with the hop-gradient operation, one must only compute the JVP without needing to backpropagate through it. Fortunately, modern autograd libraries like Py PoToch support forward-mode automatic differentiation, which allows computing the JVP efficiently with minimal overhead.
Intuitivelly applying the goo-ggra means the output of a large step with the flow map is pushed toward the output of following the PF-ODE trajectory for bit and then doing smaller step with the flow map. Without the stop-gradient, the small step is also encouraged to match the outcome of the large step, which is counterintuitive. This because learning a flow map becomes more difficult as the interval length increases. Smaller steps are typically more reliable and offer better approximations.
Ultimately, the decision to include or omit the stop-gradient operation in Eq. (18) leads to two fundamentally different derivations and objectives. In our experiments, only the AYF-EMD variant, where the stop-grasd -raddiea is applied, was able to scale effectively to large-scale image datasets and produce high-quality outputs.
D.4.2 LMD
An analogous connection can also be made between our AYF-LMD objective and the LMD loss from Boffi et al. [5]. Recall the LMD loss from their work:
V Volxx,t, w(t,, s) (x,, v v (f) s]12
(32)
Similar to before, we will show that by removing the stop-gradient operation from Eq. (21) and allowing gradients to flow through both terms, we recover the LMD loss from Boffi et al. [5].
Concretely, we can show that as - 0, the gradient of this objective with respect to 0 converges to:
1
lim c0 2 Ex
1,
8))
(33)
= Volxx,ts (10,,x))) (,)
where wl,,,) = w((,,) x 11 - 8|2
The proof is as follows:
1 Ex,t,
lim 2 E
8) (lg)) 1. 3
lim Ex,,,
[w(t, 8) (x,, (f)( + (f s')
2 c-0
lim Ex,,,, 2 c-0
[wt 8) (f,, t, (x(x,,t,5)) (c(s (c(s
lim Ex,,, 2 E00
[w 8) /u
O( [|]]]
lim Ex.,,. [ (e(t 2 c lim Ex,,, (1 0(
0
c-+0l
Ex,,,,
[w/ (t, 3) - (foxx,
As before, applying the stop-gradient operation allows us to avoid backpropagating through the JVP, which speeds up training. Unlike the EMD case though, we found that the LMD loss from Boffi et al.
32

---

[5] was already stable in practice and did not introduce training instabilities. Since both versions performed similarly in our experiments, we recommend using AYF-LMD for its improved training efficiency (note, however, that in all image generation experiments, we found AYF-EMD to perform better; see ablation study Tab. 6).
Intuitively, the stop-gradient in this loss plays a similar role as in the other objective: we want the output of large step of the flow map match the result of taking a smaller step with the flow map, followed by integrating the PF-ODE.
Another way to understand this is through Eq. (21). This loss fixes the smaller step fo s') and optimizes the larger step fg (Iz, t, s) so that aligns with the velocity field at the smaller step. As think of fy ( t, s) fixed. The model then adjusts the slope of the flow map with respect to 8 so that it matches the teacher flow at that point. Without the stop-gradient, the endpoint is no longer fixed and also move match the teacher, which changes the optimization behavior.
8,
D.5 MeanFlow Models
In this section, we will show the connection to MeanFlow Models [19], which are concurrent work focused on training flow maps from scratch. We will show that the AYF-EMD objective reduces to the MeanFlow loss assuming an Euler parametrization of flow map fa(x,, 1, 8) (s (x,,,x,,,,)) This parametrization is inspired by the first-order DDIM [68] solver of diffusion models, which uses Fuler Integration to solve the probability flow ODE.
Recall the MeanFlow objective:
Fle (0) = Ex, s,
Fe(x
dxx (dx: (t (t-s) dt
dF (Xe, 1,
(34)
dt
Taking the gradient with respect to A:
dx:
dF
(Xe, t, 8)
V Meaa (0) Va
Fo(xi,t,s)
(t
dt
dt
dxr
dF
Ex,,,
F 0
Fø
+ (t - s)
dt
dt
(35)
dx:
+ (t - 8) dF
Ex
Fg-
dt
dt
dx:
dF
oF
x,
FJ
F
+ (t - s)
dt
dt
which matches the AYF-EMD objective using an Euler parametrization up to constant, as shown in Eq. (26).
E Flow Maps in Prior and Concurrent Works
Flow maps (or specific instances of them) have appeared in various prior works. In the previous section, have derived relations to some prior and concurrent works already. In this section, provide a broader overview. Broadly, prior works on flow maps can be grouped into discrete- -ime and continuous-time methods.
E.1 Discrete-time Flow Maps
Flow maps were initially proposed as a natural generalization of consistency models (CMs), and early work primarily focused on discrete-time formulations built discrete consistency losses.
Consistency Trajectory Models (CTM) [38] were the first to explicitly introduce and study flow maps. CTM trains discrete-time flow maps using combination of discrete consistency loss, flow matching loss, and an adversarial loss. Notably, CTM was also the first to introduce 7-sampling for stochastic sampling of flow maps. While the models produced high-quality samples, their performance depended heavily on the adversarial component, with significant FID drop when was removed.
33

---

Trajectory Consistency Distillation [94] builds on CTM, extending it to text-to-image generation. They empirically demonstrate that standard CMs degrade in quality as the number of function evaluations (NFEs) increases, problem which is solved when using flow maps. These models can be viewed as discrete variant of AYF, as shown in App. D.3.
Bidirectional CMs [43] observe that most prior work only trained flow maps in the denoising direction (s t). They propose training on unordered timestep pairs, accelerating application like inversion, inpainting, interpolation, and blind restoration.
Kim et al. [37] trained CTMs connecting arbitrary distributions by operating within the flow matching framework instead of diffusions.
In parallel, Multistep CMs [22] attempted to address the poor multi-step behavior of standard CMs. They divide the denoising trajectory into subintervals and train separate CMs within each, achieving strong performance with as few as 2-8 steps. Phased CMs [80] adopt similar idea but add adversarial training within each subinterval. These models effectively learn flow maps by training on (t, s) pairs where is the start of fixed subinterval containing This requires the inference step count to be pre-determined during training, and cannot be changed to trade off compute and quality.
E.2
Continuous- Flow Maps
More recently, several methods have tackled training flow maps in continuous time.
Flow Map Matching [5] provides a formal and rigorous analysis of continuous -iime flow maps and proposes several continuous- time losses. However, their empirical validation is limited to scall-scale experiments. In App. D.3, we discuss the connection between AYF and Flow Map Matching.
Shortcut Models [17] also operate in the continuous-time flow map setting. They propose an objective combining flow matching and a self-consistency loss. Their full loss (up to constants) is:
((0) = Ex,,,, Fo(xe,),))
dx. dt
X t, fo [o(xx,, t, fo- (fo
t
t
27
+
8
2
2
(36)
where the flow map is parameterized as fo(x;, 8) X + (s F) L, s). Intuitively, the self-consistency term encourages agreement between one large step and two smaller intermediate steps along the PF-ODE denoising path. To train from scratch, they use the empirical estimate (x1 Xp) in place of dx: they use suboptimal architectures and achieve significantly worse FIDD scores than state-of-the-art methods.
ar -Although their results on ImageNet-256 and CelebAHQ-226 are promising,
dx: with outputs from pretrained The Shortcut loss can also be used for distillation by replacing dr velocity model. We include this variant in our ablations (see Tab. for details).
Inductive Moment Matching [95] is recent method for training flow maps from scratch. It proposes using an MMD loss align the distributions of fg (x,, t, and f-- (x,, s) over tuples (s, r,t) satisfying < t.
E.3
Concurrent Works
Two concurrent works have also investigated training continuous-time flow maps.
MeanFlow Models [19] study training flow maps from scratch using loss closely related to our AYF-EMD objective. In fact, their formulation corresponds to a special case of AYF-EMD under an Euler flow map parameterization: fa (x,,t,s) = x, + (s t) Fe Fa(x,, t, s), as shown in App. D.5. They refer to Fe(x,,t, s, as the Mean Flow and use their objective to train flow maps from scratch, thereby being complementary to work, which focuses distillation. Their method achieves strong one- and two-step results on ImageNet 256x256, showing that the AYF-EMD objective is effective not just for distillation, but also for training from scratch.
How to Build a Consistency Model [6] extends the authors' prior work on Flow Map Matching [5]. offering a deeper analysis of the EMD and LMD objectives. They also investigate the role of higher-order derivatives, finding them helpful in Low-dimensional settings but largely ineffective in high-dimensional ones.
34

---

Table 3: Optimal sampling hyperparameters.
Model
NFE
Stochasticity y
Guidance scale X
ImageNet-64,, AYS-S
1.0
2.0
2
1.0
2.0
0.8
2.0
ImageNet-512, AYS-S
1.0
2.0
2
1.0
2.1
0.9
2.0
In contrast to these prior and concurrent works, AYF proposes new training objectives, leverages autoguidance for distillation, and shows how brief adversarial tne-ttnnng can boost performance without reducing diversity. Moreover, we provide new theoretical insights and for the first time analytically prove CMs' deterioration with increased numbers of sampling steps. Finally, we overall scale flow map models to text-to-image generation and sate-of-theean few-step performance on ImageNet benchmarks.
F
Experiment Details
F.
ImageNet Experiments
For ImageNet experiments, models are first fine-tuned to align with the flow matching framework (see Sec. 3.4 for details) before being used as teacher models distill flow map. We run this finetuning stage for 10, 000 steps using a learning rate of 0.001.
publicly available checkpoints from EDM2 [36]. These
For the teacher model, million and 134 million images, respectively. During training, we randomize the guidance scale by sampling À uniformly from the range [1.
checkpoints corresponding to the S and XS models, trained on 2147
In all experiments, we apply tangent normalization and tangent warmup. following the approach introduced in sCM [49], setting = 0.1 and H = 10000. To ensure stable training, we define s((1,)) T- which removes the (s - 1)2 term term-ern one arising from the proportional assumption (see App. C.2.1 for details) and another from the (s t) coefficient of (x ł, s) in our flow map parameterization. We use a learning rate of 10-4 and a batch size of 2048 for all experiments for total of 50, 000 iterations. These experiments were performed using 32 NVIDIA A100 gpus and took approximately 24-48 hours converge.
detailed algorithm can be seen Algorithm 1.
Timestep scheduling We sample the interval distance |: s] from a normal distribution (P ) followed by sigmoid transformation. This prioritizes medium-length intervals and improves overall training stability. Once the interval length is determined, random interval of that length is uniformly selected, with t > s set as the two endpoints. Since we are only concerned with generation, we do not train on (t < s) pairs.
We find (Pmeam: Pau) (-0.8, works well for ImageNet-511 while (Pmean, Pau) = (-0.6, 1.6) works well for ImageNet-64.
Sampling hyperparameters At inference time, sweep the guidance scale À in the range [1, and the sampling stochasticity y in [0, 1] to determine optimal byperparameters. Tab. summarizes the selected values. For n-step sampling, intermediate timesteps uniformly distributed over the interval (0, I].
Additional ImageNet512 baselines: For completeness, we include the additional baselines from Table sCM [49] Tab. Our AYF models are able to outperform methods using only sampling steps.
35

---

Table 4: Sample quality class-conditional ImageNet 512x512. This is extension of Tab. 2 with
further baseline methods.
Method
NFE (
FID (L)
#Params
Diffusion Models
ADM-G[11]
250
7.72
559M
RIN [29]
250
4.05
501M
U-VIT-H/4 [3]
250
501M
DiT-X1.2 [58]
250
675M
SimDiff [26]
512x2
VDM++ [39]
512x2
3.15
DitITT [21]
250x2
2.67
561M
AMR-XL/3R [47]
250x2
2.50
725M
DIFFUSSM XL [84]
250x2
3.41
673M
DIM-H [75]
250x2
3.78
860M
U-DIT [77]
250x2
3.50
604M
|561
250x2
2.62
675M
MaskDiT [93]
79x2
2.24
736M
Dis-H/2 [1S]
250x2
2.88
900M
DRWvK-H/2 [16]
250x2
2.95
879M
EDM2-S [36]
63x2
280M
EDM2-M [36]
63x2
498M
EDM2-L [36]
63x2
778M
EDM2-XL [36]
63x2
1.80
1.1B
EDM2-XXL [36]
63x2
1,73
1,5B
GANs & Masked Models
BigGAN [7]
1
8.31
160M
StyleGAN-XL [65]
1x2
3.92
266M
VQGAN [13]
1024
12.57
232M
MaskGIT [9]
64x2
9.24
284M
MAGVIT-V2 [88]
64x2
9.11
1B
MAR [44]
64x2
1.95
2.31
VAR-d36-s [76])
10x2
2.63
2.3B
AYF-S (ours)
1
3.32
280M
2
1.87
280M
4
1.70
280M
AYF-$ adv. loss (ours)
1
1.92
280M
2
1.81
280M
4
280M
We also compare our flow map models against several training- free accelerated diffusion samplers [51, 91], shown in Tab. 5. While these training-free free methods enable the base model approach near- optimal performance with fewer steps, they still require roughly 32 steps or more to achieve good results. In contrast, our AYF models reach strong FID scores using only 1-4 sampling steps.
Table Comparing AYF against training-frr free accelerated diffusion samplers on class-conditional ImageNet 512x512.
FID(L)
Inference configurations
1-step
2-step
4-step
8-step
16-step
32-step
64-step
EDM2 Heun solver
566.14
453.14
388.00
142.90
9.32
1.68
1.40
EDM2 2P [51]
290.28
287.78
100.36
14.20
2.19
1.49
1.39
EDM2 + DPM-SSlver-33 [51]
290.28
287.78
100.36
21.48
2.74
1.38
EDM2 + UniPC-2M [91]
290.28
287.78
101.16
12.86
2.07
1.43
1.38
EDM2 UniPC-3M [91]
290.28
287.78
101.16
30.02
4.40
1.40
1.37
AYF
3.32
1.87
1.70
1.69
1.72
1.73
1.75
36

---

Algorithm Flow Map Distillation with AYF-EMD Loss.
1:
Input: dataset with std. dd, autoguided pretrained flow model v
suided (x,, t, 2) with guidance weight A, model Fø (x,, E, s, A), learning rate distance schedule (Pmean Psia), guidance interval [Amin )max] constant e, warmup iteration H.
2: Init: Iters <
3: repeat
4:
((..oT), I), ~~ (P Padd) A ~ Unif()min Amax) D. NO,,
mean:
5:
d t (T), Unif(0, 1 d), t X t (1 t)xo + ix1
6:
dx:
de t v guided (x,:
7:
f min Iters/H)
g - (Fg- (x,, 1, 8, A) di + r(t 8) dF (x4,,,,))
8:
> Tangent warmup
dt
9:
g - g/ g/([g] cc)
Tangent normalization
10:
((0) - [Fo (Xt: L, 1,8, A) F (x:: L, 8, )) + gll
11:
0 - 0 - v))(()
12:
Iters t Iters + 1
13:
until convergence
F.2
Adversarial Finetuning
For adversarial finetuning, we use the StyleGAN2 discriminator [33] and follow the relativistic pairing GAN (RpGAN) formulation [28, 31]. The complete algorithm provided in Algorithm 2.
The flow map is optimized by minimizing combination of the AYF-EMD loss and a weighted RpGAN objective. To compute the adversarial loss, we perform one-step sampling with the flow map (i.e. setting (t. 8) = (1, 0)) to generate negative samples. Following prior work [13], we apply an adaptive weighting scheme balance the AYF-EMD and adversarial terms. Additionally, we multiply the adversarial loss by a fixed coefficient a 0.1 to ensure stable training. For the discriminator, we apply RG and Rg regularization [28] with regularization weight of 0.1.
We use a learning rate of 2 x 10 for both networks and batch size of 1024. Finetuning is run for approximately 3000 iterations using 32 NVIDIA A100 GPUs, taking around hours in total.
F33 Training Algorithms
We provide the full AYF algorithm in Algorithm 1, and the variant with adversarial finetuning in Algorithm 2.
F.4 Text-to-Image Experiments
For our text-to-image experiments, distill FLUU..1-dde)] [41] into few-step generator by fine- tuning a LoRA [27] on top of the FLUX base model. We use the AYF-EMD objective and train the LoRA for 10,000 iterations on 8 NVIDIA A100 GPUs. Each GPU fits single 512 512 image, resulting in a total batch size of 8. To manage memory, gradient checkpointing and gradient partitioning is used. We also warm up the tangent over the first 2000 iterations. Since FLUX. FLUX.I-[dev] is guidance-distilled, we cannot use autoguidance and instead rely solely on the distilled guidance.
We train model using the text-to to-timgee22MM dataset [30] from Hugging Face, which contains over 2 million real and synthetic images. We filter this dataset and train only on the 100K images generated by FLUX. FLUX.I-[dev] using text prompts from GPT-4o.
F.5
User Study Details
To evaluate our AYF-LoRA and compare it against TCD-LoRA [94] and LCM-LORA [53] (both based on SDXL [59]). we conduct user study with 47 participants. We use a holdout set of 200 prompts generated by GPT-40 from the text-to-image-2MM dataset. each prompt, we generate five images using each of the three methods, all sampled with four steps from the same random seed, resulting in 1000 sets of three generated images. Each participant given a text prompt and one image from each method, with the image order randomized to prevent bias. Participants are asked
37

---

Algorithm 2 Adversarial Flow Map Finetuning with AYF-EMD and Adversarial losses.
Input: dataset with std. dd, autoguided pretrained flow model ve v- (xt, t, X) with guidance
1:
weight A, model Fg (x t, s, A), learning rates 7G. nD, distance schedule (Pmean- Pau). guidance interval [Amin= Amax, constant c, warmup iteration H, discriminator D& (x), adversarial weights
a, 3.
2: repeat
3:
if Generator step then
4:
Xy ~ D, X1 NO I T ~~ (N memn: P p X Unif( Amins Amnu)
5:
d t (r), s Unif(0, d), t + d, Xx t (1 - t)xo + tx1
6:
dx: t- v
guided (x)
dr
7:
r + 0.99
8:
g (Fo- (x,, 1,,8 dx r(t 8)
dF, (x1..s.A)
Tangent warmup
9:
g < g/([g +
Tangent normalization
10:
X - fo(x1, 1,0, =1 - Fx 1,
11:
(EM + Fe (x,t,,,, Fg (x,, s, A) + gl1
CADV (0) c (ottplus(D (X²)
(xn))))
13:
Wadaptioe - adaptive_ hetleve_eeiih((( weight(CADV. LADV, LEMD)
[(0) + Ladv + X Wadaptive LEMD A x 0 - V (B))
15:
16:
else Discriminator step then
17:
Xo D,
(O o
18:
Xo - (x,, X, = = Fx 1,0,)) [(v) - Softplus &D (xo) (x(x)))
19:
(x6)))
20:
vb + v - 7n V (())
21:
until convergence
select the best image based on quality and alignment or indicate tic. Fig. 7 summarizes the results. See Fig. 10 for a screenshot of the instruction. The participants were paid O. 0.10$ per evaluation with an average evaluation time of 15-30s per decision, with an hourly average pay of 18S/hr.
Additionally, we compare our distilled AYF models against the base model (FLUX.1 [dev]) and an adversarially distilled approach (FLUX.1 [schnell]) through three pairwise user studies. We generate 260 images from each model, using 50 sampling steps for FLUX.1 [dev] and 4 steps for both AYF and FLUX.1 [schnell]. Each pairwise study compares two methods at a time, with human raters voting for the preferred image or selecting 'Tic'. A summary of the results can be seen in Fig. 11.
As shown in the results, both distilled models exhibit some quality degradation relative to the base diffusion model, which is expected given FLUX.1 1 [dev]'s much higher sampling cost (50 steps). When comparing our AYF model to FLUX.1 [schnell], both achieve a similar number of preference votes in terms of image fidelity. However, FLUX.1 [schnell] shows slightly stronger text alignment, which we attribute to the limited number rendered-text examples in our finetuning dataset. Note that our method's results achieved by brief LoRA-based finetuning of the base model, whereas FLUUX11 [schnell] performs full fine-tuning of the model. Future extensions could further enhance quality by adding adversarial finetuning phase, which found to significantly improve fidelity without loss in diversity in our ImageNet experiments.
G Ablation Studies
In this section, we isolate the effects of AYF's core design choices and compare our method against recent baseline consistency model and flow map approaches.
G.1 Loss Function and Guidance Ablation
We begin by analyzing two key design decisions behind AYF: (1) using the AYF-EMD loss instead of AYF-LMD, and (2) applying autoguidance on the teacher model instead of classifier-fres guidance (CFG), which is used by existing CM methods. To compare AYF-EMD and AYF-LMD, we first
38

---

Compare the given images.
Instruction
Imperiono Moona kami parttag atto to masimam at Hilki for this dlaitty ao an senking dier leput and with insiiiio large rumiier pioplloo
Images:
Text prompt: Neon-blue glowing gecko on finger in cyberpunk city at night
Qti Accoss the best?
Consider overall
detall, whinh
image the bedt
rage the best
O Image mi cert ho mapr terance
O2: Baaluate how acourataly each Image matcche the test, Considewing sepects like relesance, accurscy, and completennes peteeeenoo of depictedd elsrentts. Which image metchee the text prompt most crosely?
Inage slgg
Image 2 alg
image aigua beat
Sub net
Figure 10: Screenshot of instructions provided to the participants for the human evaluation study.
Text Alignment
Text Aligament
Text Alignment
3.45
03
0:23
0.5
0.12
0.35
3.55
7L0001 [der]
PLUXUI (LUX Ictact
TLUCK (scwal))
AYP
Visual Fidelity
Visual Fidelity
Visual Fidelity
0.5
0.42
3.25
T111X 16vr
AYF
T5
FLUXX (6m) FLUX [shee]
T
PLUX 1 (slwel))
AVP
Figure 11: Pairwise user preferences between FLUX.1 [dev] (50-step samples), FLUX.1 [schnell] (4-step samples, adversarially trained), and our LoRA-based flow map model (4-step samples).
evaluate both qualitatively on 2D toy dataset (Fig. 12). In this setting, AYF-LMD significantly outperforms AYF-EMD. However, the trend reverses on image datasets, where AYF-EMD consis- tently yields better results. As shown in Tab. AYF-EMD leads to significantly improved generation quality.
We also evaluate the impact of autoguidance. Replacing it with CFG consistently degrades perfor- mance across all sampling steps, highlighting the benefit of autoguidance during distillation. This difference can also be visually seen in the 2D setting (Fig. 13).
Next, we compare AYF against baseline methods. Against sCD [49], the current state-of-the-an consistency model, we observe that increasing the number of steps consistently degrades performance beyond 4 steps. In contrast, AYF's performance stabilizes after steps and remains consistently better at 4 steps, narrowing the gap between the few-step student and the multi-step teacher. AYF
39

---

achieves this improved steti-step performance by having slightly worse single-step generations. This expected considering that both models share the same parameter budget but AYF solves a more difficult task. However, performing short adversarial finetuning top of a pretrained AYF model significantly boosts performance across all sampling steps (see Fig. 8), overcoming this limitation.
Note that also with autoguidance replaced by CFG, we still outperform sCD for all steps except for single-step generation. This trend remains when re-training sCD with an autoguided teacher and comparing with the full AYF with autoguided teacher. These analyses show that while autoguidance boosts performance, our model superior to the previous state-of-the-an art CM for all sampling step settings except single-step generation also when comparing under the same guidance scheme. This is due to our novel objectives for state-of -hetthhe-rtt continuous- time flow map training.
Compared to Shortcut models [17], which we re-implemented, AYF again outperforms them consis- tently. While Shorteut models improve steadily with increasing NFEs, their few-step performance is significantly worse than both AYF and sCD. Unlike shortcut models, AYF reaches strong performance earlier and plateaus around 8 steps.
Ground Truth
Distilled with EMD
Distilled with LMD
Ground Truth
No Guidance
Autoguidance
Classifict-free Guid.
Figure 12: Four-step samples from distilled AYF flow maps trained using the AYF-EMD and AYF. LMD objectives for 2D distribution.
Figure 13: Four-step samples from distilled AYF flow maps using no guidance, autoguidance, and CFG (scale 3) for a 2D distribution.
Table 6: Ablation study on ImageNet 512x512. indicates our reproduction of prior methods. Also see Fig. 8 for a visualization of the key results.
Training configurations
F1D
1-step
2-step
4-step
8-step
16-step
24-slep
32-step
48-step
64-step
AYF-S (with astoguided teacher and AYE EMD objective)
3.32
1.87
1.70
1.69
1.72
1.75
1.73
1.74
1.75
With AYF-LMD objeztive instend of AYF-EMD
12.45
8.90
6.70
6.10
5.85
CFG teacher instead of autoguidance
4.12
2.57
2.32
2.29
2.31
sCD-S' (with autoguided leacher)
2,90
1.87
1.84
2.08
2.32
2.68
291
3.72
4.62
with CHG teacher instead of nutoguidonce
3.26
2.68
2.72
2,81
3.33
Shortcut model" (with autoguided teacher)
47.60
13.12
5.37
2.31
2.05
1.92
1.85
1.83
1.81
autoguidaance (teacher)
566.13
298.60
36.41
3.09
2.21
Adversarial finetuning
G.2
Clamped Gradient Warmup and Stable Time Embeddings Ablations
Next, we ablate the two stabilization techniques introduced in Sec. 3.4: (i) stabilizing the teacher model's time embeddings and (ii) applying gradient warmup with clamping. We track the 2-step FID throughout training, as shown Tab. 7. The results indicate that both techniques are critical for maintaining stable training dynamics and removing either leads to divergence.
G.3 LoRA Rank Ablations
Finally, we ablate the effect of the LoRA rank in our text-to-image experiments. We trained four models with ranks 16, 32, 64, and 128. Qualitatively, variants produced nearly identical outputs when given the same noise input. Quantitatively, we conducted user study with 260 images per rank,
40

---

Table 7: 2-step FID values during training on ImageNet 512x512.
Number of Images Seen
Training configurations
4M
8M
12M
16M
20M
24M
28M
32M
AYF (stable time embeddings clamped gradient warmup Tiezz 0.99)
4.92
2.91
2.65
2.70
2.69
2.44
2.25
2.11
without stable time embeddings without clamped gradients (Fmaz 1.0)
8.64
5.04
4.91
5.43
5.38
5.83
10.33
20.54
4.91
2.75
2.53
2.46
2.72
4.03
51.04
asking human raters to evaluate outputs based on fidelity and text alignment (voting for the "best" image or selecting "Tic"). As shown in Fig. 14, all LoRA ranks performed comparably, with rank 16 showing slightly lower fidelity. Overall, our results indicate that performance is largely insensitive the choice of LoRA rank.
Text Alignment
Visual Fidelity
0.26
0.195
0.13
0.13
0.065
Rank16
Rank32
Rink16
Rank32
RankI2K
Figure 14: Ablation study on the impact of LoRA ranks distilling FLUX.1- dev.
H
Additional Samples
H.1
Text-to-Image
In Fig. 15, we show additional text-to-image samples generated by our FLUX.1 [dev]-based AYF flow map model using efficient LoRA fine-tuning. We also show the effect of increasing the number of sampling steps of this model Fig. 17. Additionally, we show some side-by-side comparisons between our model and prior LoRA based consistency models in Fig. 16. We find that our model produces sharper and more detailed images with better prompt adherence.
H.2 ImageNet-512
In Figs. 19 to 28, we show additional one- and two-step samples generated by our ImageNet-5 AYF model. We also show the effect of increasing the number of sampling steps of this model in Fig. 18. Only very tiny quality differences are visible.
H.3
ImageNet-64
In Figs. 29 and 30, show additional one- and two-step samples generated by ImageNet-64
AYF model.
41

---

Figure 15: Selected 4-step samples generated by our FLUX.1 [dev]-based AYF flow map model using efficient LoRA fine-tuning.
42

---

LCM
TCD
AYF (ours)
LCM
TCD
AYF (ours)
"Cute jumping spiáer in párate hot on a yellow daisy"
"Robot with goldfish aquarium head and exposed wires
Red squirrel drumming tiny twig
"Neon-bise glowing gecko on finger cyberpunk city at night"
"Misty lake dawn, with and hush pine
"Colorfal village square wish fountaaii and Rowwrr-ined housess
Figure 16: Qualitative comparison between 4-step samples from LCM [54), TCD [94], and AYF (view zoomed in).
1-step
2-step
4-step
6-step
8-step
"Cute jumping spider in pirate hat yellow daisy"
"A booded warrior with glowing crimson sword misty shadowy forest."
"A Jone samurai at dusk near crimson maple forest, koi pond, and distant sunlit temple"
Figure 17: The effect of increasing number of steps when sampling from the text-to-image AYF model.
43

---

1-step
2-step
4 step
6-step
8-step
Figure 18: The effect of increasing the number of steps when sampling from the AYF model on ImageNet512 (best viewed zoomed-in).
Figure 19: Selected one-step samples generated by our ImageNet512 AYF-S model, shown for classes 1 (goldfish) and 22 (bald eagle).
44

---

Figure 20: Selected two-step samples generated by ImageNet512 AYF-S model, shown for classes (goldfish) and 22 (bald eagle).
Figure 21: Selected one-step samples generated by our ImageNet5 12 AYF-S model, shown for classes 29 (axolotI) and 88 (macaw).
45

---

Figure 22: Selected two-step samples generated by ImageNet512 AYF-S model, shown for classes 29 (axolotI) and 88 (macaw).
Figure 23: Selected one-step samples generated by our ImageNet5 12 AYF-S model, shown for classes 270 (white wolf) and 978 (coast).
46

---

Figure 24: Selected wwo-step samples generated by ImageNet512 AYF-S model, shown for classes 270 (white wolf) and 978 (coast).
Figure 25: Selected one-step samples generated by our ImageNet5 12 AYF-S model, shown for classes 979 (valley) and 980 (volcano).
47

---

Figure 26: Selected two-step samples generated by ImageNet512 AYF-S model, shown for classes 979 (valley) and 980 (volcano).
Figure 27: Selected one-step samples generated by our ImageNet512 12 AYF-S model, shown for classes 89 (sulphur-crested cockatoo) and 985 (daisy).
48

---

Figure 28: Selected two-step samples generated by ImageNet512 AYF-S model, shown for
classes 89 (sulphur-crested cockatoo) and 985 (daisy).
49

---

Figure 29: Uncurated one-step samples generated by our ImageNet64 AYF-S model, with randomly
chosen class labels.
50

---

Figure 30: Uncurated two-step samples generated by our ImageNet64 AYF-S model, with randomly chosen class labels.
51

---

I Text Prompts for Generated Images
Here we will list all the text prompts used to generate the images in Figs. 1, and 15,
"A surreal and dreamlike scene featuring small island with lush green mountain encased in a giant, translucent sphere. The sphere reflects warm golden and green tones, blending harmoniously with the soft orange hues of serene sunset. The scene is set on a calm, reflective body of water, with gentle ripples creating perfect reflections of the sphere and the sky. lone wooden boat floats peacefully in the foreground, adding a sense of scale and solitude. Distant mountains frame the horizon, completing the ethereal and otherworldly atmosphere." "
"A dark, atmospheric forest shrouded in mist, with towering, shadowy trees. At the center stands hooded warrior clad in black armor and a flowing cloak, holding a massive, glowing crimson sword. The blade emits an intense, fiery red light that contrasts sharply with the cool blue tones of the misty forest. Scattered red flowers grow on the forest floor, their vivid color echoing the sword's glow. Fiery red embers float through the air, adding to the ominous and mystical mood. The scene is cinematic and otherworldly, with a strong sense of power and mystery."
"A mystical forest bathed moonlight, with glowing blue and green bioluminescent plants. A gentle mist rolls through the trees, and faint magical runes glow on ancient stone pillars scattered throughout the scene. In the background, a shimmering waterfall cascades into crystal-clear pool. The atmosphere is serene, with soft light beams piercing through the canopy above."
"A hyper-realistic photograph of a delicate yet powerful creature, a rabbit with the striped fur of tiger, combined with the soft, powdery wings and antennae of a moth. The rabbit's body is small and fluffy, with bold orange and black stripes covering its fur, and its back is adorned with large, soft moth wings that shimmer in muted tones. The scene is set in moonlit garden, with the creature nestled among flowers, its wings gently fluttering as it sniffs at a bloom."
"A luminous koi fish with translucent fins and shimmering galaxy-like patterns on its scales, gracefully swimming in a mystical pond under the soft light of glowing full moon. The setting features delicate lotus flowers. glowing orbs, and vibrant foliage with intricate golden details weaving through the scene. The water reflects a celestial ambiance, with tiny stars and glowing accents creating a dreamlike atmosphere. The composition is highly detailed, with rich, deep colors of navy and gold contrasted by the vivid reds and oranges of the koi and flowers. The overall style combines fantasy realism with a touch of ornate elegance."
"A serene bald monk in vibrant orange robes meditating and levitating above a pristine swimming pool, with a modern minimalist house and lush trees in the background. The atmosphere is calm and sunny, with bright daylight casting clean shadows. The scene captures surreal and harmonious balance of spirituality and luxury, with vibrant colors, sharp details, and a reflective pool surface."
"A cinematic and rugged beauty of the scottish moor, gorgeous young woman with wild, fiery red hair sits gracefully under a large, gnarled, ancient tree, leaning against the rough bark, her vibrant locks catching the golden hour light and flowing freely around her shoulders, her piereing green eyes sparkle with life and joy as she smiles warmly, her expression radiates youthful energy."
and hyper- hyper-realistic portrait photograph captured in the
"A weathered, yellow robotic cat kneels gently in a muddy alley, its mechanical paws tenderly holding a small, pure white rabbit. The robot's is design is a mix of sleek and worn, with scratches and rust marks telling story of resilience. Its glowing blue eyes exude warmth and curiosity as it carefully examines the rabbit, which has vibrant yellow flower sprouting from its back. Around them, abandoned concrete buildings loom in the background, shrouded in a soft mist, while faint droplets of rain fall, creating tiny ripples in the puddles. The scene quiet and melancholie, with subtle touch of hope."
"A lone samurai stands at the edge of a crimson maple forest, his silhouette dark against the golden light of dusk. Fallen leaves swirl around him gentle breeze carries the scent of
52

---

autumn through the air. A narrow wooden bridge stretches across tranquil koi pond, its surface reflecting the fiery hues of the trees above. In the distance, the curved rooftop of a hidden temple peeks through the dense foliage, bathed in the last light of the setting sun."
"A carnival floats above the clouds, its colorful tents and twisting roller coasters suspended in midair. The Ferris wheel glows with radiant neon lights, each cabin holding a different surreal sight-one with floating goldfish, another with a miniature city inside. Hot air balloons drift lazily between the attractions, carrying visitors who gaze down at the soft, cotton-like clouds below. The air is filled with the sounds of distant laughter and the smell of caramel popcorn carried by the wind."
"A warm and inviting cottage interior, lit by a crackling fireplace. The room features rustic wooden furniture, patchwork quilt on a cozy armchair, and shelves lined with books and potted plants. Sunlight streams through window, casting soft golden light across the space. A cat naps on a woven rug in front of the fire."
"A charming, bright-eyed octopus sports a miniature, pointed witch's hat, its brim adorned with a delicate, sparkling broom. The octopus's 's eyes, shining like two bright, black jewels, sparkle with excitement as it gazes out from beneath its witchy disguise. One of its tentacles grasps tiny, wooden broom, its bristles perfectly proportioned to the octopus's 's miniature size. Another tentacle holds a cauldron-shaped candy pail, its metal surface adorned with a miniature, glowing jack-o' -lantern face. The atmosphere is playful and lighthearted, with a hint of spooky, Halloween fun. The octopus's 's very presence seems to radiate a sense of joyful, mischievous energy, if it's ready to cast a spell of delight on all who encounter it. highest-Quality, intricate details, visually stunning, Masterpiece"
"A baby dragon with vibrant, golden-yellow scales sits on a forest floor, its enormous, glossy black eyes sparkling with curiosity. Its tiny horns curve gently upward, and a soft tuft of orange fur runs along its head, adding to its charm. The dragon's 's delicate claws rest on its chest as it tilts its head slightly, radiating an innocent, playful energy. The dappled sunlight filters through the trees, casting soft shadows around the creature while emphasizing the intricate textures of its scales and the gentle details of the forest floor." "
"A towering, moss-covered golem with glowing blue eyes trudges gently through an en- chanted forest. Tiny birds nest in the cracks of its ancient stone body, and wildflowers bloom along its shoulders. Each step takes leaves behind shimmering footprints, as if the earth itself is waking beneath it. A small, mischievous fairy perches on its shoulder, whispering secrets into its ear."
"A celestial bard with flowing, star-speckled robes strums a crystalline harp that hums with the music of the cosmos. Their silver hair drifts as if caught in an eternal breeze, and their eyes shine like twin galaxies. As they play, glowing constellations dance around them, weaving stories of forgotten legends. The air vibrates with an ethereal melody, bending reality itself to their song."
"A tiny mushroom spirit with cap like spotted toadstool scurries through moonlit meadow, carrying a lantern made from a firefly trapped in a crystal jar. Their tiny hands clutch a satchel filled with enchanted spores, which they scatter as they run, causing luminescent fungi to sprout in their wake. The night air is filled with a soft, magical glow as they embark on their secret midnight errand."
"A mischievous clockwork cat made of brass and polished wood prowls through an ancient library, its glowing emerald eyes scanning the towering bookshelves. Gears whir softly as moves, its tail ticking like pocket watch. Whenever it pounces, time seems stutter for just a fraction of a second, as if reality itself is playing along with its game." "
"A wandering candy alchemist, dressed in a coat of shimmering, sugar-spun fabric, mixes glowing elixirs in crystal vials. Their hair is a cascade of molten caramel, and their eyes sparkle like rock candy. Each step they take leaves behind a trail of edible blossoms, and their belt lined with tiny jars of potions that fizz, swirl, and pop with magical flavors."
"A towering jellyfish queen glides gracefully through an underwater kingdom, her translucent tendrils trailing behind her like an elegant gown. Bioluminescent patterns ripple across her ethereal body, pulsing in syne with the deep ocean currents. Tiny fish swim in mesmerizing formations around her, drawn to the soft. hypnotic glow that follows her every movement."
53

---

"A giant, four-am armed baker made entirely of gingerbread hums a deep, rumbling tune as he kneads dough in a cozy, fire-lit kitchen. His icing-swirled eyebrows lift delight as he pulls a tray of enchanted pastries from the oven- each one shaped like tiny, dancing creature. The warm scent of cinnamon and sugar fills the air as his candy-button eyes twinkle with pride."
"In the heart of an ancient cathedral, Excalibur rests upon an altar of marble, encased in shimmering, ethereal light. The stained-glass windows cast multicolored beams across the blade, illuminating the intricate runes carved into its steel. A quiet reverence fills the chamber- one dares to approach, for legends say that only the true king may grasp its hilt without being turned to dust."
"A mischievous minion transformed into a dark side warrior, inspired by Darth Vader, stands menacingly in a dimly lit chamber. Its yellow, cylindrical body is painted matte black, with glossy red accents glowing faintly. It wears flowing black cape, a custom helmet with sharp edges and a single menacing goggle-eye glowing red. In its hand, a tiny yet powerful red lightsaber hums with energy. The minion's 's expression is a mix of determination and its usual playful mischief, as if ready to wreak havoc while still being adorably chaotic. The dark background is illuminated by faint red and blue lights, evoking the ominous atmosphere of Sith lair."
"A cunning anthropomorphic wolf wearing a coat made of sheep's wool stands amidst a flock of sheep. The wolf's face sharp and expressive, with piercing golden eyes and a sly, toothy grin. The wool coat is textured and fluffy, blending seamlessly into the surrounding flock, while curved ram-like horns add an unusual twist to the disguise. The sheep in the background look curious but unaware of the predator among them. The scene is set in misty, overcast countryside, with soft lighting emphasizing the eerie yet whimsical mood."
"A bustling cyberpunk metropolis night, filled with towering ncon-lit skyscrapers, hover- ing vehicles, and busy streets lined with holographic advertisements. The city is alive with vibrant pink, purple, and blue lights reflecting off wet pavement. People in futuristic attire walk below, while drones fly overhead. A giant screen displays an Al figure speaking to the crowd."
"A vibrant, cherry-red 1970s muscle car, gleaming under a warm afternoon sun. Chrome accents sparkle, reflecting the azure sky. The car is parked on a winding asphalt road, surrounded by lush green countryside. Show the car's powerful engine and classic design details. Capture a sense of nostalgia and freedom."
Licenses
Our models and code are built upon the following codebases and datasets:
EDM2 (https: //gittub.u/ www. com/NVlabs/eddn2) Used for our ImageNet experiments. Li- censed under CC BY-NC-SA 4.0.
StyleGAN3 (https: htopps://g// h/t::/gittttt thub. httm///aab//styyegannn We use its metric calculation
logic to compute recall scores. Licensed under the NVIDIA Source Code License.
www /Wwaaa/sstlllaaannnnnneaa We use its discriminator im- plementation for our adversarial finetuming experiments. Licensed under the NVIDIA Source
StyleGAN2
Code License.
Used for text-to- image experiments. Licensed under the Apache License 2.0. We fine-tune a LoRA on top of FLUX. 1 1-vev, which is under a proprictary non-commercial license (https: / //huggingface -
Diffusers (https://git /tttps://ggthube http:p///gggtbbbb.ccccuuuggggggo httmm/ugginggfcc///afuueeeeeeeee
md).
Text-to-Image-2M Dataset
http://gnggaccccccttttttss/s//// text- text-to-image-220 2M): Used to train our distilled text-to-image LoRAs. Licensed under the MIT License.
ImageNet Dataset: Used for our main experiments. Distributed under non-commercial research license.
54


Reviews:
[{"review_id": "7V4w4yQ1TL", "reviewer": "Reviewer_Rb7o", "summary": "This paper proposes two new objectives for continuous-time Flow Map distillation, which use a consistency condition at either the beginning or the end of a denoising interval. The authors first formally prove that the generation quality of a learned Consistency Models (CMs) does not always improve with the number of sampling steps, given a simple data distribution $p_{data}=\\mathcal{N}(0, c^2I)$. They then derive the two objectives, based on 1-step Euler solver and 1-step Lagrangian solver. They further introduce a bunch of training techniques including autoguidance, stabilizing time embedding with modified time parametrization, and regularized tangent warmup from sCM paper, to improve the training stability and final performance. In their experiments, the proposed method demonstrates better FID than sCD and sCT, two SOTA methods in the consistency distillation family. They further show that the proposed method enables better usage of wallclock time than these baselines. The proposed method also performs well in distilling text-to-image models such as FLUX.1.", "questions": "I wonder if the authors would like to provide a dependency analysis, theoretical or analytical, for the replacement of the embedder.", "limitations": "Yes.", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "\\+ This paper is well written and easy to follow. \n\n\\+ The proof of the limitation of Consistency Models (CMs), though only on a very simple data distribution, is inspiring. \n\n\\+ The derivation of the the two new objectives is formal. \n\n\\+ The discussion of the use of existing techniques is helpful. \n\n\\+ The performance is good. \n\n\\- Directly modifying the time embedding module by supervised training a new embedder may introduce unbounded bias. \n\n\\- Most techniques covered in Sec. 3.4 are not original.", "was_revised": true, "final_justification": "Thanks the authors for their final response, please include those experiments in the revised version. Generally I think this is a paper with incremental contribution. I vote for Borderline Accept.", "rebuttal": "**Dear reviewer Rb7o,**\n\nWe are glad you found our paper well written, highlighted our model’s performance, and enjoyed our proof of the limitations of consistency models. We appreciate your feedback and address your comments below.\n\n* **EDM2 time embeddings are unstable, theoretical analysis:** Note that, as mentioned in the paper, EDM2-style models embed their timestep t by turning it into the logarithm of the inverse signal-to-noise ratio $\\log \\sigma_t = \\log (\\frac{t}{1-t})$ first, before passing the embedding on to the neural network. In such cases, when the time derivative with respect to $t$ is computed in these models, analytically the result ends up looking like: \n$$\\partial_t v(x_t, t) = \\partial_t NN(x_t, \\log \\sigma_t) = \\frac{\\partial}{\\partial \\log \\sigma_t} NN(x_t, \\log \\sigma_t) \\times (\\frac{1}{\\sigma_t}) \\times \\left(\\frac{1}{(1 - t)^2}\\right) = \\frac{\\partial}{\\partial \\log \\sigma_t} NN(x_t, \\log \\sigma_t) \\times \\left(\\frac{1}{t \\times (1 - t)}\\right)$$\nAs we can see, when $t \\to 0$ or $t \\to 1$, the second term in this equation goes to $\\infty$, which causes the time derivative to blow up, leading to instability. This is the reason we propose changing the time embedding from using $\\log \\sigma_t$ to $t$ instead. \n\n* **Using EDM2 checkpoints by stabilizing time embeddings:** To make this change, a short supervised finetuning stage is performed where we align the time embedding layers as described in Section 3.4. Optionally, we follow this with a brief full-model finetuning stage, aligning the student’s output with the teacher’s to minimize bias. In practice, we found that aligning only the time embedding yields an L2 loss of approximately 0.0011 between student and teacher outputs, which improves marginally to 0.0010 with full finetuning.\nTo quantitatively measure the effect of this finetuning, we compare the original EDM2 checkpoints and our finetuned flow-matching checkpoints using FID scores at varying numbers of function evaluations (NFEs, corresponding to sampling steps). Results are shown below:\n| NFE             | 4       | 8       | 16      | 32      | 64      | 128     |\n|----------------|---------|---------|---------|---------|---------|---------|\n| EDM2           | 388.007 | 142.896 | 9.324  | 1.675 | 1.402 | 1.370   |\n| FM-Finetuned   | 192.377 | 46.82   | 7.892   | 2.370 | 1.453 | 1.381   |\n\n  As shown, our finetuned checkpoints achieve FID scores close to the original EDM2 models, with any differences falling within a reasonable margin likely due to differing timestep schedules used during sampling. In short, we observe no significant degradation in performance.\n\n* **Stable embeddings are important for smooth training:** We would also like to reiterate the importance of the timestep embedding finetuning. In addition to the analytical derivation above, here we demonstrate this empirically. To this end, we ran an additional ablation, doing AYF distillation while comparing two teacher models: one using EDM2-style embeddings and the other using our stabilized variant. The EDM2-style embedding led to frequent instabilities mid-training that ended with the run diverging entirely, whereas the stabilized variant was robust. \nAs we are unable to share the training curve as an image during the rebuttal, we list the 2-step FID scores of model samples generated after different training iterations below:\n| Training images seen                        | 4M   | 8M   | 12M  | 16M  | 20M  | 24M  | 28M  | 32M   |\n|-----------------------------------|------|------|------|------|------|------|------|-------|\n| Teacher with EDM2-style embedding | 8.64 | 5.04 | 4.91 | 5.43 | 5.38 | 5.83 | 10.33| 20.54 |\n| Teacher with FM-style embeddings  | 4.92 | 2.91 | 2.65 | 2.70 | 2.69 | 2.44 | 2.25 | 2.11  |\n\n  We see that the distilled model from the original EDM2 teacher with the original embeddings does not reach low FID scores and eventually diverges (after around 28M to 32M seen training images). These results will be included in the final version of the paper.\n\n* **Novelty in Section 3.4:** As the reviewer mentioned, some techniques in Section 3.4 are adapted from sCM [1], which is the closest prior work to ours. Building on existing methods like this is standard practice, and in our case, we actually found several of sCM’s tricks to be unhelpful or even harmful, so we left them out. That said, there are also new contributions in this section. In particular, the connection between tangent warmup and regularization, as well as the idea of finetuning the time embedding layers to enable the use of pretrained checkpoints, are both novel and were not used in sCM. This is why we highlighted these new contributions in this section in the main text.\n\nWe hope we were able to address your questions. In particular, we hope that we could address all concerns regarding the adaptation of the time embeddings. We will include the above results and discussions in the final version of the paper. If you found our explanations convincing, we would like to kindly ask you to consider raising your score accordingly. Thank you.\n\n\n**References**:\n1. Lu, C., & Song, Y. (2024). Simplifying, stabilizing and scaling continuous-time consistency models. arXiv preprint arXiv:2410.11081."}, {"review_id": "qDbHpkUUyM", "reviewer": "Reviewer_jQYj", "summary": "This paper presents \"Align Your Flow\" (AYF) for distilling diffusion or flow-based models into few-step samplers. This paper shows that standard consistency models (CMs) are \"flawed\" for multi-step generation. They introduce two continuous-time training objectives, termed Eulerian (AYF-EMD) and Lagrangian (AYF-LMD). To improve sample quality, the paper leverages distillation of an autoguided teacher model, incorporates several stability-enhancing training techniques, and employs a final adversarial finetuning step. The method is evaluated on ImageNet and text-to-image generation with state-of-the-art performance among non-adversarially trained few-step models.", "questions": "- Why not apply the proposed methods to the largest model, like EDM-XXL? Are there any stability concerns?\n- AYF-LMD seems expensive in computation. Could you provide a cost comparison between AYF-EMD and AYF-LMD in terms of throughput or training FLOPs?\n- Why does training become unstable as the tangent warmup coefficient r approaches 1, a problem not reported for the original sCM \n- How does the lora rank change the model performance for the FLUX model?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "## Strengths\n\n- Empirical results. FID scores on ImageNet 64x64 and 512x512 are highly competitive, particularly for the 2-4 steps. The method appears to achieve these results with smaller models and lower computational cost than strong baselines like sCD. This makes a compelling case for the practical utility of the proposed distillation pipeline.\n- The visualization in Figure 2 and the corresponding mathematical framing are clear and provide good intuition.\n- This paper discusses the flaws of Consistency Models in multi-step generation.\n\n## Weaknesses\n\n- The method's reliance on a collection of very specific, finely-tuned \"tricks\" (clamped warmup, specific time embeddings, etc.) may limit its generality and ease of application to new domains. \n- Some ablation studies for some of these crucial components are missing, e.g., what happens if r is not clamped, or normalization is not applied?\n- The paper introduces two objectives, AYF-EMD and AYF-LMD, but then casually dismisses the latter, stating it \"leads to overly smoothened samples\" on images. The paper offers zero analysis or intuition for this failure. Proposing two methods and admitting one doesn't work for the main application with a hand-wavy, one-sentence explanation is a significant scientific weakness. \n- Furthermore, the Regularized Tangent Warmup section reveals that the training is unstable if the warmup coefficient r reaches 1. The proposed \"fix\" of clamping r at 0.99 feels like an ad-hoc hack. This suggests that the proposed AYF-EMD objective may be inherently less stable than the sCM objective it builds upon, a critical issue that is recorded rather than investigated.\n- Line 353, \"a fact we prove analytically for the first time,\" seems not to be a sound claim. \n- It could be better to, instead of unifying every model under flow maps, which acts not informatively, introduce and discuss the formulation and training algorithms for each model definition.\n\n### Typos\n\n- \"objective objective\" in line 216.\n- ?? in line 737.", "was_revised": true, "final_justification": "Thanks for the response. I acknowledge the explanation regarding the absence of XXL experiments, and I recommend noting the estimated training cost or GPU hours. My opinion on AYF-LMD is similar to Reviewer dC5b’s. It seems not particularly interesting for the main task and adds overhead to understanding. The LoRA rank ablation and the text-to-image evaluations provided in response to Reviewer dC5b are helpful, and I encourage including them. Authors are encouraged to improve the clarity based on the comments.", "rebuttal": "**Dear reviewer jQYj,**\n\nThank you for your feedback. We are glad you found our empirical results compelling, highlighting the practical value of our proposed methods. We are also delighted that you appreciate our visualizations, the mathematical framing, and our analysis of the flaws of consistency models. \n\nBelow, we will address your concerns. \n\n* **XXL runs:** We primarily wanted to show via our experiments the benefits of using flow maps with 4 steps compared to consistency models which excel in 1-2 steps. Simply put, one might wonder given great 1- and 2- step results, why even bother with 4-steps? As such, we aimed to show that smaller 4-step models can be shown to be competitive with, and even outperform, larger 1- or 2-step models. Specifically, we are demonstrating that through distilling small but autoguided models into efficient high-performance flow maps, these small but expressive AYF flow map models can outperform the large baselines. This is validated in our extensive evaluations. In particular, because we use small architectures, the wall clock sampling time of our 4-step models is faster than the sampling time of previous 1- or 2-step sCM [1] models using very large networks. This is why we focused only on the small EDM checkpoints. Moreover, training XXL models to state-of-the-art performance would be extremely costly and slow, potentially taking a week or more, and beyond the scope of the rebuttal. We thank the reviewer for their suggestion, though, and will consider adding EDM-XXL results in the final version.\n\n* **AYF-EMD vs. AYF-LMD computation requirements:** We would like to point out that our AYF-EMD and AYF-LMD objectives both require essentially the same computations (computing base velocity + computing a JVP operation), but these computations are performed in different orders.\n\n  Specifically, for AYF-EMD, we first compute the base velocity $v_t = v(x_t, t)$ (with guidance) at the current position. We must then run the flowmap $f(x_t, t, s)$ and compute its total time derivative with respect to $t$, i.e. $\\frac{\\mathrm{d} f(x_t, t, s)}{\\mathrm{d}t} $. To do this, we use the JVP operation in pytorch using the base velocity. The output of this JVP operation is used to obtain our regression target (see Algorithm 1 in the appendix). Note that we do not backpropagate through the regression target, which means we do not have to worry about higher-order derivatives. \n\n  On the other hand, for AYF-LMD, we must first run the flowmap $f(x_t, t, s)$ and compute its time derivative with respect to $s$. Similar to before, we do this by using the JVP operation in pytorch. Next, we compute the base velocity (with guidance) at that new point $v_s = v(f(x_t, t, s), s)$, and use that velocity along with the time derivative of s to obtain our regression loss, which is also detached (i.e. also here no backpropagation through the JVP). \n\n  As such, both methods need to compute the base velocity once, and perform a JVP operation. Therefore, they have similar computations and exhibit identical training speed. \n\n\n* **Why destabilization when $r \\to 1$?**\nThe original sCM [1] paper introduces several training stabilization techniques, one of which is called *adaptive double normalization*. It applies a normalization to the scale and shift parameters in each AdaLN layer. In our experiments, using this normalization for both the flow map and the teacher flow-matching models results in a significant quality deterioration. As such, we opted to not include this technique in our architecture, and kept the network structure almost identical to the one from EDM2. However, this implied that we needed another means of regularization to stabilize training, leading to our proposal of clamping $r=0.99$. Importantly, unlike sCM, we make the (novel) connection between this clamping operation and a simple regularization added to the original objective, to avoid it seeming like an ad-hoc solution (see Eq. (5) in the paper and below). Without any regularization (i.e. setting $r=1$ and not using adaptive double normalization) training diverges. Note that it is standard practice in this field to leverage such regularizations. \nAs we are unable to share the training curve as image during the rebuttal, we list the 2-step FID scores calculated from samples generated by the distilled model after different amounts of training:\n\n  | Training images seen         | 4M   | 8M   | 12M  | 16M  | 20M  | 24M  | 28M   |\n  |---------------------|------|------|------|------|------|------|-------|\n  | AYF without clamping| 4.91 | 2.75 | 2.53 | 2.46 | 2.72 | 4.03 | 51.04 |\n  | AYF with clamping r=0.99   | 4.92 | 2.91 | 2.65 | 2.70 | 2.69 | 2.44 | 2.25  |\n\n  In these experiments, both runs reach their maximum $r$ values at 20M images, after which the non-clamped version starts to exhibit unstable training dynamics, as seen by the diverging FIDs. In contrast, the clamped version continues to improve and exhibits smooth training curves.\n\n* **Text-to-Image LoRA rank ablations:** \nThank you for pointing this out. In our original experiments, we opted to use rank=64 to follow prior works such as TCD [2], which used the same rank. To properly evaluate the impact of the LoRA rank, we now trained additional LoRA models with ranks 16, 32, and 128. Qualitatively, all variants produced nearly identical outputs when given the same noise input. As we are unable to share images during the rebuttal, we instead ran a user study. For each rank, we generated 260 images and asked human raters to evaluate the outputs based on fidelity and text alignment (vote for “best” images or say \"Tie\"). The preference vote counts are summarized below:\n\n  |             | Rank=16 | Rank=32 | Rank=64 | Rank=128 | Tie |\n  |-------------|---------|---------|---------|----------|-----|\n  | Fidelity    | 48      | 59      | 61      | 65       | 24  |\n  | Alignment   | 53      | 60      | 66      | 56       | 22  |\n\n  The results suggest that all LoRA ranks perform comparably, with rank 16 showing slightly lower fidelity. During training, we also observed that lower-rank LoRAs converge faster, while higher-rank ones require more iterations. Our ablation shows that the results are generally not very sensitive to the LoRA rank. We will add this to the final version of the paper.\n\n* **Typos:** Thank you for spotting and pointing out the typos. We will make sure to fix those in the final version of the paper. \n\n* **Better flowmap discussion:** We will include a more in-depth explanation and discussion around prior works that propose learning flowmaps in the final version of the paper. However, we would like to kindly point to both section 4 (related work) in the main text as well as to the discussions of related methods in Appendices C.3, C.5, D.1 and D.2.\n\n* **AYF-LMD being inferior:** As the reviewer points out, we introduce two objectives for training flowmaps, AYF-EMD and AYF-LMD, but in our image experiments we only end up using the former. However, we also show in the appendix that for smaller low-dimensional datasets, such as 2D points, it is the AYF-LMD loss that performs significantly better. As such, the choice of which objective to use in practice becomes entirely dependent on the dataset and even data type (keeping in mind that future work may apply our methods to data beyond images; e.g. molecules, 3D point clouds, etc.).\n \n  As for why exactly the AYF-LMD objective leads to overly smoothened outputs in image generation, our main point is our empirical evidence (see Table 5 in the appendix). Intuitively, we believe it to be due to compounding errors in the objective itself. This objective essentially regresses $f(x_t, t, s)$ towards a target that can be obtained by computing $f(x_t, t, s + \\epsilon)$ and following the ODE flow from $s + \\epsilon$ to $s$. However, if $f(x_t, t, s + \\epsilon)$ has high error and ends up far from the true $x_{s + \\epsilon}$, then following the flow from $s + \\epsilon$ to $s$ may end up pushing the target away from $x_s$, leading to an error accumulation problem. \n  Note that this is not the case for AYF-EMD as we first follow the teacher ODE from $x_t$ to $x_{t - \\epsilon}$ with this operation introducing no additional errors. We will add these intuitions to the final version of the paper.\n\n* **we prove for the first time:** We have not seen the problem of error accumulation in consistency models being mathematically proven anywhere else. Hence, to the best of our knowledge, this is a novel contribution to the field. Note that consistency models have become an important research direction and model class. Therefore, we believe that demonstrating and studying these flaws analytically represents an important, new insight.\n\nWe hope we were able to address your questions and concerns. To summarize: we introduced two novel objectives for learning flow maps and showed that each excels in different settings (AYF-EMD for high-dimensional data, and AYF-LMD for low-dimensional tasks). Building on prior work, we also proposed training techniques that help stabilize gradients when using pretrained checkpoints and ensure stable flow map training. This allowed us to achieve state-of-the-art performance in extensive experiments and scale our AYF to the text-to-image generation level. Finally, we now extended these text-to-image experiments by ablating over different LoRA ranks, finding improvements up to rank 64, with diminishing returns beyond that.\n\nWe will include the above results and discussions in the final version of the paper. If you found our explanations convincing, we would like to kindly ask you to consider raising your score, accordingly. Thank you.\n\n\n**References:**\n1. Lu, C., & Song, Y. (2024). Simplifying, stabilizing and scaling continuous-time consistency models. arXiv preprint arXiv:2410.11081.\n2.  Zheng, J., Hu, M., Fan, Z., Wang, C., Ding, C., Tao, D., & Cham, T. (2024). Trajectory Consistency Distillation. ArXiv, abs/2402.19159."}, {"review_id": "SdTPL4fx0R", "reviewer": "Reviewer_mGSb", "summary": "This paper introduces Align Your Flow (AYF), a continuous-time flow map distillation framework that addresses the limitations of consistency models in multi-step sampling. The authors theoretically prove that consistency models suffer from error accumulation, motivating the use of flow maps to generalize between noise levels. They propose two novel training objectives (AYF-EMD and AYF-LMD), integrate autoguidance for improved quality, and demonstrate adversarial finetuning with minimal diversity loss. AYF achieves state-of-the-art few-step generation on ImageNet 64×64/512×512 and outperforms non-adversarial text-to-image samplers when distilling FLUX.1.", "questions": "Thanks for your work, I have the following questions:\n- How effective is the method proposed in the article on ImageNet 256? Is there any comparison with other methods?\n- For ImageNet 512×512, have AYF results been compared against recent works like DPM-Solver-3? Updated baselines would strengthen the significance claim.\n- In the Text-to-Image task, LCM and TCD uses SDXL while AYF uses FLUX.1 [dev] as the base model. Is this comparison fair? Since FLUX.1 [dev] performs better than SDXL, is AYF’s superior performance due to using a better baseline rather than the AYF method itself? How is the performance if the same baseline is used?", "limitations": "yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\n- AYF generalizes consistency and flow matching objectives, introducing continuous-time losses that enable effective multi-step generation. Autoguidance integration and adversarial finetuning without severe diversity loss are notable advancements.\n- AYF sets new benchmarks on ImageNet and text-to-image tasks, demonstrating superior FID metrics with fewer steps and smaller models.\n- The paper provides detailed training algorithms, ablation studies, and comparisons, enhancing reproducibility.\n\n### Weaknesses\n- The paper is not easy to follow.\n- The article does not provide experimental results on ImageNet 256, and the author does not explain the reason.\n- In the Text-to-Image task, LCM and TCD uses SDXL while AYF uses FLUX.1 [dev] as the base model, the comparison may be unfair.", "was_revised": true, "final_justification": "Having reviewed the rebuttal as well as the feedback from the other reviewers, my concerns have been addressed. I recommend accepting this paper.", "rebuttal": "**Dear reviewer mGSb,**\n\nWe appreciate your feedback and thank you for highlighting our strong empirical results, our detailed training algorithms, as well as our ablation studies and comprehensive comparisons. Below, we will address your comments and concerns. \n\n* **Paper not easy to follow:** Please do not hesitate to kindly point to areas of the paper with ambiguities, and we will do our best to explain and to clarify in the final version of the paper. Overall, we will carefully analyze the paper and improve the writing where we believe the presentation can be improved.\n\n* **ImageNet 256:** As our paper is directly related to continuous-time consistency models [1], we followed their experiments and chose ImageNet64 and ImageNet512 to perform our evaluations on. Furthermore, we would also like to point out that many papers in the field report results primarily on ImageNet64 and very small datasets such as CIFAR10. In our case, we wanted to show the scalability of our approach, so we extended our evaluations to ImageNet512 and text-to-image models. Finally, considering that we have extensive evaluations on both ImageNet64 and ImageNet512, there are no additional insights to be gained from training on the same dataset at yet another resolution. And on ImageNet64 and ImageNet512 we already compare to many dozens of baselines (see Tables. 1, 2, as well as 4 and 5 in Appendix). Last but not least, training models on ImageNet to state-of-the-art performance requires substantial GPU resources and can in some cases take a substantial amount of time (hence, this seems beyond the scope of the rebuttal). Therefore, we opted to focus our resources on ImageNet64, ImageNet512 and text-to-image experiments.\n\n* **Comparing AYF with training-free diffusion acceleration methods:** Please note that our approach is about diffusion distillation which is not a training-free approach. Such distillation approaches are inherently different from training-free acceleration techniques of diffusion models, such as improved ODE solvers like DPM-Solver-3. Specifically, distillation techniques are able to dramatically accelerate the inference of these models by reducing their sampling steps all the way down to 1-4 steps. Training-free approaches such as DPM-Solver-3, on the other hand, are much easier to plug-and-play but still require at least around 10-15 steps for reasonable results. To illustrate this, below we compare our AYF models against DPM-Solver [2] and UniPC [3], which are two of the most widely used fast ODE solver approaches. Note that we chose these approaches because they are easy to add in a plug-and-play manner, unlike methods like DPM-Solver-3 (not 3M), which requires computing model-specific statistics and are harder to use in practice. \n| Sampling steps         | 1       | 2       | 4       | 8       | 16     | 32     | 64     |\n|------------------------|---------|---------|---------|---------|--------|--------|--------|\n| EDM2 + Heun (default)  | 566.137 | 453.135 | 388.007 | 142.896 | 9.324 | 1.675  | 1.402  |\n| EDM2 + DPM-Solver-2M   | 290.284 | 287.783 | 100.359 | 14.204  | 2.189  | 1.485  | 1.388  |\n| EDM2 + DPM-Solver-3M   | 290.284 | 287.783 | 100.359 | 21.476  | 2.735  | 1.414  | 1.379  |\n| EDM2 + UniPC-2M        | 290.284 | 287.783 | 101.163 | 12.862  | 2.072  | 1.427  | 1.376  |\n| EDM2 + UniPC-3M        | 290.284 | 287.783 | 101.163 | 30.016  | 4.399  | 1.396  | 1.369  |\n| AYF                    | 3.32    | 1.87    | 1.70    | 1.69    | 1.72   | 1.73   | 1.75   |\n\n  We see that while these training-free methods help the base model reach near-optimal performance with fewer steps, they still need around 32 steps or more for good results. In contrast, our AYF models significantly reduce the step count, reaching strong FID scores in just 1-4 steps. Note that at 1 or 2 sampling steps, these ODE multistep methods behave identically and they only start to differ when the number of steps exceeds 3. This explains why several entries in the table have the same values. We will add these results as well as a discussion on this, including references to solvers like DPM-Solver and UniPC, in the final version of the paper.\n\n* **FLUX vs SDXL:** We appreciate your insight on this matter and acknowledge that the base model from which we distill has a significant effect on the final performance after distillation. We would like to point out that the focus of our text-to-image experiments was mostly to demonstrate the scalability of our approach to expressive and complex image generation tasks. An important reason why we decided on FLUX.1-dev is that AYF assumes a pretrained continuous-time diffusion or flow model, whereas SD1.5 and SDXL were trained with discrete timesteps. Therefore, we cannot directly use these models without additional complications. However, the only previous text-to-image consistency models are TCD and LCM, which were trained on SDXL/SD1.5, and it seemed appropriate to nonetheless compare to these works (and we outperform them significantly). We believe our AYF is the first continuous-time consistency or flowmap framework that has been successfully demonstrated at the text-to-image scale. We would also like to point to our extensive experiments on ImageNet64 and ImageNet512, as well as our ablation studies in the Appendix, which unambiguously demonstrate the advantages of our framework over previous discrete-time techniques, which TCD and LCM rely on.\n\n  Finally, please note that we now performed additional comparisons between our 4-step AYF text-to-image flowmap and the teacher FLUX.1-dev model as well as the adversarially distilled FLUX.1-schnell model. Please see our response to **Reviewer dC5b** for details.\n\n\nWe hope we were able to address your questions and concerns. We will include the above results and discussions in the final version of the paper. If you found our explanations convincing, we would like to kindly ask you to consider raising your score accordingly. Thank you.\n\n\n**References:**\n1. Lu, C., & Song, Y. (2024). Simplifying, stabilizing and scaling continuous-time consistency models. arXiv preprint arXiv:2410.11081.\n2. Lu, C., Zhou, Y., Bao, F., Chen, J., Li, C., & Zhu, J. (2025). Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models. Machine Intelligence Research, 1-22.\n3. Zhao, W., Bai, L., Rao, Y., Zhou, J., & Lu, J. (2023). Unipc: A unified predictor-corrector framework for fast sampling of diffusion models. Advances in Neural Information Processing Systems, 36, 49842-49869."}, {"review_id": "2rpbbwRJ3a", "reviewer": "Reviewer_dC5b", "summary": "Flow Matching and Diffusion are considered the current state-of-the-art (SOTA) method for media generation. These models learn a flow process from Gaussian noise to the data distribution by regressing the velocity field of the flow map which result in an iterative generative process. Inferencing the flow/diffusion models is compute expensive and generally require from 50 to 1000 function evaluations (NFE)  since it requires solving an ordinary differential equation. This paper suggest a new distillation method that generalize Consistency Models [1], and allows sampling with 2-4 NFE without hurting image quality. A critical contribution of the paper is that the Align Your Flow (AYF) method achieve this performance without adversarial loss. Given a trained velocity $v_t(x)$ of a flow/diffusion model,  the velocity $v_t(x)$ defines the trajectories $x_t$ (the flow process) such that $x_0\\sim p_{\\text{noise}}$ and $x_1\\sim p_{\\text{data}}$.  AYF learns the flow map between every $t<s\\in [0,1]$, $f_{\\theta}$(x,t,s) i.e., $f_{\\theta}$(x_t,t,s) = x_s$. The paper suggest two training objective to learn the flow map. A crucial point is that these losses do not require to simulate the flow during training which make them tractable. Additionally, the paper provide a few \"training tricks\" that stabilize optimization and improve performance. Finally, the paper validates the proposed method on a number of experiments, including distillation of class-conditioned image generation models and a text-to-image model.\n\n[1] Song, Yang, et al. \"Consistency models.\" (2023).", "questions": "If you can provide any of the comparisons I mentioned on weakness 3. specially the third one, i.e., comparison of AYF with the non-distilled FLUX model.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "> Strengths:\n1. The paper tackles the important challenge of fast sampling for flow/diffusion models.\n2. AYF is able to distill flow/diffusion models without adversarial loss and achieve SOTA results for first time.\n3. The contribution of the paper is novel and potentially have a large impact.\n4. The paper provide an extensive comparison of the AYF method to baselines, specially on the class-conditioned image generation.\n5. The paper is well written and provide proofs in the appendix for all of its claims.\n\n> Weaknesses:\n1. The paper is not entirely self contained. For example, in line 221 it seems the authors assume the time dependency of $F_{\\theta}(x,t,s)$ is inserted through a function $c_{noise}$ without mentioning it explicitly. Took me a while to understand it is assumed the reader is familiarise with the fine details of EDM [2].\n2. Two training objectives are presented in the paper AYF-EMD, AYF-LMD in theorems 3.2, 3.3, however right after Theorem 3.3 the authors state that the AYF-LMD objective perform poorly on the image generation and subsequently all experiments on images in the main paper are done with the AYF-EMD objective. The AYF-LMD objective is not considered a baseline and does not show any advantage in the paper. Hence, it is my opinion that the AYF-LMD is not so interesting to the reader and adds an overhead for understanding the paper. I would recommend moving it to the appendix.\n3. While on the text-to-image task the reported user study shows a great advantage to AYF, the evaluation still seems lacking and not completely fair:\n  - The AYF on the FLUX [3] model is compared two baselines on the SDXL [4] and on FLUX model as well. It could certainly be that the user in general prefer images generated by the FLUX model.\n  - No comparison of AYF and a  distillation baseline that uses adversarial loss. To the best of my knowledge, these type of distillation method are still considered the SOTA. \n  - Most importantly, a comparison between the performance of a distilled AYF FLUX model and the non-distilled FLUX model is missing. Hence, the reader cannot know if to what degree of degradation the AYF distillation process causes (if any).\n\nFinally, concurrent to your work, [5] seems to achieve the same method with a loss similar to the AYF-EMD, where your $F_{\\theta}(x,t,s)$ is their \"average velocity\". They provide a different and interesting point of view on the AYF-EMD loss, and you can consider adding reference to their paper for completeness of discussion on this training objective. However, I want to stress that I acknowledge this is a concurrent work and do not count as a weaknesses or a strengths, I only provide here for the completeness of discussion.\n\n[2] Karras, Tero, et al. \"Elucidating the design space of diffusion-based generative models.\" Advances in neural information processing systems 35 (2022): 26565-26577. \n\n[3] Black Forest Labs. Flux. https://github.com/black-forest-labs/flux, 2023.\n\n[4] Podell, Dustin, et al. \"Sdxl: Improving latent diffusion models for high-resolution image synthesis.\" arXiv preprint arXiv:2307.01952 (2023).\n\n[5] Geng, Zhengyang, et al. \"Mean flows for one-step generative modeling.\" arXiv preprint arXiv:2505.13447 (2025).", "was_revised": true, "final_justification": "1. The paper tackles the important challenge of fast sampling for flow/diffusion models.\n\n2. AYF is able to distill flow/diffusion models without adversarial loss and achieve SOTA results for first time.\n\n3. The contribution of the paper is novel and potentially have a large impact.\n\n4. The paper with the additional provided results during the rebuttal provide an extensive comparison of the AYF method to baselines of class-conditioned image generation as well as on the text-to-image task. On the text-to-image class with the FLUX.1 model, they show that the AYF achieves competitive image quality to the original model (30 NFE)  with only 4 NFE. However, it appears that on the provided experiments the adversarial distillation method is still slightly better. \n\n5. The paper is well written and provide proofs in the appendix for all of its claims.", "rebuttal": "**Dear reviewer dC5b,**\n\nWe are glad you found our paper well written and our ideas novel. Thank you for highlighting our state-of-the-art empirical results, our extensive baseline comparisons as well as our proofs. Below we will address your questions. \n\n* **Insufficient explanation in Sec 3.4:** Thank you for pointing out the ambiguities in Section 3.4. We will make sure to clarify this point in the final version of the paper. Please do not hesitate to let us know if you see other areas of the paper where the presentation can be improved.\n\n* **AYF-LMD not being competitive in image generation**:  We appreciate the feedback. Although we did not find AYF-LMD to be competitive in image generation tasks, we did find that it was superior in our low-dimensional 2D settings in the Appendix. As such, the choice of which objective proves to be most beneficial for different applications may differ and depend on the dataset and data type (in the future, our framework may be applied in other domains, such as molecule or 3D point cloud generation, and it is not clear which objective will be better there). This is why we opted to include it in the original paper. This loss is also one of the things that differentiates flow maps from standard consistency models as it is only useful for the former. Nonetheless, we will consider your suggestion and may rearrange the content of the paper in the final version of the paper.\n\n* **Additional Text-to-Image Evaluations:** We appreciate your comments on our text-to-image experiments. The goal of our text-to-image experiments was mostly to demonstrate the scalability of our distillation approach to expressive and complex image generation tasks. An important reason why we decided on FLUX.1-dev is that AYF assumes a pretrained *continuous-time* diffusion or flow model, whereas SD1.5 and SDXL were trained with *discrete timesteps*. Therefore, we cannot directly use these models without additional complications. However, to the best of our knowledge, the only previous text-to-image consistency models are TCD and LCM, which were trained on SDXL/SD1.5, and it seemed appropriate to nonetheless compare to these works (and we outperform them significantly). We believe our AYF is the first *continuous-time* consistency or flowmap framework that has been successfully demonstrated at the text-to-image scale.\n\n  However, we agree that including additional comparisons between our few-step flow maps against an adversarially trained baseline (FLUX.1-schnell) as well as the original teacher model (FLUX.1-dev) is appropriate. As we are not able to share any images during this review process, we have conducted 3 additional pairwise user studies to compare the different approaches in terms of image fidelity and text alignment. We will include additional qualitative comparisons in the final version of the paper.\n\n  We generate 260 images from each model, using 30 steps for FLUX.1-dev and 4 steps for AYF and FLUX.1-Schnell (this follows the model's default sampling settings described on Huggingface). We then perform pairwise comparisons between each two methods. See the results below (numbers indicate preference votes on image pairs generated by the two methods with the same text prompt):\n\n  |    *FLUX.1-dev vs. AYF (ours)*        | FLUX.1-dev | AYF (ours) | Tie |\n  |------------|------------|-----|-----|\n  | Fidelity   | 266        | 229 | 25  |\n  | Alignment  | 262        | 238 | 20  |\n\n  |    *FLUX.1-dev vs. FLUX.1-schnell*        | FLUX.1-dev | FLUX.1-schnell | Tie |\n  |------------|------------|----------------|-----|\n  | Fidelity   | 261        | 238            | 21  |\n  | Alignment  | 253        | 242            | 25  |\n\n  |   *FLUX.1-schnell vs. AYF (ours)*         | FLUX.1-schnell | AYF (ours) | Tie |\n  |------------|----------------|-----|-----|\n  | Fidelity   | 246            | 249 | 25  |\n  | Alignment  | 273            | 223 | 20  |\n\n  As we can see from the results, both types of distillation cause some quality degradation compared to the base diffusion model. This is expected, but the base model's better quality comes at a much higher sampling cost (30 steps). However, when comparing our AYF model against FLUX.1-schnell, both methods achieve roughly the same number of votes in terms of image fidelity. However, the FLUX.1-schnell model appears to exhibit better text-alignment, which we hypothesize may be due to the limited number of rendered text examples in our finetuning dataset (this could be easily addressed in the future). We would like to emphasize that our method’s results are achieved by a brief LoRA-based fine-tuning of the base model, whereas FLUX.1-schnell relies on complex adversarial objectives and full fine-tuning of the model[1]. Therefore, we consider our results as highly successful and very promising, achieving such strong performance without any adversarial objectives at all here. However, future work could further boost performance with an additional adversarial fine-tuning phase, which we observed to significantly improve quality without loss of diversity in our ImageNet experiments.\n\n* **MeanFlow models:** Thank you for pointing out the concurrent MeanFlows work. We agree that this work is highly relevant to ours. The paper explores a loss similar to our AYF-EMD loss to train flow maps from scratch and achieves strong results. We will definitely include it in the discussion for the final version of the paper.\n\nWe hope we were able to address your questions and concerns. We will include the above results and discussions in the final version of the paper. Thank you.\n\n\n**References:**\n1. Sauer, A., Boesel, F., Dockhorn, T., Blattmann, A., Esser, P., & Rombach, R. (2024, December). Fast high-resolution image synthesis with latent adversarial diffusion distillation. In SIGGRAPH Asia 2024 Conference Papers (pp. 1-11)."}]

Author rebuttal:

Meta-review: The paper introduces a framework for creating  efficient, few-step generative models.  The main contributions are a proof on limitation of Consistency Models (CMs) and  they propose two novel training objectives for flow maps, which generalize CMs and diffusion models. 
The method, called Align Your Flow (AYF), uses several techniques like autoguidance and adversarial finetuning
 from smaller and  efficient models.

Reviewers all agree that the paper presents strong contributions, including several tricks for enhancing stability and performances, backed by compelling empirical evidences. Therefore, they recommend acceptance.
Decision: Accept (poster)