SCoT: Unifying Consistency Models and Rectified Flows via Straight-Consistent Trajectories
Authors: zhangkai wu
Xuhui Fan
Hongyu Wu
Longbing Cao
Venue: NeurIPS 2025 poster ()
Keywords: Generative Model
Primary area: deep_learning

Abstract: Pre-trained diffusion models are commonly used to generate clean data (e.g., images) from random noises, effectively forming pairs of noises and corresponding clean images. Distillation on these pre-trained models can be viewed as the process of constructing advanced trajectories within the pair to accelerate sampling. For instance, consistency model distillation develops consistent projection functions to regulate trajectories, although sampling efficiency remains a concern. Rectified flow method enforces straight trajectories to enable faster sampling, yet relies on numerical ODE solvers, which may introduce approximation errors. In this work, we bridge the gap between the consistency model and the rectified flow method by proposing a Straight-Consistent Trajectories~(SCoT) model. SCoT enjoys the benefits of both approaches for fast sampling, producing trajectories with consistent and straight properties simultaneously. These dual properties are strategically balanced by targeting two critical objectives: (1) regulating the gradient of SCoT's mapping function to a constant and (2) ensuring trajectory consistency. Extensive experimental results demonstrate the effectiveness and efficiency of SCoT.

SCoT: Unifying Consistency Models and Rectified Flows via Straight-Consisten Trajectories
Zhangkai Wu School of Computing Macquarie University zhangkai.. wuêmq edu
Xuhui Fan" School of Computing Macquarie University xuhui fan edu au
Hongyu Wu School of Computing Macquarie University hongyu. wwustudents mq. edu. au
Longbing Cao School of Computing Macquarie University longbing. waw.gggacmmg.m.m edu, au
Abstract
Pre-trained diffusion models are commonly used to generate clean data (e.g., images) from random noises, effectively forming pairs of noises and corresponding clean images. Distillation on these pre-trained models can be viewed the process of constructing advanced trajectories within the pair to accelerate sampling. For instance, consistency model distillation develops consistent projection functions to regulate trajectories, although sampling efficiency remains concern. Rectified flow method enforces straight trajectories to enable faster sampling, yet relies numerical ODE solvers, which may introduce approximation errors. In this work, we bridge the gap between the consistency model and the rectified flow method by proposing Straight-Conssiteet Trajectories (SCoT) model. SCoT enjoys the benefits of both approaches for fast sampling, producing trajectories with consistent and straight properties simultancously. These dual properties are strategically balanced by targeting two critical objectives: (1) regulating the gradient of SCoT'ss mapping function constant and (2) ensuring trajectory consistency. Extensive experimental results demonstrate the effectiveness and efficiency of SCoT.
Introduction
; Pre-trained diffusion models Ho et al. (2020); Song et (202Tb); Rombach et al. (2022) Poole
et al.] (2023); Esser et al. (2024) have demonstrated impressive performance in real-world tasks such as high-quality image synthesis and image editing. However, such practical models usually require extensive computational resources to train, well as large number of model evaluations to generate high-quality samples (e.g. images). Using these pre-trained teacher models to generate pairs of random noises and their corresponding clean images, popular choice for low-cost training and fast sampling is to train a student distillation model with advanced "trajectories" within the pair Salimans
and Ho (2022); ; Wang al. (2023 2024); Yin al. (2024b); Luo et al. (2024); Xie et al. (2024);
Nguyen and Tran (2024): Yin et al. (2024a): Sauer et al. (2024 Xu et al. 20024) Zhu et al. 2025 ;
Frans al. 2025a): Sauer al. (2025).
We can group these trajectory-basee based distillation methods into two categories: consistency model
distillation et al. (2023) Kim et al. (2024): Lu and Song (2025) and rectified flow distillation Liu
*Corresponding author: xuhui .aaaaann..eeee
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Reflow [ICLR 2023]
(b) CTM [ICLR 2024]
(c) ShortCut [ICLR 2025]
(d) SCoT [Ours]
Figure I: Comparison of trajectory distillation methods. The black line in each panel denotes the teacher trajectory of pre-trained diffusion models, which are connected within the pair of a random noise (left dot) and a clean image (right dot). The red solid line is the student trajectory of the distillation model. Panel (a) Reflow Liu et al. (202bb) straightens its student trajectory by enforcing its velocity be close to constant. However, its trajectory maps different points to different values due to the lack of consistency. Panel (b) CTM Kim et al. 2024) places consistency requirement for the student trajectory. However, it might be difficult to track the student trajectory when it is of high curvatures. In panel (c), the Shortcut model Frans et al. (2025b) focuses on velocity estimation, while uses straight lines approximate the trajectory and ensure the consistency. Our proposed SCoT model in panel (d) enforces straightness for consistent student trajectory. By avoiding the approximating errors of solving ODEs and by straightening the student trajectory, SCoT successfully bridges the gap between rectified flows and CTM distillations and enjoys the benefits of both approaches.
et al. (2023c); Zhu et al. 2025). Consistency model distillation focuses on the trajectory itself, requiring a "valid" consistent trajectory. That is, the student trajectory should map to the same point regardless of different initial points. Rectified flow distillation seeks a straight student trajectory by enforcing its velocities be close to the magnitude of point changes. While consistency distillation directly models pointwise changes along the trajectory, the straight trajectory of rectified flow distillation may reduce the number of steps required to generate high-quality images.
However, both categories have their own limitations. Consistency model distillation typically requires multiple steps to generate high-quality samples Song et al. (2023): Kim et al. (2024). This likely due to the challenge of tracking the mapping function for trajectories with high curvatures. In the case of rectified flow distillation, numerical ODE solvers are required for a learned velocities, which introduce approximation errors that degrade the final output quality.
In order address these issues and enjoy the benefits of both approaches, we bridge the gap between consistency models and rectified flows by proposing the Straight- ConsistCnttt Trajectories (SCoTT) model in this paper. SCoT aims to produce trajectories that are consistent, which is a valid condition for trajectories, and are straight, which simplifies the point changes in the trajectory. In detail, SCoT (1) trajectory straightmess by optimizing velocities towards the amount of point changes in the pre-trained model, (2) trajectory consistency by ensuring points from different time steps are projected to converge at the same value at future time steps.
regulates the trajectory through two aspects:
In summary, the proposed SCoT is the first to produce consistent and straight trajectories between random noise and clean images, which unifies the consistency models and rectified flows. Consis- tency guarantees valid trajectories, while straightness facilitates the approximation of the trajectory projection function and faster sample generation. With this design, SCoT achieves state-of-the-art results, enabling compact models to generate high-quality data in N-steps, or even single step. Figure [ highlights the key differences among consistency distillation, rectified flow distillation, ShortCut model Frans et al. (2025b), and the proposed SCoT.
2 trajectory perspective on Pre-trained diffusion model distillations
Notation We consider the trajectory to be working within the time interval [o, 1]. with the time steps specified as 1 = iN t > t 0. Since image synthesis is common task for diffusion models, we denote Xo as the clean image, X; as the noisy image at the time step f. and x1 ~ ~N I) O and
the random noise sampled from standard Gaussian distribution. Furthermore, & to represent the parameters of the pre-trained teacher model and the student distillation model, respectively. Detailed descriptions of the notation are provided in ?? and Table
Diffusion models Diffusion models (DMs) Ho et al. 2200 Song et al. (002a):; Dhariwal and Nichol (202I) generate data by learning the score function of noisy images at multiple noise scales. At each

---

time step i, clean image Xg is first diffused into a noisy image x, through a forward process == 0,X0 + x (0,1),, where a, and of are the diffusion coefficient and variance, respectively.
Using (xx) to denote the probability of diffusion in Xt, DMs learn neural network ca (x(, 1) that matches the score of the corrupted image Steal (Xt) = Xx, log Pt (x,) = --
o;xo) by
minimizing the loss Ex,x, [w)()] (xx,) - Sreal (x+ |1 where (() is weighting function.
Rectified flows Lipman al. 2023 Liu et al. (2023b); Albergo al. (2023) can be regarded as an extension of diffusion models by defining linear interpolation between random noise x1 and a clean image X as X, ix + (1 x)x,, 0 S < 1. In details, they use the ODE dx, = (e t) to transport between the noise distribution /o I) and data distribution To (xx), whereas the velocity (x t) is learned by minimizing the loss E, [l] - Xo) vo(x,, ))
Given random noise X1 (O, new clean image Xn be generated either through an iterative denoising process with a trained t)) (in DMs), or through an ODE solver as &o Xi S (x t)dt (in rectified flows). In either case, pair of random noise and its corresponding clean image is formed as (x,. Xn)) Trajectory distillation focuses on the construction of advanced trajectories to accelerate sampling without compromising image qualities.
Reflow distillation The teacher trajectory produced by the pre-trained velocity vo t) may not be straight, since their training set X ~ AN, I) and Xg (T((()()))))) are paired. To address this issue, reflow distillation Liu et al. 2023a c Zhu et al. (2025); Li et al. (2025) works on the pair (x1, Xn) by training a new velocity V& X t) that approximates the direction X X within the pair. With the trained V (xe: 1) expected to approximate the straight direction from X1 to Xu. reflow distillations may use fewer steps to generate high- gugh-qualtty images. Other approaches uses one neural network to approximate the magnitude of changes over the whole time period Liu et al. (202cc).
Consistency distillation Consistency model (CM) Song et al. (2023) can be regarded as one trajectory distillation method. CM studies a consistent projection function (x., 1) that maps any noisy image X2 to the clean image Xo: fo (x,,,) t) Xo, Vtt E [0, Its distillation objective function can be written as a weighted distance metric D).,)) between the mapped clean images from two
adjacent points Lcm = Es [()) (fo(Xi+ar, t + At), t)) where Φ is the exponential
moving average of the past values Φ, and x is obtained from the pre-trane pre-trained model as x Xt+At tAtV X.+ log (x(+aa)) CM obtains a "valid" trajectory by mapping different points to the same clean image.
Building on CM, Consistency Trajectory Model (CTM) Kim al. 2024) introduces multi-step consistent mapping function defined as:
Go(xe,t, 8) (s/t)x² + (1 (x,,
in which go (X,, f, s) is left unconstrained, and is parameterized by a neural network go(-). Equa- tion (1 ensures Gx(x,, s) satisfies the boundary condition as G (x1, 1, X1.
Concurrent works MeanFlow Geng et al. (2025) is a concurrent work to SCoT that also unifies consistency models and rectified flows via velocity integration. Its key distinction lies in introducing a correction term derived from the gradient with respect to the starting step f, unlike SCoT's use of the gradient at the terminating step s. Although rectified flow is regarded as a special case of MeanFlow under an infinitesimal integration range, it is not clear if MeanFFlow itself can learn straight trajectories in practice. Another concurrent work FlowMap Sabour et al. (2025): ) Boffi et al. (2025) also shares the same target. By using the CTM as the backbone, it is unclear if such a setting can achieve both straight and consistent trajectories in practice.
3 Straight-Consisten Trajectories (SCoT) model
Given pre-trained noise-image pair (x1, 8y), the proposed Etraight-Consisse Trajectories (SCoT) model aims to learn a trajectory-based projection function G (x 4, 8) that produces straight and consistent trajectories within the pair. Similar to the CTM, the projection function Gø(X,, L 8) takes the current time step I, the values x, at step 1,, as well as the future time step 8 as inputs, and outputs the values at time step X, (x( s).
3

---

3.1 Main Components of SCoT
We introduce two regulators that refine the SCoT trajectory regarding its velocity and consistency.
Constant-valued velocity In order to ensure that SCoT produces a straight trajectory, we regularize the gradient of its projection function G (x,, s). In detail, we encourage the partial derivative of (x( Xt, 1,8) with respect to s to approximate the magnitude of the change observed within the pre-trained pair (x1, Ro). In this way, the velocity loss function can be defined as:
Lvelecay .x,,,,, .t.s ||8G (x,, s)/0s (80- -xi)l
where 1, 8 are sampled based on the step schedule.
Let H (x,, s) denote the velocity of the student trajectory at time step and let denote the value at initial step i, G (xx, t, s) is equivalent to the solution of the ODE / u (8,, $), Vs E [t. 1]. That is:
x J." (x, r)dr = (X,, (x((x:
8, =
(3)
Os
Equation can be alternatively understood enforcing the student trajectory's 's velocity ns (Xa: 8) close to the magnitude of changes in the pre-trained teacher model.
In fact, this alternative formulation of the velocity loss | (Ro (oo-xx)][ x1) is equivalent to that of the reflow distillation. By minimizing Lvelocity, SCoT achieves the same straightening effect on the student trajectory reflow distillation. For reflow distillation, it learns the velocity of trajectory and thus needs to numerically solve ODEs to obtain the trajectory. SCoT directly learns the student trajectory through ( s) and avoids such ODE approximation emror.. More importantly, SCoT can be regularized to satisfy the consistency requirement.
We compute the partial derivative ÖG (x,, s) as using PyTorch's 's torch. autograd, .rad func- tion, which enables automatic differentiation of (x,,x, with respect 8.
Trajectory consistency SCoT also requires the tra- jectory to be consistent [Song et al. (2023); Kim et al. (2024); Frans et al. (2025a). That is, the projection function G t, s) maps the same fu- ture value to all points along the trajectory, regard- less of their current time step as Gø(x,, st,s) l2, 8), where 8 < 4 < l2. Xt, is ob- tained through teacher model's s ODE solver as Xx = Solver Xxe: l2, L1; 0) X + f (x r)dr, such that xt, and X1, are located on the same teacher trajectory.
X X12
dip ds us (Ss, .)
X
X0
Figure 2: From two different points X Xt2. SCoT maps to the same point X.. The velocity (&( 8) at time step is independent from previous time steps (1 12.
Kim We adopt the soft-consistency loss et al. 2024 which is defined by comparing the mapped outputs Go(Xe,, where sg(-) is the exponential moving average stop-
R
Gogf db) Solvvr (Xty l2, 1 0), gradient operator. Following CTM (iim et al. (2024). these two mapping values are further projected to the clean image space to construct the consistency loss as:
Loonsistency = E (eoo)))
[D [D(uagal (Xt,, 12, 11, 8), Xea (Xx, 12, ()),
(4)
where D(.,-) is the distance metric, Xes (2.8) s) Gzg(e) (Go (xt2s b2 s), 8, 0) represents the estimated clean image obtained by projecting the point Xt, forward to s using the student model G&, followed by decoding to time step 0 via the EMA model Gug(o))- and Xarget (Xt, f2, f1. Guglsb (Gugte)) (Solver(Xt,. 12, t1 f,, s), 0) serves as the target output, computed by first in- tegrating Xt, to Xx, along the teacher trajectory via the ODE solver, and then applying the same projection-dest -ecoding pipeline. This formulation ensures that both paths-althoup starting from different intermediate states on the same should yield consistent outputs at the final clean image space.
This consistency ensures the "validity" of our projection function in the trajectory. Compared to the consistency models and their variants, is expected that this trajectory straightness makes it easier to approximate the projection function than those with high curvatures.

---

Objective function Summarizing the above targets, SCoT's objective function may be written as:
Lscor
Aveeltvelcciy + Acon Cconsistency
(5)
where Aval. Acon denote the weighting factors for Lvelacity: Cconsistency. Each individual component addresses the straightness and consistency of trajectories, respectively. In this way, the proposed SCoT successfully unifies the consistency models and rectified flows.
The two loss terms, Lvelocity and Lconsistency, have different dependencies on the pre-trained model. Specifically, Lvclocity operates on a data-noise pair (Ro, x1), while Cconsistency requires the model's parameters. To cope with their requirements, we propose generating the data point &o dynamically for each training step using the pre- vre-trained model & x1 + Si (e(x,, t)dt. This avoids the need for pre-computed pairs. An additional potential improvement involves using shared random noise vector X₁ for both losses, which may enhance training consistency when sampling intermediate points. We leave detailed study of this method to future work.
3.2 Training from scratch and Sampling
It is noted that SCoT may also be trained from scratch, by modifying consistency loss and randomly choosing noise X1 and data Xg- Such exploration was not conducted due resource constraints. Regarding sampling, Algorithm Nottines the SCoT sample generation process, which may be imple- mented as a multi-step or single step procedure. Table and Table Jdebonsddmoottrate the corresponding experimental validation. Following the same setting as in Table 2 and Table our model is trained with the DSM loss adopted from CTM to enhance sample quality and training stability.
3.3
Connections to Other Distillation Methods
Methods
Reflow Liu 2023b
InstaFlow Liu et al. 2023cc
Objective function
Ex
f) (Re- x1 x1)
Ex
| 1) (8o- xx ]|
Methods
FlowDreamer
Li ct al. 2025
Lvelocity in SCoT (Ours)
Objective function
Bx,
[v, vo(x, {)]
Ex..c.a [| (1)) - (%o x, )
Table 1: Different objective functions in enforcing straight student trajectory. In addition to Reflow and SCoT, InstaFlow enables one step sampling by learning the magnitude of changes based on random noise X1, whereas FlowDreamer aims to approximate teacher model's 's velocity ve (x,, t).
The proposed SCoT bridges the gap between the consistency model and rectified flow distil- lations. On one hand, SCoT adopts the format of CTM Kim et al. (2024)'s 's project function Gx(x,, t, s). SCoT''s consistency loss and out- put reconstruction loss can be combined into CTM's integrated soft-consistency loss. While there is restriction on velocity, approximat- ing G (,, can be challenging when the student trajectory exhibits high curvature. As a result, multiple steps are usually required to generate high-quality images in CTM Kim al. (024)..
Algorithm 1 Sampling Procedure of SCoT
Input: Trained SCoT projection (x,,,t,)) steps IN = 1 >>
t =
Output: Generated image 8o
1: Sample initial: X,y π
2: for n = N-N
do
3: X GG (Sc, tu tn-1)
4: end for
5: return X
On the other hand, the velocity loss in SCoT shares the same target as Reflow Liu et al. (2023b). By enforcing constant velocity in Equation (2) SCoT induces straightening effect on the student trajectory. Table summarizes the different objective functions used to encourage straight student trajectories.
The Shorteut model Frans et al. (2025a) shares the same targets of trajectory consistency and straightness. While emphasizing learning trajectory velocities instead of the full trajectory. the Shortcut model still relies on an ODE solver for image sampling. Also, its usage of one-step Euler solver to enforce trajectory consistency might not be optimal. Other concurrent works such
5

---

as McanFlow Geng et al. (2025) and FlowMap Boff et al. (2025) Sabour et al. (025) have been discussed in Section
Boot Gu et al. 2023) also considers the velocity alignment with the pre-trained model, by comparing
the gradient of function. InstaFlow Liu et al. (222cc) and SlimPlow (hhu et al. (2025) focus on output
reconstruction. As velocity and consistency are not regularized, the trajectory is not well defined.
4 Experiments
4.1 Experimental Setup
Teacher-Student Distillation Setup. To simulate the generation trajectory from to the pre- rre-trained models are used to generate paired data samples (Xo.Xi). Furthermore, to provide the intermediate states required for computing the consistency term Equation (4). we distill knowledge from a teacher model 0 into student model Φ. Following established practices in CM ct al. (202)). SlimFlow Zhu et al. (2025). and CTM Kim et al. 2024). we evaluate SCoT on both CIFAR-10 10 and ImageNet, using pre- pre-trained diffusion checkpoints from EDM 2
(for CIFAR-10) and CM (for ImageNet) as teacher models. For the student model, we adopt EDM's DDPM++ implementation on CIFAR-10, and the Ablated Diffusion Model (ADM) architecture from Dhariwal and Nichol (2021) for ImageNet. To support the additional time conditioning 8 in Equation ( we incorporate several architectural modifications inspired by CTM. Specifically, we extend the temporal embedding by adding auxiliary s-embedding information to the original (-embedding. We also adopt other training heuristics from CTM (see Table 8), including: (1) using larger value of in the stop-gradient EMA to slow the teacher update rate and improve training stability; (2) setting the student EMA rate to 0.999 to smooth parameter updates over time and reduce training noise; and (3) reusing skip connections from the pre-trained diffusion model to facilitate gradient flow and preserve hierarchical features.
Model Architectures. To support the additional time conditioning variable s in our generator 9 (x, f, s). we incorporate a conditional embedding module into the baseline model architectures. following prior design choices in CM et al. (2023), SlimFlow Zhu et al. (2025), and CTM Kim al. (2024). For CIFAR-10. 10, we adopt the DDPM++ implementation from | Karras et al. (2022). extending its time embedding module to additionally encode s using structure symmetric to the original / embedding, and applying identical normalization strategies. For ImageNet, we use the Ablated Diffusion Model (ADM) Dhariwal and Nichol (202]) where s is embedded jointly with the class conditioning variable c, requiring no architectural change. A detailed comparison of DDPM+ and ADM in terms of their ResNet backbones, attention configurations, and noise conditioning strategies is presented in Table To improve the attention module, we address compatibility issues in DXVFlashAttenti by modilying the dimension operations in QKVAttenti onLegacy to match expected checkpoint formats. Additionally, we integrate xf ormers' ScaledDo trrouotoottuucctt attention an alternative backend. These improvements ensure correct weight loading while enhancing efficiency and flexibility of the attention mechanism.
Training and Sampling Hyperparameters. To ensure stable convergence and fair comparison during the training of SCoT,, we adopt higher learning rate for smaller datasets (e.g., CIFAR- 10) and scale the batch size appropriately for larger datasets (e.g., ImageNet). We use the Adam optimizer for both settings, and enable mixed-precision training (FP16) to improve memory efficiency and computational speed. For CIFAR-10, we train the model for 130k iterations with a batch size of 512, while for ImageNet we use a larger batch size of 2048 and train for 40k iterations to accommodate the increased data complexity and training cost. We adopt an exponential moving average (EMA) of the student model with decay rate of 0.9999 and employ stop- stop-gradient variant with EMA coefficient = 0.9999 to stabilize the training objective. A complete list of hyperparameter settings is provided in Table 8
httpe///ttthbb..mm//lalababab

---

Model Size
Generation Quality
NFE
Methods
FLOPss (G)
MACs (G)
Params (NB)
FID.
50
DDIM Song al.
a
12.2
6.1
35.7
4.67
20
DDIM Song Ci al.
2
12.2
6.1
35.7
6.84
DDIM Song el sl.
20214
12.2
6.1
35.7
8.23
RPM--sllve-22
202-2
5.94
DPM-solver-fase
2022
12.2
6.1
35.7
4.70
10
3-DEIS Zhang and Chen
2022
20.6
10.3
61.8
4.17
2
PD Salimans and Ho 2022
41.2
20.6
55.7
5.58
2
CT)
Song et al. 4124
41.2
20.6
55.7
2.93
2
CT
Song el al 2023
41.2
20.6
55.7
5.83
2
CTM
Kitn et
2024
41.2
20.6
55.7
1.87
SCo1
41.2
20.6
2.30
1
Rectiifie Flow (+Distill)
Liu
2023a
20.6
10.3
61.8
378
2-Rectified Flow (+Distill)) LiU
2023a
12.21
3-Rectified Flow (+Distill)
20.6
10.3
6188
8.15
CTM Kim al.
2024
41.2
20.6
55.7
1.90
SCol
41.2
20.6
55.7
2.40
Table 2: Comparison of -teppp (NFE) generation performance across diffusion models on CIFAR-10
comparable model scales. We report sample quality metrics-FID (4)-for N {1, 10, 20, 50).
Entries highlighted in bold denote our proposed method. CTM indicates the inclusion of GAN
loss. Baseline results are sourced from Song et al. (2023); Zhu et al. (2025): Frans et al. (2025b). "L" indicates that the result is not available.
Datasets. For evaluation, we adopt two large-scale real-world datasets with different image resolu- tions: CIFAR-10 (32 x 32) and ImageNet (64 x 64), following standard protocol. Dataset statistics, including resolution, total size, and sample count, are summarized in Table 10
Evaluation Metrics. We assess SCoT on unconditioned image generation using standard eval- uation metrics, including Fréchet Inception Distance (FID) Heusel et al. (2017), Inception Score (SSySmalaanss et al. (2016), and Recall (Rec.) (Rec.)[Sajjaii et (2018). To evaluate model efficiency, we also report parameter count, floating-poin point operations (FLOPs), and (MACs). See Section C.2 for further details.
accumulate operations
Time Efficiency. We measure the training throughput of the trajectory generator 9+(+) under different combinations of loss functions to evaluate time efficiency. Throughput results, reported in images per second per GPU, are summarized in Table II
4.2 Distillation Results
CIFAR-10. On CIFAR-10, 10, SCoT achieves an FID of 2.30 with only 2 NFEs, outperforming baselines such CD (FID 2.93) and CT (FID 5.83) while requiring fewer function evaluations. CTM attains a lower FID 1.87 with just 2 NFE, but its results incorporate a GAN loss, which significantly improves sample fidelity by providing strong adversarial signal during training. In contrast, do not adopt the GAN loss due to its instability under our training configuration and resource constraints. Despite this, SCoT delivers competitive performance without adversarial training, demonstrating favorable balance between generation quality and computational efficiency.
ImageNet. On ImageNet, SCoT achieves an FID of 2.60 with just NFEs, outperforming baselines such as CD (FID 6.20) and PD (FID 15.39), and closely approaching the performance of EDM (FID 2.44). While CTM achieves the best FID of 1.92, this result benefits from the integration of GAN loss, which enhances visual fidelity by introducing adversarial supervision during training. In contrast, our method does not incorporate GAN loss due its instability under our training regime, yet still attains competitive sample quality. SCoT also demonstrates strong diversity, with recall of 0.61 and an Inception Score of 68.20, highlighting its effectiveness balancing quality, diversity, and computational cost.

---

Model Size
Generation Quality
NFE,
Methods
FLOPs
MACs
Params
FID.
Rec.t
ISt
250
ADM Karras et al. 2022
2.07
0.63
79
EDM Dhariwal and Nichol
2021
219.4
103.4
295.9
2.44
0.67
48.88
2
PD
Salimans and Ho 2022
219.4
103.4
295.9
15.39
0.62
2
CDiSong et al. 2023
219.4
103.4
295.9
6.20
0.63
40.08
2
CTM Kim ct al. 2024
219.4
103.4
295.9
1.70
0.57
64.29
2
SCoT
219.4
103.4
295.9
2.60
0.61
68.20
CD Song et al. 2023
219.4
103.4
295.9
6.20
0.63
1
CT Song al.
2023
219.4
103.4
295.9
13.00
0.47
SlimF low Zhu al.
[2025
31.0
80.7
12.34
1
Shorttut(unnonnittonall Frans ct al.
2025b
219.4
103.4
295.9
20.50
Shorteut(conditional
Frans et al.
20256
219.4
103.4
295.9
40.30
CTM
Kim el al. (2024
219.4
103.4
295.9
1.92
0.57
64.29
SCoT
219.4
103.4
295.9
4.80
0.57
67.60
Table 3: Comparison of N-step generation performance by different DMs on ImageNet across corresponding model scales. Bold red numbers indicate the number of parameters for each distilled model. We report sample quality metrics -ID (4), Rec. (1), and IS (t) for N E f1,2, 10, 79, 250}. Entries highlighted in bold denote our proposed method. CTM" indicates the use of an additional
GAN loss. Baseline results are taken from SSnn et al. (2023); Zhu et al. (2025) Frans et al. (2025b).
"_" indicates that the result is not available.
4.3 Trajectory Analysis
We use CTM's soft consistency which proves to outperform local consistency and perform comparable to global consistency. Specifically, local consistency distills only 1-step teacher, so the teacher of time interval |0, At] is not used to train the neural jump starting from XT Rather, teacher on [t At. with t E [0, T At] is distilled to student from neural jump starting from Xt, not X7 The student, thus, has to extrapolate the learnt but scattered teacher across time intervals to estimate the jump from Xy', which could potentially lead to imprecise estimation. In contrast, the amount of teacher to be distilled in soft consistency is determined by random u, where μ O represents distilling teacher on the entire interval [0, T]. Hence, soft matching serves as a computationally efficient and high-perthee performirf loss.
4.4 Consistency Guarantee
The results in Table 4 highlight the benefits of consistency. By adding this loss, SCoT is encouraged to focus improving output reconstruction and consistency, which is reflected in the consistent reduction of FID from 15.7 to 5.6 in NFE = 1 and from 16.4 to 3.9 in NFE = 2. This optimization drives the model generate samples closer to the target distribution. As result, the IS increases from 30.4 to 63.1 in NFE = 1 and from 29.8 to 61.4 in NFE = 2, indicating notable improvements in both fidelity and diversity. Additionally, Precision and Recall show substantial gains rising from 0.49 to 0.72 and 0.45 to 0.57 in NFE 1, respectively, demonstrating that the model captures more accurate and diverse samples.
4.5 Velocity Guarantee
The incorporation of the velocity guarantee loss, as shown in Table 5, further improves the generative trajectory by encouraging temporal smoothness. This constraint ensures that the trajectories are straightened, making the sampling process more stable and efficient. The benefits are evident in the substantial reduction of FID, dropping from 14.7 to 4.8 in NFE and from 15.2 to 3.6 in NFE = The IS improves accordingly, reaching 67.6 and 68.2 in NFE- 1 and NFE 2, respectively. The model also achieves stronger coverage and accuracy, with Precision improving from 0.56 0.71 and Recall from 0.54 to 0.58 in NFE = 1. Similar trends are observed in NFE = 2. These confirm that velocity regularization helps refine the generative path, enabling the model to produce high-quality samples with better alignment to the underlying data manifold, especially under reduced NFE.
8

---

NFE- =
NFE- = 2
Metric
5k
10k
15k
20k
5k
10k
15k
20k
FID (L)
15.7
15.2
11.4
5.6
16.4
14.7
10.2
3.9
sFID (1)
33.8
32.8
31.7
16.2
36.5
34.9
29.7
13.6
IS (1)
30.4
34.2
43.6
63.1
29.8
33.7
42.2
61.4
Precision (1)
0.49
0.55
0.64
0.72
0.47
0.53
0.65
0.73
Recall (1)
0.45
0.49
0.57
0.54
0.44
0.47
0.58
0.56
Table 4: Comparison of training results for NFE and NFE = 2 for loss from Equation (4. All
models are trained on ImageNet and tested on 6k samples. Bold values indicate the best performance per metric.
NFE
1
NFE
2
Metric
5k
10k
15k
20k
5k
10k
15k
20k
FID (L)
14.7
14.1
10.4
4.8
15.2
13.8
9.1
2.6
sFID (1)
32.0
31.0
29.9
14.6
35.4
33.7
28.3
12.4
IS (1)
33.4
37.8
48.6
67.6
34.1
38.5
50.0
68.2
Precision (1)
0.56
0.61
0.67
0.71
0.55
0.60
0.68
0.73
Recall (1)
0.54
0.56
0.58
0.57
0.55
0.57
0.59
0.58
Table Training Equation (5)
Results for NFE and NFE 2. For metrics with 4, lower values are better; for metrics with 1. higher values are better. Bold values represent the best performance in each metric.
4.6 Loss Weighting
In this section, we explore the impact of loss weighting parameters our training objective defined by Equation (5). focusing specifically on Acon for the consistency loss and A for the velocity loss.
Weighting Acon in Equation We implemented adaptive weighting strategy for Acon to effectively balance the contributions between consistency and denoising losses during training. The primary motivation behind this adaptive mechanism to dynamically adjust the weighting based on the relative importance and magnitude of each loss component. Specifically, this adjustment is made through gradient magnitude comparisons. ensuring that no single loss dominates excessively, thereby stabilizing training and improving convergence.
To address the instability associated with the velocity loss defined in Equation (2). explored several loss weighting strategies. The primary source of instability stems from the computation of second-order derivatives using automatic differentiation tools such PyTorch's 's torch. autograd. grad(). (). These second-order computations require repeated differ- entiation, which can amplify small numerical inaccuracies through successive applications of the chain rule. As result, accumulated numerical errors may lead to unstable gradient magnitudes, manifesting as gradient explosion or vanishing gradients, and ultimately impairing training stability, Motivated by these challenges, we investigated alternative weighting approaches mitigate such instabilities.
Weighting on Avel Equation (5)
The first strategy, Adaptive Weighting, dynamically adjusts the weight of the velocity loss based on the scale of individual loss elements. As shown in Table this method performs relatively well in the early stages of training, but FID inereases notably in later iterations. This indicates that while adaptive scaling can help stabilize early training, it may overemphasize certain loss components later on, leading to degraded generative quality.

---

The second strategy, Without Scal- ing, removes all adaptive scaling fac- tors. This approach is motivated by the observation that unstable second- order gradients may produce extreme weight values, which can disrupt train- ing. As evidenced in Table 6 this method leads to slightly worse perfor- mance at early iterations but achieves improved stability and better FID scores in later stages.
5k
10k
15k
20k
Adaptive Weighting
15.3
14.5
11.2
6.2
Without Scaling
16.1
15.4
12.5
7.0
Normalization
15.2
13.8
9.1
3.6
Table 6: FID (1) comparison across different velocity loss weighting strategies at various training iterations (5k, 10k, 15k, 20k), using global batch size of 2048. Adaptive Weighting dynamically adjusts weights based on the relative scale loss components; Without Scaling removes adaptive weighting entirely; and Normalization stabilizes training by bounding the magnitude of the velocity loss.
The final strategy, Normalized Veloc- ity Loss, clips the adaptive weight within predefined range [0.01, 10) to prevent extreme values from dom-
inating the loss. This normalization consistently achieves the best FID scores across all iterations, suggesting it provides a stable and balanced contribution of the velocity loss during optimization. Notably, after 20k iterations, this strategy yields the lowest FID, demonstrating its effectiveness in maintaining training stability and improving overall model performance.
5
Conclusion
In this paper, we propose the SCoT model that successfully bridges the gap between consistency model and rectified flows distillation. SCoT performs pre-trained diffusion model distillations, and produces straight and consistent trajectories for fast sampling. By aligning the output, velocity, and consistency with the teacher model, SCoT achieves high-quality generation with fewer sampling steps. Experimental results on CIFAR-10 10 and ImageNet show that SCoT outperforms existing distillation methods both efficiency and generation quality. Ablation studies confirm the effectiveness of our design, highlighting the importance of trajectory consistency and velocity approximation. In the future, we aim extend SCoT to higolresol image synthesis and explore its integration with conditional generation tasks. We believe SCoT provides strong foundation for efficient generative modeling and paves the way for real-time high-fidelity image synthesis.
Acknowledgments and Disclosure of Funding
This work is sponsored by AMD's High Performance Compute Fund project. This work is also partially sponsored by the Australian Research Council Discovery grant DP240102050, ARC LIEF grant LE240100131, and ARC Linkage grant LP230201022.
10

---

References
Michael S Albergo, Nicholas M Boffi, and Eric Vanden-Eijneen Stochastic Interpolants: A Unitying Framework for Flows and Diffusions. Journal of Machine Learning Research, 2023.
Nicholas M. Boffi, Michael S. Albergo, and Eric Vanden-Eijnden. Flow map matching with stochastic inter- polants: A mathematical framework for consistency models, 2025.
Prafulla Dhariwal and Alexander Nichol. Diffusion Models Beat GANs on Image Synthesis. Advances in Neural Information Processing Systems, 2021.
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling Rectified Rlow Transformers for nigh-Resolution Image Synthesis. International Conference on Machine Learning, 2024.
Kevin Frans, Danijar Hafner, Sergey Levine, and Pieter Abbeel. One Step Diffusion via Shortcut Models. International Conference on Learning Representations, 2025a.
Kevin Frans, Danijar Hafner, Sergey Levine, and Pieter Abheel. One Step Diffusion via Shortcut Models. International Conference on Learning Representations, 2025b.
Zhengyang Geng, Mingyang Deng, Xingjian Bai, J. Zico Kolter, and Kaiming He. Mcan flows for one-step generative modeling, 2025.
Jiatao Gu. Shuangfei Zhai, Yizhe Zhang, Lingjie Liu, and Joshua M Susskind. Boot: Data-free Distillation of
Denoising Diffusion Models with Bootstrapping, International Conference on Muchine Learning, 2023.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans Trained by a Two Time-Scale Update Rule Converge a Local Nash Equilibrium. Advances in Neural Information Processing Systems, 2017.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising Diffusion Probabilistic Models, Advances Neural Information Processing Systems, 2020.
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating Design Space Diffusion-baan based Generative Models. Advances in Neural Information Processing Systems, 2022.
Dongjun Kim, Chieh-Hsin Lai, Wei-Hssang Liao, Naoki Murata, Yuhta Takida, Toshimitsu Uesaka, Yutong He, Yuki Mitsufuji, and Stefano Ermon. Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion. International Conference on Learning Representations, 2024.
Tuomas Kynkäännicmi, Tero Karras, Samuli Laine, Jaakko Lchtinen, and Timo Aila. Improved Precision and
Recall Metric for Assessing Generative Models. Advances in Neural Information Processing Systems, 2019.
Hangyu Li, Xiangxiang Chu, Dingyuan Shi, and Wang Lin. Flowdreamer: Exploring High Fidelity Text-to-3D Deneration via Rectified Flow. International Conference Learning Representations, 2025.
Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matl Le. Flow Matching for Generative Modeling. International Conference on Learning Representations, 2023.
Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow. International Conference Learning Representations, 2023a.
Xingchao Liu, Chengyue Gong, and qiang liu. Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow. International Conference on Learning Representations, 2023b.
Xingchao Liu, Xiwen Zhang, Jianzhu Ma. Jian Peng, et al. Instaflow: One Step is enough for High- Diffusion- based Text-to-Image Generation. International Conference on Learning Representations. 2023c.
Cheng Lu and Yang Song. Simplifying, Stabilizing and Scaling Continuous-time Consistency Models. Interna- tional Conference on Learning Representations, 2025.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in around 10 Steps. Advances in Neural Information Processing Systems, 2022.
Artem Lukoianov, Haitz Sáez de Ocáriz Borde, Kristjan Greenewald, Vitor Campagnolo Guizilini, Timur Bagautdinov, Vincent Sitzmann, and Justin Solomon. Score Distillation via Reparametrized DDIM. Advances in Neural Information Processing Systems, 2024.

---

Weijian Luo, Tianyang Hu, Shifeng Zhang, Jiacheng Sun, Zhenguo Li, and Zhihua Zhang, Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Models. Advances in Neural Information Processing Systems, 2024.
Thuan Hoang Nguyen and Anh Tran. Swiftbrush: One-Step Text-to- rext-to-Image Diffusion Model with Variational Score Distillation. JEEF/CVF Conference on Computer Vision and Pattern Recognition, 2024.
Ben Poole, Ajay Jain, Jonathan Barron, and Ben Mildenhall. Dreamfusion: Text-to-3D using Diffusion. International Conference on Learning Representations, 2023.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High figh-resolution Image Synthesis with Latent Diffusion Models. IEEE/CVF Conference Computer Vision and Pattern Recognition, 2022.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Adilya Khosla, Michael Bernstein, al. Imagenet Large Scale Visual Recognition Challenge. IJCV, 2015.
Amirmojtaba Sabour. Sanja Fidler, and Karsten Kreis. Align your flow: Scaling continuous-timg flow map distillation, 2025.
Mehdi SM Sajjadi, Olivier Bachem, Mario Lucic, Olivier Bousquet, and Sylvain Gelly. Assessing Generative Models via Precision and Recall. Advances in Neural Information Processing Systems, 2018.
Tim Salimans and Jonathan Ho. Progressive Distillation for Fast Sampling of Diffusion Models. International Conference on Learning Representations, 2022.
Tim Salimans, lan Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved Techniques for Training GANs. Advances in Neural Information Processing Systems, 2016.
Axel Sauer, Frederic Boesel, Tim Dockhom, Andreas Blatmmann, Patrick Esser, and Robin Rombach. Fast Hligh-eeoolttinn Image Synthesis with Latent Adversarial Diffusion Distillation. SIGGRAPH, 2024.
Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial Diffusion Distillation. European Conference on Computer Vision 2025.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising Diffusion Implicit Models, International
Conference on Learning Representations, 2021a.
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based Generative Modeling through Stochastic Differential Equations. International Conference on Learning Representations, 2021b.
Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency Models. International Conference on Machine Learning, 2023.
Christian Szegedy, Vincent Vanhoucke, Sergey Toffe, Jon Shlens, and Zbigniew Wojna. Rethinking the Inception Architecture for Computer Vision. IEEE//VF Conference on Computer Vision and Pattern Recognition, 2016.
Zhendong Wang, Huangjic Zheng, Pengeheng He, Weizhu Chen, and Mingyuan Zhou. Diffusion- GAN: Training GANs with Diffusion. International Conference on Learning Representations, 2023.
Zhengyi Wang, Cheng I Au, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun 7hu. Prolificdreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation. Advances in Neural Information Processing Systems, 2024.
Sirui Xie, Zhisheng Xiao, Diederik P Kingma. Tingbo Hou, Ying Nian Wu, Kevin Patrick Murphy, Tim Salimans, Ben Poole, and Ruiqi Gao. Em Distillation for One-Step Diffusion Models. Advances in Neural Information Processing Systems, 2024.
Yanwu Xu, Yang Zhao, Zhisheng Xiao, and Tingbo Hou. UFOGen: You Forward Once Large Scale Text-to- Image Generation via Diffusion GANs. IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.
Tianwei Yin, Michaël Gharbi, Taesung Park, Richard Zhang, Eli Shechtman, Fredo Durand, and William Freeman. Improved Distribution Matching Distillation for Fast Image Synthesis. Advances in Neural Information Processing Systems, 2024a.
12

---

Tianwei Yin, Michael Gharbi, Richard Zhang, Eli Shechtman, Fredo Durand. William T Freeman, and Taesung Park. One-Step Diffusion with Distribution Matching Distillation. IEEE/CVF Conference Computer Vision and Puttern Recognition, 2024b.
Qinsheng Zhang and Yongxin Chen. Fast Sampling of Diffusion Models with Exponential Integrator. Interna- tional Conference on Learning Representations, 2022.
Mingyuan Zhou, Huangjie Zheng, Zhendong Wang, Mingzhang Yin, and Hai Huang. Score Identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generation. International Conference on Machine Learning, 2024.
Mingyuan Zhou, Huangjie Zheng. Yi Gu, Zhendong Wang, and Hai Huang, Adversarial Score Identity Distilla- tion: Rapidly Surpassing the Teacher in One Step. International Conference on Learning Representations, 2025.
Yuanzhi Zhu, Xingchao Liu, and Qiang Liu. Slimflow: Training Smaller One-Step Diffusion Models with Rectified Flow. European Conference on Computer Vision, 2025.
13

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No] or [NA]. .
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justificatin right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]" ", is perfectly acceptable to answer "[No] "provided proper justification is given (c.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
general, answering
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Abstract and introduction accurately describe SCoT's dual objectives (consis- tency and straightness) and summarize experimental results CIFAR- 10 and ImageNet (Abstract; Sec. 1).
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
14

---

Justification:
Conclude in Conclusion.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approxinations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proof'
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification:
We contribute to the theory in distillation.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: All hyperparameters, model architectures, data splits, and training details are fully specified in Sec. 5 and Appendix A.2-A.4.
Guidelines:
15

---

The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors welcome deseribe the particular way they provide for reproducibilityy In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: Public data and code is in anonymous repo
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.. cc/
inspreeSSsumissiooooooccy for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https c/ipps.cc/publi///uue
Codeeuumiiii wettsbbmmssiiioi ssionoolicy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
16

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g.. data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Data splits, optimizer settings, learning rates, batch sizes, and evaluation metries are detailed in Sec. 5.1 and App. A.2-A.4.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Each group of experiments repeated multiple times eliminate the randomness.
Guidelines:
The answer NA means that the paper does include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report I-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: GPU type, throughput measurements, and total training iterations are specified in Sec. 5.1 and App. A.5.
Guidelines:
The answer NA means that the paper does not include experiments.
17

---

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics httpp::/aorrrss...ppall/t/ttittt
Answer: [Yes]
Justification: Research conforms to NeurIPS Ethics Guidelines; no human or sensitive data used.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification:
No discussion of potential societal impacts included.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path any negative applications, the authors should point it out. For example, it is legitimate point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (c.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
18

---

Justification:
No high-risk models or data releases involved.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging. and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification: Third-party model checkpoints and datasets are cited but licenses not explicitly stated.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi www.wwthhde...m caree.comdaee com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer:
[NA]
Justification:
No new datasets or code assets are released.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/mooel as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
19

---

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
No human subjects research conducted.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Question:
Answer: [No]
Justification:
LLMs not used in core methodology.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [No]
Justification:
LLMs not used in core methodology.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our L.MM policy /https://neurba wtt.rrisss.cccooeeeeeeee /c/eeonneerenccs//222////11 for what should or should be described.
20

---

Contents
Introduction
2
A trajectory perspective on Pre-frained diffusion model distillations
Straight-Consistent Trajectories (SCoT) model
3.1
Main Components of SCoT
3.2
Training from scratch and Sampling
Connections to Other Distillation Methods
4
Experiments
4.1
Experimental Setup
Distillation Results
4.3
Trajectory Analysis
4.4
Consistency Guaranted
8
4.5
Velocity Guarantee
4.6
Loss Weighting
5
Conclusion
10
Acknowledgments and Disclosure Funding
10
Theoretical Motivation for Loss Design
22
Related Work
22
Technical Details
23
Datasets
23
Metrics
23
Additional Experiments
25
D.I Time Efficiency
25
D.2 Sample Comparisons
25
21

---

A Theoretical Motivation for Loss Design
To better understand the design of our loss function, we provide brief explanation of why combining velocity and consistency objectives helps the model perform better.
The trajectory projection function is defined as
(1-7) ( (x(
x, t,
(6)
where Ig is a neural network that learns a residual correction. The first term encourages a straight path from x, to the target, while the second term provides flexibility.
The velocity loss encourages the trajectory to follow constant direction:
aGo
Liellcity
(Xo - x1)
Ös
The consistency loss encourages points from different timesteps to align at the same target point:
Cconsistaccy = t2,, [IG (XL, s)
Gx t2, })]?]
(8)
These two losses serve different purposes. The velocity loss reduces trajectory curvature, making it easier to approximate with fewer steps. The consistency loss ensures that the mapping valid across time. Together, they help improve both the quality and efficiency of generation, especially in low-step settings like 1-step or 2-step sampling.
B Related Work
Pre-trained diffusion distillation methods have emerged as a powerful strategy to alleviate the significant computational burden inherent in diffusion models, which traditionally require large number of function evaluations to produce high-quality samples. By compressing the iterative denoising process into far fewer steps. these techniques not only expedite sample generation but also make it fcasible to deploy such models in low-resource environments. In this work, we systematically categorize these distillation approaches into three major groups based on their underlying objectives and operational paradigms.
Output reconstruction based methods: These aim to minimize the discrepancy between the outputs of the teacher (i.e., the pre-trained diffusion model) and the student model by directly reconstructing image outputs. Some approaches, such Progressive Distillation [alimans and Ho (2022), focus on aligning output values by enforcing a close correspondence between the denoising steps of the teacher and the student. Other methods, for instance SDS and its variants Poole et al. (2023); Wang itt al. (2024); Yin et al.] (2024baa). concentrate on matching output distributions to preserve the one-step
statistical characteristics of generated images. Additionally, certain studies operate denoising image space Lukoianov et al. (2024); Karras et al. (002)), allowing for the direct generation of high-quality images from single function evaluation, while others employ Fisher divergence objectives Zhou et al. (2024) 2025) to more rigorously align the gradients of score functions. Together, these techniques effectively reduce the number of sampling steps required while maintaining the fidelity of generated outputs.
Trajectory distillation based methods: Instead of concentrating solely on the final output, trajectory- based methods focus on the entire denoising path-from the initial random noise to the eventual clean image. By distilling the full trajectory, these approaches ensure that the student model replicates not only the final result but also the dynamic behavior of the teacher model throughout the diffusion process. Consistency distillation techniques emphasize the self- eeff-consistency of the denoising trajectory, ensuring stable and accurate progression across different time steps. In contrast, rectified flow distillation methods such as InstaFlow and SlimFlow Liu et al. (2023c); Zhu et al. (2025) focus on producing a straighter, more direct trajectory, thereby mitigating the accumulation Moreover, recent studies have demonstrated that integrating consistency modeling directly into rectified flows flows/Frans et al. (2025a) can further enhance the fidelity of generated trajectories, effectively combining the strengths of both approaches.
al. (2023); Kim al. (2024); Lu and Song (2025)
approximation errors that typically arise from curved paths.
22

---

Adversarial distillation based methods: This category of distillation methods leverages adversarial learning to refine the student model's output distribution. By incorporating an adversarial often implemented via pre-trained classifier discriminator, these methods drive the student model to more closely approximate the target distribution provided by the teacher. Notably, studies such as Sauer et al. [024]] 2025) have successfully employed this strategy to achieve competitive perfor- mance with significantly fewer sampling steps. The adversarial framework not only enhances the perceptual quality of generated images but also provides a flexible plug-and-play mechanism that can complement other distillation strategies.
Notation
Description
Parameters of the generator/student model
Parameters the velocity/teacher model
L
Current time step (4 [0, 1))
8
Target time step for state projection
N
Number of discretized steps/evaluations
To
Data distribution at initial state (t = 0)
T
Noise distribution at terminal state
g
Integration approximation function
Velocity prediction function
G(-)
Distilled generator function
Table 7: Notations and corresponding descriptions used training SCoT.
Hyperparameters
CIFAR-10
ImageNet
Learning rate
0,0004
0,000008
Batch
512
2048
Student's stop-grad EMA parameter
0.9999
0.9999
Optimizer
Adam
Adam
Optimization
N
18
40
Training iterations
130K
40K
Hixed-Precision (FP16)
Enabled
Enabled
ODE Solver EMA decay rate
Heun
Heun
Score
0.999
0.999
Table 8: Training configuration of SCoT in different model sizes CIFAR-10 and ImageNet.
C Technical Details
C.1 Datasets
CIFAR-10 This dataset contains 60,000 32x32 color images evenly distributed over 10 distinet classes (6,000 images per class). is split into training set of 50,000 images and test set of 10,000 images, making a standard benchmark in machine learning and computer vision.
IetgeNet ImageNet is one of the most influential benchmarks computer vision. comprises over 1.2 million training images and around 50,000 validation images, categorized into 1,000 diverse classes. The dataset's vast scale and rich annotations have made it an essential resource for developing and evaluating deep learning models.
C.2 Metrics
Fréchet Inception Distance (FID). FID evaluates the similarity between real and generated samples by modeling their feature distributions multivariate Gaussians in pre-trained InceptionV3 feature
httpp::/www. cs. toronto. wwwpoeddddd"""dc//e httl http:://ww. mmage-net.orgg
23

---

Component
Functionality
DDPM+ (CIFAR-10)
ADM (ImageNet)
Downsampling method
Resize + Conv
Strided Conv
Upsampling method
Resize + Conv
Transposed Conv
ResNet Block
Time embedding type
Fourier
Positional
Normalization type
GroupNorm
LayerNorm
Residual blocks per resolution
3
Applied resolutions
16
32, 16,
Attention heads
1-8-12
Attention Module
Attention blocks (Down)
2
9
Attention blocks (Up)
13
Attention implementation
Vanilla
Multi-head self-attention
Label embedding
None
Learned class embedding
Extra temporal input (s)
Additive embedding
Additive embedding
Conditioning
Positional encoding added
No
Yes
Skip connections
Present
Present
Output scaling Activation
Yes
Yes
Output Head
Identity
Identity
Table 9: Comparison of U-Net architectural components used in DDPM+ (CIFAR-10) and ADM (ImageNet) within our distillation framework. Differences include normalization strategies, attention configuration, conditioning. and up/downsampling methods.
Data Shape
Dataset
Samples
Data Size
(3 x 32 x 32)
CIFAR-10
60K
160M
FFHQ-64
70K
5 GB
(3 x 64 x 64)
ImageNet
1.2M
100 GB
Table 10: Experimental details of datasets.
space and computing the Fréchet distance. This metric jointly captures fidelity and diversity. We adopt clean-fid [ for FID computation, which standardizes image preprocessing and Inception activations, thereby improving reproducibility across experiments.
Sliced Fréchet Inception Distance (sFID). sFID is computationally efficient variant of FID that approximates the Fréchet distance using one-dimensional projections feature embeddings. Instead of computing the full covariance matrix, sFID calculates the Wasserstein distance between sliced marginal distributions the Inception V3 feature space, offering lightweight and scalable measure of generation quality. We follow the same evaluation protocol as clean- ensuring consistency in feature extraction and preprocessing.
Precision and Recal We adopt the manifold-basee metrics introduced in Kynkäänniemi et al. (2019) to assess sample fidelity and diversity. Precision quantifies the fraction of generated samples that fall within the manifold of real data, while Recall measures how much of the real data distribution is covered by generated samples. The manifolds approximated via k-nearest neighbors in the Inception V3 feature space. Our implementation follows the version in the ADM repository ?
24

---

IS reflects both sample quality and class diversity by computing the KL divergence between the conditional class distribution and the marginal class distribution over all samples. It is calculated using logits from an Inception V3 model (Szegedy et al. (2016) trained on ImageNet[ [Russakovsky et al. (2015). Higher scores suggest high-confidence predictions and class diversity. However, when evaluating datasets with limited categorical variation (e.g., CelebA or FFHQ), IS primarily reflects fidelity. We follow the ADM implementation and evaluate IS using 10k generated images.
Inception Score (IS) 10
Parameter Counts (MB), We report the total number of learnable parameters in the model, measured in megabytes. This includes weights and biases from all trainable layers, such as convolutional filters and dense layer matrices. Higher parameter counts may indicate stronger representational capacity, while smaller models are better suited for deployment under resource constraints. For wrasformer-based architectures, include attention and feed-forward components.
Floating Point Operations (FLOPs). FLOPs measure the total number of floating-poin operations (additions and multiplications) required during a forward pass, serving as a proxy for computational cost. For convolutional layer of kernel size k over feature map of spatial size H x W, the FLOPs are computed as:
FLOPs 2 x H X W x k2 Cin Cout-
We compute FLOPs using the calflops utility providing an estimate of overall model complexity.
Multiply-Accumulate Operations (MACs), MACs count the number fused multiply-add compu- tations, which are often optimized as single hardware instructions modern accelerators. For the same convolutional layer, MACs are given by:
MACS = H x W x k22 x Cin x Cous-
We report MACs using the same tool 2, as they offer hardware- indicator of inference cost, especially relevant for edge deployment.
D
Additional Experiments
D.1
Time Efficiency
Table II concludes the throughput in training different loss function of our SCoT.
Dataset CIFAR-10
Equation
Equation 44
282.3
71.5
ImageNet
131.7
31.7
Table 11:
Time efficiency comparison (imgs/sec. on H100 GPU) of SCoT under different loss functions across datasets.
D.2 Sample Comparisons
Figure 3 Figure 4 and Figure 5 present the qualitative results of 1-step image generation on ImageNet using the SCoT sampler described in Algorithm These samples are taken different stages of training to illustrate the progressive refinement of the model. the initial stage of training, where the model produces blurry images with limited semantic structure. As training progresses to 10k steps (Figure 4), the generated images become more coherent, with clearer object boundaries and texture. 30k steps (Figure 5) the model generates high-quality, semantically consistent images, demonstrating the effectiveness of the training process. To ensure efficient evaluation and maintain consistent comparison across stages, we generate 6,000 samples at each step. Following the same setting as in Table and and Table
shows samples generated at
our model is trained with the DSM loss adopted from CTM Kim et al, (2044) to enhance sample quality and training stability.
10 "Batppe//ggtaab.ccc///paaaapappp htttp:://gituub..co//hrttrtt
flops. pytorch
25

---

Figure 3: 1-Step generation on the initial training stage for ImageNet by SCoT Algorithm sampler.
26

---

Figure 4: 1-Step generation on the 10k training stage for ImageNet by SCoT Algorithm sampler.
27

---

Figure 5: 1-Step generation on the 30k training stage for ImageNet by SCoT Algorithm sampler.
28


Reviews:
[{"review_id": "w1AeyGt1cn", "reviewer": "Reviewer_nd3F", "summary": "The paper presents SCoT (Straight Consistent Trajectories), a framework for distillation of pre-trained diffusion model for inference with fewer steps.  The approach combines the benefits of Rectified flow, for straightening the trajectory of diffusion models and Consistency models, that ensures that all points along a trajectory map to the same future step. The method is compared to several relevant baselines and state of the art quantitative performance is demonstrated", "questions": "1. What is the quantitative performance of generation of higher resolution images from the models that it was currently tested on?\n2. If CTM performs about the same, what is the need for using rectified flow for ensuring performance of a few step distilled model?", "limitations": "The authors fail to provide an adequate treatment of the limitations of the approach. In particular, the proposed approach has not been validated for high resolution outputs (which has been left for future work). Providing some context about the difficulty of the balancing the losses and the failure modes seen in the outputs of the distilled models would be helpful.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "## Strengths:   \n\n1. **Simple and elegant**: The presented framework is an elegant combination of the best aspects of rectified flow based models and consistency models\n2. **Ablation**: The authors provide a number of ablations that justify their design choices.\n3. **Supplementary**: The appendix provides a lot of useful information about the the loss design. \n4. **Quantitative performance**: The approach shows quantitative performance similar to state of the art CTM model. \n5. **Clarity**: The paper is well written with all the necessary details explained adequately. \n\n## Weaknesses:\n\n1. **Novelty**: The novelty of the approach is somewhat limited to the combination of losses from Rectified flow and consistency models. Details regarding the key considerations were needed to make these two losses work together would be insightful. \n2. **Qualitative results**: The authors show a number of tables demonstrating quantitative performance, however only include few images for the qualitative results. Including the qualitative performance of CIFAR would be helpful. Furthermore, demonstrating the performance on a more general class of diffusion models would help strengthen the claims of the paper.  \n3. **Experiments on limited resolution**: The experiments are performed on images upto a resolution of 64x64 where the effect of faster sampling might not be too apparent. In particular, showing performance improvement on larger resolution while still maintaining quality would be beneficial.", "was_revised": true, "final_justification": "The rebuttal does a reasonable job of addressing all of the concerns raised. To that end, I am keeping my current score of acceptance.", "rebuttal": "We thank the reviewer **nd3F** ’s constructive comments and provide our responses below. \n\n> \n\n>**W1**: Novelty: The novelty of the approach is somewhat limited to the combination of losses from Rectified flow and consistency models.\n\n>\n  \n\n**[SCoT DOES NOT use the loss from Rectified flow.]** In fact, **our SCoT is the first to discover the connections between CTM and rectified flow, by observing that $\\frac{\\partial G\\_{\\phi}(x\\_t, t, s)}{\\partial s}$ matches to the velocity of $x\\_s$.** This discovery is nontrivial. As a result, the velocity loss in SCoT is not the same as the one in rectified flow. \n\n\n> \n\n>**W1.2**: Details regarding the key considerations were needed to make these two losses work together would be insightful.\n\n>\n\nWe agree that combining trajectory straightness and consistency requires careful design. These two objectives act on different aspects of the trajectory and are not in conflict: one regularizes the *direction* (velocity), and the other ensures *alignment* across time. To further balance their effects, we introduce a joint objective in Equation (4), with normalization strategies discussed in Section 4.6 to ensure stable and effective optimization. We clarify this rationale further in Appendix A.\n\nThe intuition behind this dual-objective design is also visually demonstrated in Figure 1 and theoretically motivated in Appendix A.\n\n>\n\n>**W2**: Qualitative results: The authors show a number of tables demonstrating quantitative performance, however only include few images for the qualitative results. Including the qualitative performance of CIFAR would be helpful. Furthermore, demonstrating the performance on a more general class of diffusion models would help strengthen the claims of the paper.\n\n>\n\n**[Instead, we reported qualitative results on ImageNet in Appendix Figures 3–5 of SCoT, and quantitative results on CIFAR-10 are provided in Table 2.]** \n\n \n\nDue to the **policy prohibiting  updates to the anonymous repository this year** we were unable to include CIFAR-10 visualizations. We chose to show ImageNet-64 (1M samples with $64\\times 64$ resolution) samples instead, as the task is more challenging than CIFAR-10 (50K training samples with $32 \\times 32$ resolution). Nevertheless, Table 2 shows strong quantitative results on CIFAR-10 across multiple metrics, indicating high-quality generation. \n\n \n>\n\n>**W3**: Experiments on limited resolution: The experiments are performed on images upto a resolution of 64x64 where the effect of faster sampling might not be too apparent. In particular, showing performance improvement on larger resolution while still maintaining quality would be beneficial.\n**Q1**: What is the quantitative performance of generation of higher resolution images from the models that it was currently tested on?\n\n>\n \n\n**[We follow previous protocols to evaluate SCoT, CIFAR-10 and ImageNet-64 provide sufficient validation of model effectiveness, especially given that high resolution data is typically handled by latent diffusion models that compress images to 64×64 in practical applications.]**  \n\n \n\nOur experiments are based on widely used benchmarks such as CIFAR-10 and ImageNet-64, following the setup in Consistency Models (ICML23), CTM (ICLR24). These datasets support consistent and reproducible evaluation in low resolution settings, which is the current focus of most trajectory distillation studies. \n\n  \n\nAlso, for larger datasets such as ImageNet-256, suitable pretrained models with compatible outputs like velocity fields or projection functions are not publicly available. Training such teachers from scratch would require substantial computational resources, which fall outside the current scope of this work. \n\n \n\nFurther, many diffusion models can already be observed at $64\\times 64$ resolution and adapt to latent diffusion models. In high resolution image synthesis (e.g., $512\\times 512$ or $1024\\times 1024$), it is common to use VAE encoder in latent diffusion models to compress images into a lower-dimensional latent space, often around $64\\times 64$. Therefore, studying models at this resolution is both meaningful and aligned with how large-scale generation is typically implemented. \n  \n\n \n>\n\n>**Q2**: If CTM performs about the same, what is the need for using rectified flow for ensuring performance of a few step distilled model?\n\n>\n\n**[The necessity of using velocity loss has been verified in Section 4.4, 4.5, which executes ablation studies for individual components of Equation (2) and Equation (3).]** Specifically, we evaluate the individual impact of the velocity loss and the consistency loss, showing how each contributes to the final performance. The results demonstrates that both velocity loss and consistency loss are essential for achieving fast and high-quality generation. \n  \n \n>\n\n>**L1**: The authors fail to provide an adequate treatment of the limitations of the approach. In particular, the proposed approach has not been validated for high resolution outputs (which has been left for future work). Providing some context about the difficulty of the balancing the losses and the failure modes seen in the outputs of the distilled models would be helpful.\n\n\n>\nPlease refer to **Reviewer ovkA L1**."}, {"review_id": "CtaopjqDXg", "reviewer": "Reviewer_dSXG", "summary": "The paper proposes the Straight Consistent Trajectory (SCoT) model, a new distillation methodology for generative diffusion models based on a pre-trained teacher model that is purported to enjoy the advantages of rectified flow and consistency model distillation, two leading modes of diffusion model distillation, without their limitations (namely reduced sampling efficiency and error accumulation). SCoT is designed to be fast and to yield straight and consistent trajectories. This is achieved via a carefully designed projection function -- mapping the current value of the trajectory to any future value -- by minimizing a loss function comprised of two terms, one ensuring that the the gradient of the projection mapping remains as constant as possible, the other corresponding to the soft-consistency loss of Kiem at al. (2024), enforcing consistency for the student's trajectories. The good empirical performance of SCoT relative to several other distillation procedures is demonstrated through various numerical experiments on the CIFAR-10 and  ImageNet datasets using various evaluation metrics for efficiency and quality of the sampled images. Though the CMT methodology of Kim et al. (2024) achieves lower FID scores, SCoT is a close second. Further experiments demonstrate the benefits of a two-pronged loss and the use of a loss weighting parameter to achieve an optimal trade-off between the consistency and velocity components of the loss function.", "questions": "I have only very minor comments/questions.\n\n- Line 130: there seems to be a typo in the chain of inequalities - shouldn't it be s > t_1 > t_2 (according to eq (3)) or s < t_1 < t_2 (according to the notation introduced on page 2 that t_1 < t_{i+1}?\n\n- The legends of Tables 4 and 5 do not specify that the columns correspond to training iterations. Thts is explained only later, in the legend of table 6.", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "# Strength\nThe SCoT distillation procedure for diffusion models is well-motivated and shown empirically to be very competitive. The paper is clearly and carefully written, provides good reference and extensive context for its results. The numerical experiments are carefully designed and compelling.\n\n\n\n# Weaknesses\nThere are no evident weaknesses in the paper or the proposed algorithm. The fact that the CMT methodology of Kim et al. (2024) outperforms SCoT in terms of FID scores should not be considered a weakness.", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank reviewer **dSXG** for the encouraging feedback of our work and the constructive suggestions. Below we address the two points raised:\n\n>\n\n>**Q1**: Line 130: there seems to be a typo in the chain of inequalities - shouldn't it be s > t_1 > t_2 (according to eq (3)) or s < t_1 < t_2 (according to the notation introduced on page 2 that t_1 < t_{i+1}?\n\n\n>\n\n**[The inequality has been corrected to reflect the correct temporal order as defined in Equation (3).]**\n\nThank you for pointing this out. The original inequality was inconsistent with the temporal ordering $ s< t_1 < t_2$ described on Page 2. We have revised Line 130 to ensure it aligns with the notation and definitions used throughout the paper.\n\n>\n\n>**Q2**: The legends of Tables 4 and 5 do not specify that the columns correspond to training iterations. Thts is explained only later, in the legend of table 6.\n\n>\n\n**[We have updated the captions of Tables 4 and 5 to clarify that the columns correspond to training iterations.]**\n\nWe appreciate the observation. The original legends did not clearly indicate that the columns refer to specific training steps. We have now revised the captions of Tables 4 and 5 to explicitly state that the columns correspond to results at different training iterations consistent with Table 6."}, {"review_id": "OlXYfPrMt1", "reviewer": "Reviewer_ovkA", "summary": "This paper introduces SCoT, a method that combines consistency models with straight trajectory approaches to improve few-shot image generation via distillation. SCoT enforces straighter trajectories through a novel velocity regularizer, while a consistency time-distillation loss maintains vector field consistency. Experimental results on CIFAR-10 and ImageNet show improvements over selected baselines.", "questions": "Please see the listed weaknesses. \nAdditionally, is there an analysis which actually measures the straightness to confirm whether the given objective function is working as intended?", "limitations": "Although author has highligthed future work as part of the conclusion, I would recommend adding the limitations explicitely and talk about the", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n- The proposed objective, particularly the velocity loss (L_velocity), is a novel and intuitive way to reduce vector field curvature in consistency-based flow models.\n- The paper is clearly written and easy to follow.\n- Empirical results demonstrate competitive improvements.\n\nWeaknesses:\n- Missing comparisons to several relevant baselines, including Score Implicit Distillation (SiD), Score Implicit Matching (SIM), Flow Consistency Distillation, Rectified Flow++, and SlimFlow.\n- Most state-of-the-art methods include adversarial training for enhanced performance; it is unclear if SCoT is compatible with such approaches or how it would perform.\n- The method involves double higher-order gradients, which can be unstable and may introduce reproducibility challenges. The paper lacks analysis or discussion of optimization stability and computational overhead.\n- Limited ablation studies:\n    - Lacks >2 NFE (Number of Function Evaluations) FID curves.\n    - No NFE vs. FID comparison with baselines.\n    - No detailed study of the impact of individual loss functions.", "was_revised": true, "final_justification": "Authors have addressed most of concerns. However, novelty stays limited. Hence, I will only increase the score to borderline accept.", "rebuttal": "We appreciate the reviewer **ovkA** for the constructive and thoughtful feedback. We answer the reviewer’s comments below. \n \n>\n\n>**W1**: Missing comparisons to several relevant baselines, including Score Implicit Distillation (SiD), Score Implicit Matching (SIM), Flow Consistency Distillation, Rectified Flow++, and SlimFlow.\n \n\n>\n  \n\n**[We have compared with several relevant baselines and excluded others due to fundamental differences in methodology.]** \n\nAlthough the comparison is not strictly equivalent due to differences in model architectures and pretraining strategies, we still list the best CIFAR-10 results as reported in the original papers: SiD [A] (Table 1), SiM [B] (Table 1), CFD [C] (not reported), RectFlow++ [D] (Table 3), and SlimFlow-15.7MB [E] (Table 1).\n\n| Models | Goal | FID |\n|--------------|----------------|-------------------|\n| SiD [A]      | Aligning student's score with the teacher's       | 1.923   |\n| SiM [B]        | Minimize score difference over generated samples    | 2.06   |\n| CFD [C]         | Focus on 3D data generation using NeRF-based techniques        | NA   |\n| RectFlow++ [D]         | Improved techniques for learning consistent velocities      | 3.07    |\n| SlimFlow-15.7MB [E]         |  Results included in Table 3 | 5.02     |\n| SCoT(ours)            | Straight and consistent trajectory function       | 2.40        |\n  \n\n[A] Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generation. (ICML2024 ) \n\n[B] One-Step Diffusion Distillation through Score Implicit Matching.( NeurIPS 2024) \n\n[C] Consistent Flow Distillation for Text-to-3D Generation. (ICLR2024) \n\n[D] Improving the Training of Rectified Flows.( NeurIPS 2024)  \n\n[E] SlimFlow: Training Smaller One-Step Diffusion Models with Rectified Flow.(ECCV 2024) \n\n  \n\n \n\n \n>\n\n>**W2**: Most state-of-the-art methods include adversarial training for enhanced performance; it is unclear if SCoT is compatible with such approaches or how it would perform.\n\n>\n\nPlease refer to **Reviewer qykR  Q 1.1, 1.3** \n\n>\n\n>**W3**: The method involves double higher-order gradients, which can be unstable and may introduce reproducibility challenges. The paper lacks analysis or discussion of optimization stability and computational overhead.\n\n>\n\nPlease refer to **Reviewer Yq6Q W1**. \n  \n>\n\n>**W4.1**: Lacks >2 NFE (Number of Function Evaluations) FID curves.\n\n>\n\n**[FID curves vs NFE  on ImagNet-64 and CIFAR-10 are provided here.]**  \n\nWe report FID scores across multiple NFE values, including 3, 18, and 20 on both CIFAR-10 and ImageNet-64. \n\n| NFE  | CIFAR-10 FID ↓ | ImageNet-64 FID ↓ |\n|--------------|----------------|-------------------|\n| 3            | 2.07           | 3.21              |\n| 18           | 1.86           | 1.93              |\n| 20           | 1.89           | 1.94              |\n\nThese results help assess the scalability and convergence behavior of SCoT under increased sampling steps. \n \n  \n \n>\n\n>**W4.2**: No NFE vs. FID comparison with baselines.\n\n> \n\n**[We included the FID compared with different NFE in Table 2 and 3.]** \nWe appreciate the reviewer’s comment regarding the comparison of FID performance across different NFEs. As shown in Table 2 (CIFAR-10) and Table 3 (ImageNet), our paper already provides a detailed comparison of FID scores under various NFE settings (N $\\in$ {1, 2, 10, 20, 50}) across multiple baseline methods, including CTM, SlimFlow, and DDIM variants. These tables reflect the exact trends of FID vs. NFE that the reviewer is concerned about. \n\n \n\n \n>\n\n>**W4.3**: No detailed study of the impact of individual loss functions.\n\n>\n\n**[The performance of individual loss functions are presented in Table 4, Table 5]** \n\n>\n  \n>**Q1**:  is there an analysis which actually measures the straightness to confirm whether the given objective function is working as intended?\n\n>\n\n**[The straightness is evaluated in NFE=1 in table 2,3 and section 4.4.]** \n  \n>\n\n>**L1** Limitations Claim.\n\n>\n\nWhile SCoT achieves efficient and high-quality generation through trajectory straightness and consistency, several limitations remain for real-world deployment. First, the method depends on access to high-quality pre-trained teacher models, which may not be available in certain domains such as medical imaging, scientific simulation, or low-resource languages. This reliance constrains SCoT’s direct applicability to cases where teacher models cannot be easily obtained. Second, although SCoT demonstrates strong performance on low-resolution datasets like CIFAR-10 and  ImageNet-64, its scalability to high-resolution generation (e.g., $256 \\times 256$ or above), video synthesis, or other modalities such as audio or 3D remains unexplored."}, {"review_id": "ZblRK01rN4", "reviewer": "Reviewer_Yq6Q", "summary": "This paper introduces SCoT (Straight Consistent Trajectory), a distillation method for pre-trained diffusion models that combines the advantages of consistency model distillation and rectified flow approaches. The method generates trajectories that are simultaneously straight and consistent by optimizing two objectives: maintaining constant velocity gradients and ensuring trajectory consistency across different time steps. SCoT avoids the approximation errors inherent in ODE solvers used by rectified flows while addressing the sampling efficiency limitations of consistency models.", "questions": "See Weakness. \n\n\nI would be willing to increase the score if the evaluation protocol and training-related concerns I raised are adequately addressed.", "limitations": "While the paper mentions limitations in the conclusion, they are not sufficiently explored.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strength**\n\nThe paper addresses key challenges in diffusion model distillation with a well-motivated approach. It clearly distinguishes itself from existing methods by showing comparisons with other approaches through clear figures and tables which helps understanding. The idea is straightforward and yields solid results, particularly achieving comparable performance to CTM without requiring GAN loss. In particular, the diversity-related measures, which tend to be somewhat weak in consistency model approaches, show favorable results here.\n\n**Weakness**\n\n1. Using gradients in the loss function likely increases training memory requirements. A comparison of memory usage against other methods like rectified flow would be valuable.\n2. Learning curves showing training stability over time would strengthen the evaluation.\n3. The amount of teacher-generated paired datasets used is unclear, which is concerning since this represents an inefficient procedure that consistency models can avoid. An analysis of how performance degrades with smaller paired datasets would be valuable for understanding the method's practical limitations.\n4. Table 2's NLL column either should include SCoT values or be removed entirely, as current version provide little insight.\n5. I have concern about the evaluation consistency across the methods. Using clean-fid implementation may not ensure fair comparison with other reported results. For fair comparison, it would be necessary to demonstrate equivalence with other implementations or adopt consistent evaluation protocols across all methods.\n6. I think the paper is missing method details. Section 4.4 discusses \"output reconstruction loss\" not described in the methodology, creating confusion about the actual loss formulation.\n7. Line 253 appears to contain an error regarding SlimFlow's FID score.\n\nMinor: Table 3 has swapped references for ADM and EDM.", "was_revised": true, "final_justification": "The authors have provided a satisfactory response to my key concerns, particularly regarding the memory issues and the proposed evaluation protocol.", "rebuttal": "We appreciate the reviewer **Yq6Q** for the deep understanding of our work. \n\n>\n\n>**W1**: Using gradients in the loss function likely increases training memory requirements. A comparison of memory usage against other methods like rectified flow would be valuable.\n\n>\n\n**[SCoT introduces only ~10% additional memory usage ]**. We reported memory usage under identical training conditions:   \n\n| Method  | Peak memory (GB) | Relative overhead |\n|---------|------------------|-------------------|\n| CTM     | 67.5             | -                 |\n| **SCoT** | 76.2             | +10.0%            |\n\n\n\nThe results show that SCoT introduces only ~10% additional memory usage, which is negligible compared to the total training cost.   \n\nIn detail, computing the second order derivatives is implemented in our anonymous repository cm/karras_diffusion.py as:   \n\n```python \n# Second-order gradient for velocity loss \ngrads = torch.autograd.grad( \n    outputs=group,        # aggregated scalar over batch \n    inputs=s,             # low-dimensional time embedding \n    grad_outputs=torch.ones_like(group).detach(), \n    create_graph=True,    # allow second-order gradient \n    retain_graph=True, \n    only_inputs=True \n)[0] \n\n``` \n This design keeps the computational graph lightweight because the time embedding $s$ is a per-sample scalar with shape $[B]$ rather than a full spatial tensor. Although $s$ is broadcast to $[B,1,1,1]$ when combined with $x_t$, the autograd graph is only built for the original $[B]$ tensor. Therefore, no full Hessian over image features or model parameters is constructed, and the memory cost scales linearly with the batch size without growing with image resolution. \n\nTo quantify this overhead, we measured the peak GPU memory usage on ImageNet-64 (global batch size = 64, microbatch = 18) using `torch.cuda.max_memory_allocated()` on a single H100 80GB GPU. \n\n \n>\n\n>**W2**: Learning curves showing training stability over time would strengthen the evaluation.\n\n>\n\nMore details can be seen in **Reviewer qykR Q1.1** \n\n>\n\n>**W3**: The amount of teacher-generated paired datasets used is unclear, which is concerning since this represents an inefficient procedure that consistency models can avoid. An analysis of how performance degrades with smaller paired datasets would be valuable for understanding the method's practical limitations.\n\n>\n\n**[SCoT DOES NOT need to use student-teacher paired datasets.]** In each time calculating the velocity loss, SCoT can use pre-trained diffusion model to replace the teacher model's result $\\widehat{x}\\_0$ as $\\widehat{x}\\_0=G\\_{\\phi}(x\\_1, 1, 0)$. This usage is the same as consistency distillation. There is no algorithmic difference in training SCoT.\n\n**[We retrained SCoT with smaller data on CIFAR-10.]** \n\n| Iteration | FID (Full 50k) | FID (25k Subset) |\n| --------- | -------------- | ---------------- |\n| 5k        | 80.6           | 86.7              |\n| 10k       | 41.2           | 44.9             |\n| 15k       | 31.4           | 37.3             |\n| 20k       | 20.3           | 26.8             |\n| 25k       | 12.4           | 19.7             |\n\nThe second column displays the results on the entire dataset (50k), and the third column shows the results on half of the dataset (25k). This table evaluates how FID evolves over training iterations using the reduced dataset (NFE=1, BZ=512). \nWe can see the diversity of base samples impacts the quality of learned trajectories. Due to limited time, we were unable to run more and over experiments.  \n  \n \n\n  \n\n \n\n >\n\n>**W4**: Table 2's NLL column either should include SCoT values or be removed entirely, as current version provide little insight.\n\n >\n\n**[We will delete this column].** Thanks for the suggestion. \n\n  \n\n>\n\n>**W5**: I have concern about the evaluation consistency across the methods. Using clean-fid implementation may not ensure fair comparison with other reported results. For fair comparison, it would be necessary to demonstrate equivalence with other implementations or adopt consistent evaluation protocols across all methods.\n\n>\n\n**[We adopt the Clean-fid from CTM without modifications and with the unified hypers].**   \n\nWe clarify that our work does not introduce a new variant of the FID metric. Instead, we follow the clean-fid implementation in CTM. This implementation retains the original computation of the Fréchet distance on InceptionV3 pool3 features but addresses known issues in the earlier TensorFlow-based version. For example, it ensures reproducibility by locking the InceptionV3 model weights and consistently extracting features from the “pool3:0” layer. To improve numerical stability, it adds a small diagonal offset when the covariance matrix is close to singular. \n\n  \n\nThese adjustments do not change the underlying definition of FID, but help ensure consistent and reliable evaluation across frameworks.   In addition to FID, we also report Precision/Recall, and Inception Score to provide a more complete picture of both fidelity and diversity.  \n\n  \n\n>\n\n>**W6**: I think the paper is missing method details. Section 4.4 discusses \"output reconstruction loss\" not described in the methodology, creating confusion about the actual loss formulation.\n\n>\n \n\n**[The reconstruction loss is included in Eq. (3)].** The “output reconstruction loss” mentioned in Section 4.4 refers to the consistency loss already defined in Equation (3). We will clarify this naming to avoid confusion and ensure it aligns with the formulation in the methodology. When the target time step $s = 0$, this loss reduces to reconstructing the clean image $x_0$, effectively acting as an output reconstruction loss. We agree the current wording is unclear and will unify the terminology to avoid ambiguity. \n\n  \n>\n\n>**W7**: Line 253 appears to contain an error regarding SlimFlow's FID score.\n\n>\n\n \n\n**[SlimFLow's FID score is consistent with the original value in one-step 81MB SlimFlow's paper Table2].**  We will update the text to make it clearer."}, {"review_id": "6vfPfN62RJ", "reviewer": "Reviewer_qykR", "summary": "The paper proposes SCoT, an improved consistency distillation method that unifies the benefits of consistency models and rectified flows. SCoT introduces a projection function that ensures straight and consistent trajectories between noise and clean images, enabling fast and accurate sampling. \n\nIt avoids the approximation errors of ODE solvers by directly learning the trajectory, improving both performance and stability. \n\nThe key contribution is simple and trivial: calculated the derivative of the trajectory in continuous domain.", "questions": "Q1. It is unclear why the authors did not include a comparison with CTM augmented with GAN loss. Based on the current results, the proposed method does not appear to outperform existing consistency-based distillation approaches. \n\nSuch omission is particularly confused me given that the anonymized code provided by the authors already includes a discriminator module. As it currently stands, the authors’ method for computing trajectory derivatives does not appear to substantiate the claim made in the introduction that it enables more effective and stable distillation.\n\nThe authors should clarify this experimental setup.\n\nQ2. The paper lacks comparisons with several important baselines, such as the Diff-Instruct series, DMDs, and sCM. Notably, many of these related works report results that surpass those presented by the authors.\n\nThe emphasis on experimental results stems from the simplicity of the proposed approach, which primarily involves converting discrete steps into a continuous domain. As such, stronger empirical evidence is needed to convincingly demonstrate the effectiveness of the method and to substantiate the authors’ claims.\n\nQ3. I do not consider the claimed connection to flow matching to be a primary contribution of the paper. Flow matching and diffusion models differ mainly in parameterization, while their core principles remain largely the same. For a more detailed discussion, please refer to sCM.\n\n[1] Luo, Weijian, et al. \"Diff-instruct: A universal approach for transferring knowledge from pre-trained diffusion models.\" Advances in Neural Information Processing Systems 36 (2023): 76525-76546.\n\n[2] Yin, T., Gharbi, M., Zhang, R., Shechtman, E., Durand, F., Freeman, W. T., & Park, T. (2024). One-step diffusion with distribution matching distillation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 6613-6623).\n\n[3] Lu, C., & Song, Y. (2024). Simplifying, stabilizing and scaling continuous-time consistency models. ICLR 2025", "limitations": "The authors do not claim the limitations.", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n\n- The motivation is simple and sounds valid: to better estimate the mapping of the trajectory\n\n- The entire work is written very clearly. Figure 1 is very intuitive.\n\nWeaknesses\n\n- The comparison is seriously lacking, which significantly undermines the effectiveness of the proposed approach.\n\n- The discussion of related work is seriously insufficient, especially given that the main contribution of this paper lies in the distillation aspect.\n\n- The authors' explanation of the results is not sufficiently convincing.\n\nPlease check the Questions part for the detail.", "was_revised": true, "final_justification": "Despite the author providing some rebuttals, the experimental results cannot prove that the proposed solution provides effective improvements. Therefore, I recommend rejection.", "rebuttal": "We appreciate the reviewer **qykR**  comments and suggestions, which help further improve the paper quality. Below, we respond to each of your comments and clarify your misreading/misunderstanding as well.  \n\n> \n\n> **Q3** : I do not consider the claimed connection to flow matching to be a primary contribution of the paper. Flow matching and diffusion models differ mainly in parameterization, while their core principles remain largely the same. For a more detailed discussion, please refer to sCM.\n\n> \n**[The primary contribution of the paper is connecting consistency model and flow matching (also diffusion model), NOT connecting diffusion model to flow matching.]** The close connection between diffusion model and flow matching, including the reparameterization aspects, is well known. That is the reason we put diffusion model and flow matching (rectified flow) in the same part (lines 160 - 174).  \n\nWhile sCM also considers using flow matching in consistency model, it focuses on improving training consistency model in continuous time steps and does not involve straight trajectory.\n\n\n\n>\n\n> **Q1.1**: GAN loss not included in SCoT. \n\n>\n\n**[The reason of removing GAN loss indicated in Line 240].** We didn't include the GAN loss mainly due to reproduction issues with CTM's[F] setup and some training stability problems. The CTM official repo doesn't have experimental settings of pretrained classifier for CIFAR-10, making it impossible for us to reproduce their results or adapt them to SCoT. When we tried using the $64 \\times 64$ classifier for ImageNet-64, there are some training stability issues even with warm-up strategy.  \n\nDue to the **policy prohibiting updates to the anonymous repository this year**, we cannot update our anonymous repo and provide the learning curves and accuracy of classifier over iterations, but we tabularize  some  loss from Equation (3)  to show our evidence in training SCoT (ImageNet-64,one-step generation).  \n\n| Iterations | W/o GAN Loss | With GAN Loss | Classifier Accuracy |\n|------------|---------------|----------------|----------------------|\n| 10k | 0.205   | 0.230| 0.71  |\n| 15k | 0.190  | 0.185| 0.78 |\n| 20k | 0.184  | 0.170| 0.80 |\n| 25k | 0.178  | 0.196| 0.79  |\n\nThe column of \"With GAN Loss\" showed clear fluctuations, which affected training stability in column3 and  creates a disconnect where high accuracy doesn't necessarily mean better data likelihood because of its adversarial nature in column 4.  \n\n[F] Consistency Trajectory Models: Learning probability Flow ODE Trajectory of Diffusion. (ICLR2024) \n \n> \n\n> **Q1.2**: SOTA concern compared with CTM. \n\n> \n\n**[SCoT is comparable with CTM without GAN loss.]**  When we compare with CTM using the same experimental setup (without GAN loss), the comparison is fair. The CTM results we report in Tables 2-3 are their results *with* GAN loss, as noted in our captions. Here's the issue: while CTM's published results look impressive, we couldn't actually reproduce them due to missing implementation details,backbone differences (like $\\texttt{QKVFlashAttention}$ in CTM vs $\\texttt{XformersAttention}$ for other setups - see CTM repo issues 5 and 8 about transformer structures and sampling) and GAN loss issue(Q1.1). CTM's own appendix C.4 Table 3 shows how much GAN loss matters on CIFAR-10. At NFE=1, their FID improves dramatically from 5.19 to 2.28 with GAN loss. Since we don't use GAN loss, comparing against their GAN-enhanced numbers wouldn't be fair. But when we compare both methods without GAN loss, we're competitive: our FID of 4.80 vs their 5.19 at NFE=1 on CIFAR-10. \n> \n\n>  **Q1.3**: Such omission is particularly confused me given that the anonymized code provided by the authors already includes a discriminator module.  \n\n> \n\n**[We did not use this adversarial module in practice since there are some training stability issues.]** \n\n> \n\n> **Q2.1**: The paper lacks comparisons with several important baselines, such as the Diff-Instruct series, DMDs, and sCM. \n\n> \n\n**[We include all baselines of trajectory distillation methods and exclude some baselines for unfair comparsion.]** \n\nAlthough a direct comparison is not entirely fair due to the use of different pretrained models and different network architecture, we still report the best results from the original papers: Diff-Instruct[G] (Tables 1 and 2), DMD[H] (Table 2), and sCM[I] (Table 1). \n| Models         | CIFAR-10 | ImageNet-64 |\n|----------------|----------|--------------|\n| Diff-Instruct[G] | 4.53     | 5.57         |\n| DMD[H]         | 2.66     | 2.62         |\n| sCM[I]         | 3.66     | 2.44         |\n| SCoT (Ours)    | 2.40     | 4.80         |\n\n- Diff-Instruct series and DMD follow different strategies with the proposed SCoT. The generator developed by these methods is no longer a generative model. Also, these student models cannot do multi-step generation. \n\n- Further, DMD introduces an LPIPS perceptual loss computed using an external pre-trained VGG-like network. This provides an additional perceptual prior beyond the distilled diffusion model itself, which significantly improves visual fidelity but also introduces extra supervision unavailable in our setting. Without LPIPS, as shown in DMD's own ablation studies, performance degrades notably due to mode collapse. \n\n**References:** \n\n- [G] Diff-instruct: A universal approach for transferring knowledge from pre-trained diffusion models. (NeurIPS 2023) \n\n- [H] One-step diffusion with distribution matching distillation. (CVPR 2024) \n\n- [I] Stabilizing and scaling continuous-time consistency models. (ICLR 2025) \n\n>\n\n> **Q2.2** Notably, many of these related works report results that surpass those presented by the authors. \n\n>\n\nWhen we directly compare our method with the eight additional baselines suggested by **Reviewer ovkA** and **Reviewer  qykR**, our model outperforms some of them(more details see **Reviewer ovkA W1** and **Reviewer qykR Q2.1**. However, as noted in our response, these comparisons are not entirely fair.\n\nFirst, although we report **the best results for each method from their original papers on the corresponding datasets**, the pretrained models they use differ. For example, If DMD [H] did not use LPIPS loss in its main setup, the FID score drops from  2.66 to 5.58 (Table 2 in [H]). Diff-Instruct [G] also shows a wide range of performance (6.62 to 2.27), depending on the GAN backbone used. These differences make direct numerical comparison problematic.\n\nSecond, our method is based on trajectory distillation, while several of the baselines adopt **fundamentally different mechanisms**. For instance, Diff-Instruct [G] uses a GAN-based model that does not support multi-step generation, and thus should not be directly compared under our setting."}]

Author rebuttal:

Meta-review: Tl;dr: Based on the reviews, rebuttal and ensuing discussion I recommend accept.

### Paper Summary

The paper introduces a new method for distilling pre-trained diffusion models. The primary claim is that SCoT can generate high-quality samples in very few steps by producing trajectories that are both straight and consistent. This is achieved by combining two objectives: a velocity loss that encourages straight trajectories and a consistency loss that ensures that points along the trajectory are mapped to the same final output. Experimental results on CIFAR-10 and ImageNet are presented, with competitive performance compared to SOTA methods, especially in low NFE settings.

### Key strengths and weaknesses

Strengths: 1) Novel and simple method, 2) Good empirical results, particularly on ImageNet, with meaningful ablation studies, 3) Clear presentation, with intuitive figures relating to other approaches.

Weaknesses: 1) Some lingering concerns about novelty among reviewers as a simple combination of existing losses. Though, the authors have clarified that the velocity loss is not identical to the one in rectified flow, 2) Insufficient ablation in the initial draft, those these were addressed during the discussion, 3) Lack of high resolution experiments.

### Decision justification

The proposed method is novel and achieves strong empirical results. The authors have done a good job of addressing the concerns raised by the reviewers during the rebuttal period, providing additional experiments and clarifications. While there are some limitations, e.g. the lack of high resolution experiments, the paper makes a solid contribution to the field and is likely to be of interest to the NeurIPS community.

The discussion period was very active with significant back and forth. The main points raised by the reviewers were: 1) Novelty (R:nd3F), 2) Missing baselines and ablations (R:ovkA, R:Yq6Q, R:qykR), 3) Lack of high resolution experiments (R:nd3F).

The authors provided additional results in the rebuttal, including comparisons with more baselines and ablations on the impact of individual loss components and performance at different NFEs. This has significantly improved the paper and addressed many of the reviewer concerns. Reviewers ovkA and Yq6Q raised their scores after the rebuttal.
Decision: Accept (poster)