Improved Training Technique for Shortcut Models
Authors: Anh Nguyen
Viet Van Nguyen
Duc Vu
Trung Tuan Dao
Chi Tran
Toan Tran
Anh Tuan Tran
Venue: NeurIPS 2025 poster ()
Keywords: generative models
Primary area: applications
TL;DR: This paper tackle the five core issues that held shortcut models back.

Abstract: Shortcut models represent a promising, non-adversarial paradigm for generative modeling, uniquely supporting one-step, few-step, and multi-step sampling from a single trained network. However, their widespread adoption has been stymied by critical performance bottlenecks. This paper tackles the five core issues that held shortcut models back: (1) the hidden flaw of compounding guidance, which we are the first to formalize, causing severe image artifacts; (2) inflexible fixed guidance that restricts inference-time control; (3) a pervasive frequency bias driven by a reliance on low-level distances in the direct domain, which biases reconstructions toward low frequencies; (4) divergent self-consistency arising from a conflict with EMA training; and (5) curvy flow trajectories that impede convergence. To address these challenges, we introduce iSM, a unified training framework that systematically resolves each limitation. Our framework is built on four key improvements: Intrinsic Guidance provides explicit, dynamic control over guidance strength, resolving both compounding guidance and inflexibility. A Multi-Level Wavelet Loss mitigates frequency bias to restore high-frequency details. Scaling Optimal Transport (sOT) reduces training variance and learns straighter, more stable generative paths. Finally, a Twin EMA strategy reconciles training stability with self-consistency. Extensive experiments on ImageNet 256x256 demonstrate that our approach yields substantial FID improvements over baseline shortcut models across one-step, few-step, and multi-step generation, making shortcut models a viable and competitive class of generative models.

Improved Training Technique for Shortcut Models
Anh Nguyen"
Viet Nguyen"!
Duc Vu
Trung Dao
Chi Tran
Toan Tran
Anh Tran
Qualcomm Al Research
Abstract
Shortcut models represent promising, non-adversarial paradigm for generative modeling, uniquely supporting one-step, few-step, and multi-step sampling from single trained network. However, their widespread adoption has been stymied by critical performance bottlenecks. This paper tackles the five core issues that held shortcut models back: (1) the hidden flaw of compounding guidance, which we are the first to formalize, causing severe image artifacts; (2) inflexible fixed guidance that restricts inference-time control; (3) a pervasive frequency bias driven by reliance on low-level distances in the direct domain, which biases reconstructions toward low frequencies; (4) divergent self-consistency arising from a conflict with EMA training; and (5) curvy flow trajectories that impede convergence. To address these challenges, we introduce iSM, a unified training framework that systematically resolves each limitation. Our framework is built on four key improvements: Intrinsic Guidance provides explicit, dynamic control guidance strength, resolving both compounding guidance and inflexibility. A Mulii-Level Wavelet Loss mitigates frequency bias to restore high-frequency details. Scaling Optimal Transport (sOT) reduces training variance and learns straighter, more stable generative paths. Finally, a Twin EMA strategy reconciles training stability with self-consisteng demonstrate that our approach yields substantial FID improvements over baseline shortcut models across one-step. few-step, and multi-step generation, making shortcut models a viable and competitive class of generative models.
Extensive experiments ImageNet 256 x 256
Introduction
Recent generative diffusion models based on flow matching have achieved remarkable success in While demonstrably success- ful, a significant bottleneck persists: generating samples requires integrating the Iearned vector field over many discrete timesteps, resulting in high computational costs that limit deployment in resource-constrained environments and latency-sensitive applications. Accelerating sampling without sacrificing quality has thus become critical research area.
synthesizing high-fidelity data across various domains [14 32 58
Various acceleration techniques have emerged, including timestep distillation
advanced numerical solvers [40], lightweight architectures [7] [1] 64), and single-stage training procedures [54
Shortcut models (SM) [88 offer particularly elegant approach, using generator conditioned on both noise level / and desired step size d with additional self-consistency loss. This enables the prediction of multiple timesteps ahead single forward pass, supporting variable sampling budgets at inference using the same network. Despite this promising method, widespread adoption has been hindered by critical performance bottlenecks. This paper tackles the
$ "Equal contribution
I Work done while at Qualcomm.
Qualcomm AI Research is initiative of Qualcomm Technologies, Inc.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

five core issues that held shortcut models back: the hidden flaw of compounding guidance, inflexible fixed guidance, frequency bias, divergent self-consistency, and curvy flow trajectories.
The first two issues stem from a flawed integration of Classifier-Fres Guidance (CFG). Not only are users locked into fixed guidance scale at training time-sacrifiiing control over the diversity-fidelity trade-off- but this design also conceals a deeper flaw. We are the first to formalize that this fixed guidance compounds exponentially across the implicit sub-steps of large generation step, causing severe image artifacts. The other three issues further degrade sample quality: a reliance on pixel-wise losses creates a frequency bias toward blurry images; random noise-data pairings create unstable, high-curvature generative paths; and temporal lag in the EMA targer prevents the model from leamnigg true self-consistency for large jumps.
To address these challenges, we introduce the Improved Shortcut Model (iSM), a unified training framework that systematically resolves each limitation through four key components.
1. Intrinsic Guidance: We resolve both guidance-related flaws by making the guidance scale explicit input to the model. This provides dynamic, inference-time control, works out-of- the-box for one-step generation, and reduces inference time by ~50 compared to standard CFG, all while correcting the compounding effect.
2. Multi-Level Wavelet Loss: We replace standard pixel-wise objectives with a frequency- aware loss that forces the model to reconstruct high-frequency details, mitigating frequency bias.
3. Scaling Optimal Transport (sOT): We straighten generative trajectories by periodically pooling samples from several mini-batches to compute a large-scale transport plan. This decouples the OT batch size from the training batch size, yielding more stable paths without heavy computational cost.
4. Twin EMA Strategy:
We eliminate target lag by maintaining two EMA networks- fast- decay one to generate fresh, up-to-date consistency targets and slow-decay one to ensure stable, high-quality inference.
With these improvements, iSM achieves FID scores of 5.27 and 2.05 on ImageNet 256 x 256 with just one and four sampling steps, respectively. Our work closes the performance gap with leading generative models and establishes improved shorteut models as a flexible, efficient, and highly competitive modeling paradigm.
2
Preliminaries
Flow Matching (FM) 37 38 offers elegant framework for generative modeling of data distri- butions Pdma (z). At its core, FM defines H; = (1 - t)æ0 + tæx as a linear interpolation between a noise sample 25g drawn from a standard normal distribution (O, I), denoted A, and data sample 2, drawn from the data distribution D. Here, 1 E [0, 1J represents the timestep, parameterizing the interpolation from noise 0) data (t 1). It then trains veloeity model (z t, c) match the ground-truti velocity field v = σ1 Zg by minimizing the following velocity loss:
Livelocity (0) E Eng P NN (m), (1),,, - =]
(-p(t)
Here, a. is the sample interpolated at time I, and represents any conditioning information, such as text or class labels, associated with the data sample 21. Sampling from such model typically involves discretizing the learned ODE using numerical methods like Fuler integration, often requiring dozens or hundreds of steps. Naively taking large steps with üy leads to significant discretization errors, as the predicted velocity points towards average of potential target data points, causing mode collapse.
Shortcut Models (SM). To address this limitation and enable efficient few-step and one-step genera- tion, [18 proposes training a shortcut model that conditions the neural network sp x., t, c, d) not only the current timestep & and the condition c, but also desired step size d. This allows the model to predict the normalized displacement needed to directly reach the next point 2'4d at time I ++ d, thereby bypassing intermediate steps of the probability flow ODE.
The training approach includes flow-matching for infinitesimal steps and self-consistency for larger steps. At infinitesimal step sizes (d < 0), shortcut models use the flow-matche now-mitching objective, regressing

---

the model sa( c, = 0) to predict the empirical velocity Evelocity, similar to traditional flow- matching models. For larger step sizes (d > 0), shorteut models leverage a self-consistency property where one large shortcut step is equivalent to two consecutive smaller shorteut steps of half the size. This property allows the model learn efficient large-step transitions by recursively breaking them into smaller steps. To implement this self-consisteng objective for training, they select base number steps (N = 128 in practice), which defines the smallest time unit for the ODE 8 distinct shortcut lengths available during training, specifically E {1/128, 1/64, ...,................ 1/2, 1], over which the self-consistency loss applied. When at the smallest value (e.g.. 1/128), they instead query the model at d = 0. CFG [24], widely used to improve conditional sample quality, is integrated within SM to form guided targets gø" based on the model output sg:
approximation. This results in logg (N)
9ø (zt t, c, d) so( t, c, (so( t, c, d) s0 2,,,t,,,d,, d)),
where 2 denotes the null condition. Using the guidance-scale scaled output gØ the self-consistency target Sconsistency is constructed by simulating two consecutive steps of size d:
8 ronsistancy
g (a+, 1, d)/2 + -0" (a'(da: / + d, d)/2,
where 2'44d = 0t + g (It, t, c, d)d. These components then form a unified loss function:
S Eego ((,d)~p(t,d)
(2 (.)
[150(2t, e, = 0) Sylocityy ( L, 2d)
Aconsistency
Flow Now-Matching
Self-Consistency
This approach allows shortcut models to achieve effective multi-step. few-step, and single-step generation using single trained network.
3 Improved Guidance Sampling for Shorteut Models
Shortcut models [18] represent a significant step toward efficient one-step, few-step, and many-step generation. However, their integration of CFG introduces unique challenges compared to standard diffusion methods. This section revisits the shortcut model formulation to identify and address two critical limitations: (1) inflexibility due to fixed guidance scales, (2) an overlooked flaw of accumulation when applying CFG.
3.1 Problem Statement
Inflexibility of Fixed Guidance. The original shortcut model framework requires using a fixed guidance scale during training. This approach is restrictive for two main reasons. First, selecting an optimal w is difficult hyperparameter tuning problem that depends on the model's final behavior and the intended number of inference steps. Second, and more critically, removes a key control for balancing the fidelity-diversity trade-off at inference time, substantially reducing the model's flexibility in production settings.
Flaw of Accumulated Guidance. A critical, previously overlooked issue in shortcut models the accumulation of guidance effects. We are the first to identify and formalize this critical flaw, which stems from the recursive application of a fixed guidance scale. When a shortcut model generates a sample in single large step. implicitly combines the effects of many smaller, guided steps. This compounding characteristic leads to an unintended amplification of the guidance signal.
To formalize this, consider shorteut model se and its corresponding classifier-free guided output 9h with guidance scale w. To analyze the generation process, we define a sequence of intermediate points (x'_ N This sequence is constructed by recursively applying consecutive shortcut R
steps. each with the smallest size 1/N. The process begins from the initial noise wo at conditioned on c. The starting point of the sequence is thus defined as z Do. and subsequent intermediate points are generated as follow:
x 11 N
x'
+ 38
x' N (dddd d, for = 0, O,...,
(5)
3

---

Shortcut Nodele
Improved Storteut Nodels
8-126
Figure 1: Visualizing the acciouulated guidance prohlem in standard shortcut models (left) versus our integrated- guidance approach (right). Top shows multi-step generation; bottom shows one-step generation. In the original shortcut model. the fixed guidance strength learned at training time effectively compounds across many implicit sub-steps, leading oversaturated colors and blurriness. By conditioning network on the guidance scale and training it to apply guidance explicitly at each step, our method produces consistent, artifact-free images even in the extreme one-step and few-step regimes.
Proposition 1. The model's prediction for single large shortcut step of size Nd approximately equals the average of the guided displacements corresponding to the N smallest steps, but with an exponentially compounded guidance scale:
80 s0 O, Nd)=
Σ
98
m loss
x 4' N. c,
i
(6)
N
1-0
Proof. See Appendix A.
The original shortcut model [18] defines a fixed CFG scale w = 1.5, exclusively applied when constructing self-consistency targets Eq. (3). Consequently, the one-step generation 86 (æo, 0, 1) implicitly aggregates the cumulative effect of N intermediate steps. As formalized in Proposition I the guidance scale each of these implicit intermediate steps not but much higher value w/ (N)) For N 128 base steps and
1.5, this calculation yields an extremely high intermediate guidance scale for single large shortcut step cases (w' (1266) & 17). Such high implicit guidance causes artifacts like saersaatu,t,in, particularly in one-step and few-step generations (see Fig. I].
3.2 Intrinsic Guidance Training for Shortcut Models
To address these limitations of shortcut models [8], we introduce an Intrinsic Guidance training framework. In contrast to CFG implementations that operate between model outputs, commonly practiced multi-step diffusion models, we integrate the guidance mechanism into the network's internal space. The core idea make the guidance scale an explicit inpul the model and to train the model to produce guided oulputs directly across a range of scales. This framework enables flexible CFG for few-step and many-step generation and, importantly, provides a principled way to apply CFG for one-step generation while resolving the accumulated guidance issue.
neural network 8g. Unlike previous shortcut models [18], Sp receives the CFG scale w > O as an additional input, alongside the current sample 2,, time condition c, and the target step size d. The network is thus trained to directly output the CFG-modula modulated velocity 8p (zx, t, c, d, w). allowing guided sample generation with single network evaluation per step. For implementation details, please refer to Appendix C.
Model Parameterization. We define the velocity field generator
Flow Matching Objective. This objective 37 38 guides the model to predict the conditional and unconditional velocity for infinitesimal steps (d 0) without guidance (w 0), establishing the

---

base vector fields upon which guidance will be built:
Liclocity (0) == Eegn (21,,))- 80 ae, i, d = 0, 0) &velacity
(-p(t) where 3t t) tæ and Svelocity 2 30-
When sampling the condition for training, we randomly include the null condition 2 via stochastic dropout, following the standard scheme
(41.
Intrinsic Guidance Objective. This objective guides the model to directly produce the CFG-scaled output for infinitesimal steps (d 0) when being conditioned non-zero guidance strengths 0).
We begin by recalling the standard formulation of CFG with w > 0:
80 80( e, d = -0,,) w) x 80( G, O, w = 0) + w (80( C, 0, - 0) - 80 Ø,
0, w 0)),
Aguidancc
(8)
where Sguidance is the estimated guidance direction using the model's base prediction (w 0).
We substitute the unguided velocity sol-, 0, w 0) in Eq. 7 based on the approximation of sa(+,, e, d = 0, w 0) in Eq. (8):
se s0 c, O, w = 0) so(. c, d 0, w) w Sguidances
|s0(,, 0,
Svalocity (100 c, 0, w) w 8guidance) - vvelocity 1|
cs 0, w) (Svelocity + w $guidance ) ||?
Integrating this derived target into Eq. 77). we define a new training objective called the intrinsic guidance loss Lguidance- Crucially, we apply the stop-gradient operator sg(-) to the guidance direction Sguidance within this loss. This isolates the optimization focus solely on how the network learns to scale its output in response to w > 0, preventing interference with the optimization of the foundational O predictions in Cvelocity and improving training stability.
Eguidance (0) := Boo (Z.,)) [150 1, c, 0, w) (8velocity + w sg( (gguddance) )) (t,w))p(t.wn)
where guidance scale >> is drawn from distribution p(w). This loss trains the network to learn how CFG works for infinitesimal steps (d = 0).
Guided Seff-Consistency Objective. This objective generalizes the self-consistency principle from
18 to operate with arbitrary step sizes (d > 0) and any guidance scale (w > 0). The objective
maintains the foundational properties of shortcut models, where a single, large guided shoricut step
yields an output consistent with the composition of two smaller, consecutive guided steps.
Cconsistency (0) Em (m) ()) [| t, c, 2d, w) Sconsistency ||]. (tw.:d)-((t.w.d)
(10)
where Sconsistency 80 (Ke, 1 c, w)/2 s0- (x' d: l, d. and 2/44 = 2t + so t, c, d, w)d,
where 0- is the EMA target network. The stop-gradient operator sg(.) is applied to the entire consistency target to stabilize training, following standard practice for self-consisenncy objectives.
Final Training Objective. The final objective is a weighted sum of the individual components as follows:
Liota (0) a< (0) + @Leguidance (0) + "Cconsistency (0),
(11)
where a; 0, > 0 are hyperparameters. For simplicity, we set a = 3 = = 1 in our experiments.
3.3 Interval Guidance in Training
Findings presented in 31 suggest that during the inference of diffusion models, strong classifier-fres guidance can be detrimental during the early stages of the reverse process, corresponding to high noise levels (time approaching 0 under the convention where t 0 is pure noise).

---

At high noise levels, the signal-to-nois noise ratio is low. Hence, the unconditional prediction so(æ,, 2, 0, w = 0) points towards the average image in the dataset D, while the condi- tional prediction so s0( L, d 0, w 0) points vaguely towards the average image for that condition. Consequently, the guidance vector &guidance points roughly from the global data mean towards the conditional data mean. Applying strong extrapolation (w > 0) along this coarse direction to a high-entropy state 2e can cause the sample diversity to collapse prematurely towards a few dominant modes.
To mitigate this issue, we incorporate this insight into the training objective Lguidance in Eq. 9 by applying guidance only within a defined time interval [intervall 1). In particular, we modify the guidance scale used within the Eq. (9) to be time-dependent:
w(t) = w, if E [inervvll: 1)
(12)
0, otherwise,
where w is the sampled guidance scale. This strategy trains the model to apply zero guidance at high noise levels, preserving diversity in the critical early phase of generation. We find that tinterall = 0.3
works well empirically, and this threshold is represented by the red dashed line in Figs, (2b and 2c
Notably, in the high-noise regime where i < 0.3 (i.e., before guidance is applied), both flow flow-matching and ceff-consistency losses exhibit lower loss pattern.
4 Multi-Level Wavelet Function Against Frequency Bias
Deep neural networks often prioritize learning low-frequency information over high-frequency details, phenomenon known as frequency bias [47 50
3, 62 633. This bias is detrimental to generative
models 10 17 29 (1], particularly shortcut models I8 optimized for few-step generation with low-level direct domain losses, such as the &g loss. These models typically capture global structure but lack fine-gr aree-graded, high-fre high-frequency details, resulting in blurred textures. The low-level & loss contributes to this, is often dominated by errors in now-frequency components (68).
To mitigate this frequency bias, we introduce optimizing the reconstruction objective in the waveler do- main. Specifically, discrete wavelet transform (DWT) is employed to decompose both the prediction and the target (Eqs. ) and (IO)) into their respective wavelet coefficients, yielding multi-band representations, which introduce a frrqqeeccy-wwar
error signal. By isolating reconstruction errors at different frequency levels, the optimization process is better guided to preserve gighffrequenc details that are often neglected in standard losses. Consequently, the model learns to generate outputs with significantly improved perceptual quality, capturing both the global structure and fine-grained details.
To further refine supervision across the frequency spectrum, we extend this to a multi-level DWT decomposition. This process involves recursively decomposing the wavelet sub-bands from the previous level. Our experiments consistently show that using deeper multi-level decomposition, with five levels (L = 5) in our experiments, substantially improves the synthesis of high-fre content compared
single-level DWT loss or standard pixel-wise objectives. Pseudocode is provided in Appendix B.
5 Scaling Optimal Transport for Better Trajectories
Shortcut models, similar to standard flow matching, often encounter considerable loss variance during training. This instability, as highlighted in prior works [46, 33), primarily stems from the conventional random pairing of initial noise samples vo (0. with target images (æ) D). Such random coupling frequently results in intersecting forward inputs might be mapped to vastly different data targets. Consequently, the learned generative ODE exhibits high-curvature reverse paths, posing a significant challenge for shortcut models. Optimal Transport (OT) within
forward trajectories (æt), where similar noise
mitigate this by re-assigning noise-image pairs to minimize transport cost, typically L2 distance, yielding smoother trajectories 46 [3]
Although mini-batch OT helps, we recognize that the efficacy of such mini-batch OT is fundamentally linked to the number of samples considered. Theoretical and empirical findings [3] show that OT better approximates the true transport plan the number of samples increases. However, scaling

---

Jn 67
6-1
No 07
d=| n-
0-4
2-
Timestop
(a) Forward and reverse trajectories.
(b) Self-consistency Loss.
(c) Velocity & Guidance Losses.
Figure 2: Efficacy of Scaling Optimal Transport (sOT) Matching in improving shortcut model training, demonstrated by varying the OT scaling factor K. (a) On bimodal target, forward trajectories (top row) without OT exhibit frequent intersections (red), compelling the reverse generative process (bottom row) to follow high-curvature paths that initially average target modes (blue, green). Our sOT approach, by increasing the effective OT scaling K, progressively disentangles these forward couplings, yielding substantially struighter reverse trajectories. (b, <) The benefits of this trajectory straightening are reflected in the training losses: both self-consistency (b) and velocity & guidance (c) losses are consistently lower and more stable with larger K, especially after the interval guidance point finterval 0.3 (left-sidc of the red dashed line). Note for (a): To avoid overly dense trajectory, only intersecting forward paths and sampling trajectories with high curvature are highlighted. Zoom for detail.
the mini-batch size M to achieve this is often limited by GPU memory constraints during training. To tackle this scalability challenge, we introduce Scaling Optimal Transport (sOT), a strategy that decouples the effective batch size for OT computation from the batch size used for model training. Our method allows us to use large number of samples for OT without increasing the memory needed for model training. Specifically, for every K training batches of size M, we aggregate all noise and image samples into a single set of size K x M and compute one OT plan over it. The resulting K M matched pairs are then split back into K standard training steps. This strategy allows us to benefit from an effective OT batch size of K x M while maintaining computationally tractable batch size M for model training. The computational overhead from sOT is modest, adding approximately 4% to the total training time in our experiments.
of size M and used for the next K
Fig. 2a demonstrates sOT's effectiveness on 2D) bimodal dataset. Random pairing (leftmost column) creates overlapping forward trajectories, forcing the reverse process to follow high-curvature paths that initially move toward an average direction before bending toward target modes. As the effective OT batch size K increases (columns 2-4), sOT produces progressively more disentangled forward couplings with fewer intersections, resulting in straighter generative trajectories.
Figs. 2b and 2e further illustrate the benefits of sOT by showing a consistent decrease in self- consistency and now-matching losses, the sOT scaling factor K increases, with these loss statisties computed over 1.2M ImageNet samples. Regarding self-consistency loss (big.2b), sOT straightens the learned generative paths (as seen in Fig. 2ap. On these smoother trajectories, single large shortcut step more accurately approximates the outcome of two composed smaller steps, thus reducing the discrepancy and lowering the loss. This indicates that the model learns more reliable long-range predictions. For flow-matching and guidance loss (Fig. [c). sOT provides more structured noise-data pairings. This results in a less conflicting target velocity field for the model learn, as similar noise inputs are mapped to more consistently related data targets. This simplified regression task allows the model learn more coherent target, thereby reducing the error.
6 Twin EMA Improves Self- Calf-Consistency Property
Self-consistency objectives seek to align large-step generations with sequences of smaller steps. Specifically, they train an online network 0 to produce outputs from a 2d-sized step that match the consistency target Scomsistency. which is obtained by applying two consecutive d-sized steps. Instead of using the online network 9 to compute Sconsistencys the original paper suggests using a target network 0=, typically implemented an EMA of O. When utilized for generating self-consistency targets, EMA parameters provide crucial stability that dampens the oscillatory behaviors that would otherwise propagate and amplify across different timesteps.
Despite its essential stabilizing role, conventional EMA implementations introduce a tension in self-consistency training. When the target network 0- is maintained with standard slow decay rate, it inevitably represents a historical state of the online network rather than its target distribution. This temporal lag means that self coniisensi targets Scousistency derived from outdated networks.
7

---

Consequently, the online network 0 faces a con- flicting objective: it must simultaneously optimize for its current trajectory during many-step genera- tion (i.e., flow matching loss) while aligning with targets generated from its historical states during few-step (i.e, self-consistency loss). As training progresses, this misalignment forces the model to learn and balance multiple, potentially contradic- tory generative mappings rather than converging toward a unified process. The slower the EMA de- cay rate, the more pronounced this distributional divergence becomes.
Table 1: Quantitative results class-conditional ImageNet 256 across different model types.
Model
FID-50K (J)
NPE ())
#Params
GAN
BigGAN-deep GigaGAN 26
112M
3.45
1
569M
StyleGAN-XI
Masked & AR
VQGAN 16
26.52
1024
227M
Mus&GIT
6
6.18
227M
VAR
56
2.57
10
600M
VAR-430
56
1.92
10
2B
MAR 35
1.98
100
400M
To resolve this tension while preserving the es- sential stabilizing benefits of EMA, we introduce Twin EMA approach that maintains two dis- tinct parameter sets: (1) Inference Parameters (0 m: Updated with conventional slow decay rate and used exclusively during inference, ensur- ing high-quality sample generation benefits from long-term parameter averaging. (2) Target Pa- rameters (argeet): Updated with significantly faster decay rate to maintain close proximity to the online network's 's current state while still providing stabilization. This decoupling addresses the fun- damental conflict between stability and recency re- quirements. By generating self-consistency targets from a near-contemporary version of itself, the on- line network more effectively enforces consistency across different timestep discretizations. Mean- while, the separate maintenance of slow-decay parameters for inference preserves the stability crucial for high-quality sample generation.
Diffusion Flow Models
ADM
P2
10.94
250
554M
SimDiff
M6
2.77
512
2B
LDM-4-G
48
3,60
250
400M
U-DiT-L.
57
3.37
250
916M
VTT-T
2.29
50
501M
DIT-XL/2
2.27
250
675M
SIT-XL/2
41
2.15
250
675M
REPA-XL,2
1.42
FloWDCN 60
2,00
250
618M
One-to-Many Models
34.24
20.3
675M
SM 18
10.50
675M
7,80
4
675M
3,80
128
675M
IMM
3.99
2
675M
2.51
675M
1.99
675M
ISM (ours)
5.27
675M
2,44
675M
2.05
675M
1.88
128
675M
7 Experiment
We combine all the improvements detailed from Sections 3 to to train our improved shortcut models. We extensively benchmark our models ImageNet 256 256 against var- ious models across multiple training paradigms. We evalu-
Table 2: Broader evaluation with FD-
DINOv2 and IS ImageNet 256 x 256.
Model
NFE
FD-DINOv2]
1St
ate sample quality using FID-50K (20). For all experiments, our models use the XL./ variant of the SiT architecture [41] We report evaluations against other models at 800K iterations, while component-wiss analyses are conducted at 250K iterations. We summarize our main quantitative benchmarks in Table provide a broader evaluation using additional metrics in Table 2 present detailed component- wise analysis in Table 3, and demonstrate the scalability and generality of our framework in Table
SM LB
500.92
IMM (
247.78
128.87
(ours)
232.31
223.52
SM 18
2
329.53
125.66
IMM
152.08
M (ours)
107.63
302.29
SM B
265.90
136.79
IMM (69
110.88
204.95
(ours)
7.1 Quantitative result and comparison
Our model, iSM, achieves strong FID scores across various NFE settings, demonstrating high sample quality and inference efficiency. With 8 inference steps, iSM surpasses the 10-step VAR (56) of similar model size and performs comparably to VAR's much larger 2B variant. A key advantage of iSM is its inherent support for flexible NFE counts-from single-step to many-step generation feature absent in models like VAR, with FID scores significantly improving as NFE increases from 1 8. Furthermore, iSM performs favorably against other recent variable-step models, including the original shortcut model [18] and IMM [69]. For instance, in single-step generation, iSM attains
8

---

Table 3: Ablation study ImageNet 256 x 256. We investigate the impact of key hyperparameters for each of proposed components. The best-performing setting from each block is carried forward to the next. All models are trained for 250K iterations.
Method
Manxx
finterval
L
K
FIDx-1
FIDy-4
2.0
0
D
0
0.9999
10.10
3,21
Intrinsic Guidance (Scc.
3,5
0
0
0.9999
9.62
3.17
5.0
0
0
0
0.9999
10.38
3,34
3.5
0.0
0
0
0,9999
9.62
3.17
Interval Guidance (Sec.
3,5
0.1
D
0.9999 0
0.9999
8.58
3.14
3.5
0.3
0
8.49
2.81
3,5
0.5
0
0,9999
19.22
2,84
3.5
0.3
0
0
0,9999
8.49
2.81
Multi level Wavelet (Sec.
3,5
0.3
1
0.9999
8.21
2.79
3.5
0.3
3
0.9999
8.17
2.75
3.5
0.3
5
0.9999
8.12
2.64
3.5
0.3
5
0
0,9999
8.12
2.64
Scaling OT (Sec.)
3,5
0.3
5
1
0.9999
8.07
2.51
3.5
0.3
5
8
0.9999
8.03
2.28
3.5
0.3
5
32
0.9999
7.97
2,23
3.5
0.3
5
32
0.9999
7.97
2.23
Twin EMA (Sec.
3.5
0.3
5
32
0.999
7.43
2,21
3.5
0.3
32
0.95
6.56
2.16
FID of 5,27, outperforming both. This strong performance extends to few-step scenarios, with FIDs of 2.05 (4 steps) and 1.93 (8 steps), highlighting iSM's robustness across different inference budgets.
To provide a more comprehensive validation, we extend our evaluation to include FD-DINOv2 and Inception Score (IS). As presented in Table 2 our iSM framework consistently outperforms the baselines across these metries. The strong FD-DINOv2 scores confirm that our model's high fidelity is robust and not specific to the Inception feature space; for example, at 4 NFE, iSM achieves score of 83.70, greater than 3 times improvement over the baseline SM's 265.90. The Inception Scores show particularly large margin of improvement; for example, at inference steps, model's IS of 302.29 more than doubles the baseline score of 125.66. The consistent gains across these distinct evaluation metrics provide strong evidence that iSM yields significant and robust improvements to shortcut model generation.
7.2 womponent-wise Analysis
We explore whether our proposed techniques enhance shorteut model training. As shown in Table 3 each component contributes to improved generation quality, with their combination achieving a significantly better FID score compared to the vanilla model. Below, we present detailed analysis of the individual impact of each component.
Intrinsic Guidance. We examine the effect of varying the range of the conditioning CFG scale on model performance. During training, we sample discrete values of w from the interval [0, Wmaax] using step size of 0.25. Through empirical evaluation, the best results. When Wmax is set too high (e.g., 5.0), the model is exposed to a broader range of conditioning strengths. This can introduce unnecessary complexity, as the model must learn to generalize over a wider set of conditioning values, potentially degrading overall performance. On the other hand, if Wmax is too low (e.g.. 2.0), the model only learns from outputs corresponding low CFG scales, which may lack high quality.
find that setting Wmax 3.5 leads to
Interval Guidance in Training, We examined the effect of varying finterval. the threshold below which CFG is disabled. We observed that increasing fnterval from very low values up to a moderate range (c.g., 0.1 to 0.3) improved both metries, consistent with the benefit of avoiding strong, coarse guidance high-noise regimes. However, further increasing finterval to higher values (e.g., 0.5) results in significant increase in FID. Training the model without guidance for substantial portion of the early reverse process may impair its ability to effectively leverage guidance when > finterval-
Multi-level Wavelet Function. We also investigate the impact of varying the number of decompo- sition levels the wavelet transform. Given that the latent representation of size 32 x 32, the maximum feasible number of levels is (32) 5. Our experiments show that utilizing the full 5

---

levels yields better performance compared to using fewer levels (e.g., 1 or 3) or not using the wavelet transform entirely.
Scaling Optimal Transport. We also examine the impact of OT matching on the performance of our previous improvements. Our results show that using traditional OT matching results in lower FID scores than not using OT. We then assess the effectiveness of our proposed enhanced OT by applying it at two different scales: K 8 and K 32. Our results reveal a clear correlation between increasing OT scale and improved FID scores. Although scaling beyond K 32 may further enhance 32 trade-off between computational overhead and effectiveness.
performance, we select
Twin EMA, Finally, our proposed Twin EMA was evaluated using decay rates of 0.999 and 0.95 for the target network Ourget. A decay rate of 0.95 provided an effective balance between training stability and mitigating distributional divergence from self-consistency targets. This was evidenced by substantial reduction in FID, achieving 6.56 and 2.16 for one-step and four-step generation, respectively.
7.3 Generalizing Higher Resolutions and Architectures
To demonstrate the architectural generality and resolu- tion scalability of our framework, applied our iSM training to FCN [59]. a fully convolutional gener-
Table 4: Scalability and generality of iSM on ImageNet 512 x 512 with the FlowDCN archi- tecture, reported after 300K training iterations.
ative model with group-wise deformable convolution blocks, on ImageNet 512 x 512. As shown in Table 4 at 300K training iterations, our method yields sub- stantial FID improvement over the baseline, confirming
Model
NFE
FIDI
Precisiont
Recall
SM
43.81
0.56
0.11
ISM
37.05
0.55
its effectiveness across different model families and at
SM
12.16
0.86
0.19
iSM (ours)
9.94
0.78
0.62
higher scales. For instance, in 4-step generation, our approach reduces the FID from 12.16 16 to 9.94. The improvement also pronounced in the one-step generation, where iSM improves sample diversity by boosting Recall over 5 times from 0.11 1 to 0.55.
8 Conclusion
This work confronts the foundational obstacles that have previously limited the performance of shortcut models. We systematically address five core challenges: the damaging effects of com- pounding guidance, inflexible fixed guidance, a pervasive low-frequency bias, conflicts between self-consistency and EMA updates, and instability from high-curvature flow trajectories. To resolve these issues, we present iSM, a unified training framework composed of four key components. By incorporating Intrinsic Guidance for dynamic control while resolving the flaw of compounding guidance, a Multi-Level Wavelet Loss for high-frequency fidelity, Scaling Optimal Transport (sOT) for smoother trajectories, and Twin EMA strategy for stable consistency, iSM elevates shortcut models into a competitive paradigm for generative modeling.
Our extensive experiments ImageNet 256 x 256 demonstrate that iSM yields substantial FID improvements over baseline shortcut models across one-step, few-step, and multi-step generation. These results close the performance gap with leading GANs and other diffusion-basee models, establishing shortcut models as a viable and competitive alternative that uniquely offers variable sampling budgets from single network.
10

---

References
[1] S. AI. Fine-tuned vae for stable diffusion (sd- fsd-vae-ft-mse). https:// co/ Saabllityai/sd-vee wat-a//--daeee----eeeee ft-mee, 2023, Accessed: 2025-04-28.
[2] F. Bao, S. Nie, K. Xue, Y. Cao, Li, H. Su, and J. Zhu. All are worth words: A vit backbone for diffusion models. Proceedings of the IEEF/CVE conference on computer vision and paltern recognition, pages
2023.
[3] R. Basri, M. Galun, A. Geifman, D. Jacobs. Y. Kasten, and S. Kritchman. Frequency bias neural networks for input of non-uniform density. In International conference on machine learning, pages 685-694, PMLR, 2020.
[4] A. Blattmann, T. Dockhorn, S. Kulal, D. Mendelevitch, M. Kilian, D. Lorenz, Y. Levi, Z. English, Voleti, Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.1511
2023.
[5] A. Brock. Large scale gan training for high fidelity natural image synthesis. arXiv preprint arXiv: 1809. 1096, 2018.
[6] H. Chang, H. Zhang. L. Jiang, Liu, and W. T. Freeman. Maskgit: Masked generative image transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11315-11323 11325, 2022.
[7] C. Chen, Qian, W. Hu, T.-J Fu, J. Tong, X. Wang, Li, B. Zhang, A. Schwing, W. Liu, et al. Dit-air: Revisiting the efficiency of diffusion model architecture design in text to image generation. XrXiv preprint WxXv:2503310066 2025.
[8] Chen, Xue, Y. Zhao, Yu, Paul, Chen, H. Cai, Xie, and Han. Sana-sprint: One-step diffusion with continuous-time consistency distillation, 2025.
[9] J. Chen, J. Yu, C. Ge, L. Yao, E. Xie, Y. Wu, Z. Wang. J. Kwok, P. Luo, H. Lu, et al. Pixart-o: Fast training of diffusion transformer for photorealistic text-to-image synthesis. arXiv preprint arXiv:2310.0042
2023.
[10] Y. Chen, G. Li, C. Jin, S. Liu, and Li. Ssd-gan: Measuring the realness in the spatial and spectral domains. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 1105-1112, 2021.
[11] T. Dao, T. H. Nguyen, T. Le, D. Vu, K. Nguyen, C. Pham, and A. Tran. Swiftbrush v2: Make your one-step diffusion model better than its teacher. In European Conference Computer Vision, pages 176-192. Springer, 2024.
[12] Dhariwal and A. Nichol. Diffusion models heat gans on image synthesis. Advances in neural information processing systems, 11:8780-8794 2021.
[13] P. Dvurechensky, A. Gasnikov, and A. Kroshnin. Computational optimal transport: Complex- ity by accelerated gradient descent is better than by sinkhorn's 's algorithm. In International conference machine learning, pages 1367-1376. PMLR, 2018
[14] P. Esser, S. Kulal, A. Blattmann, R. Entezari, J. Müller, H. Saini, Y. Levi, D. Lorenz, A. Sauer, Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference machine learning, 2024.
[15] P. Esser, S. Kulal, A. Blattmann, R. Entezari, J. Müller, H. Saini, Y. Levi, D. Lorenz, A. Sauer, Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first International Conference on Machine Learning, 2024.
[16] P. Esser, R. Rombach, and B. Ommer. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12873-12883, 2021.
[17] J. Frank, T. Eisenhofer, I.. Schönherr, A. Fischer, D. Kolossa, and T. Holz. Leveraging frequency analysis for deep fake image recognition. In International conference on machine learning, pages 3247-3258. PMLR, 2020.

---

[18] K. Frans, D. Hafner, S. Levine. and P. Abbeel. One step diffusion via shortcut models. arXiv preprint arXiv:2410.125521 2024.
[19] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial networks. Communications of the ACM, 63(11):139-144,
2020.
[20] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In I. Guyon, U. von Luxburg, S. Bengio, H. M. Wallach, R. Fergus, S. V. N. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 6626-6637, 2017.
[21] J. Ho. W. Chan, C. Saharia, J. Whang, R. Gao, A. Gritsenko, D. P. Kingma, B. Poole, M. Norouzi, J. Fleet, et al. Imagen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.02300, 2022.
[22] J. Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851,, 2020.
[23] J. Ho, C. Saharia, W. Chan, D. J. Fleet, M. Norouzi, and T. Salimans. Cascaded diffusion models for high fidelity image generation. Journal of Machine Learning Research, 23(47):1-33, 2022.
[24] J. Ho and T. Salimans. Classifier-free diffusion guidance. arXiv preprint 2022.
[25] E. Hoogeboom, J. Heek, and T. Salimans. simple diffusion: End-to-end diffusion for high resolution images. In International Conference on Machine Learning, pages 13213-13232. PMLR, 2023.
[26] M. Kang, J.-Y. Zhu, R. Zhang, J. Park, E. Shechtman, S. Paris, and T. Park. Scaling up gans for text-to-image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 10124-10134 2023.
[27] T. Karras, T. Aila, S. Laine, and J. Lehtinen. Progressive growing of gans for improved quality, stability, and variation. In International Conference on Learning Representations, 2018.
[28] Karras, S. Laine, M. Aittala, J. Hellsten, J. Lehtinen, and T. Aila. Analyzing and improving the image quality of stylegan. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 81 11-81119 19, 2020.
[29] M. Khayatkhoei and A. Elgammal. Spatial frequency bias in convolutional generative adversarial networks. In Proceedings of the AAAI Conference Artificial Intelligence, volume 36, pages 7152-7159, 2022.
[30] D. Kingma, T. Salimans, B. Poole, and J. Ho. Variational diffusion models. Advances in neural information processing systems, 34:21696-21707, 2021.
[31] Kynkäääniemmi, M. Aittala. T. Karras, S. Laine, T. Aila, and Lehtinen. Applying guidance in a limited interval improves sample and distribution quality in diffusion models. arXiv preprint arXiv:2404.07722 077224, 2024.
B. F. Labs. Flux. http://t.////.ga..oooo//ooo/ooar
2024.
[33] S. Lee, B. Kim, and J. C. Ye. Minimizing trajectory curvature of ode-based generative models. In International Conference Machine Learning, pages 18957-18973. PMLR, 2023.
[34] C.-L. Li. W.-C Chang. Y. Cheng, Y. Yang, and B. Póczos. Mmd gan: Towards deeper understanding of moment matching network. Advances in neural information processing systems, 30, 2017.
[35] Li, Y. Tian, Li, M. Deng, and He. Autoregressive image generation without vector quantization. arXiv preprint 24Xx:2406.118388 2024.
12

---

[36] Y. Li, K. Swersky, and R. Zemel. Generative moment matching networks. In International conference on machine learning, pages 1718-1727. PMLR, 2015.
[37] Y. Lipman, T. Chen, H. Ben-Hamu, M. Nickel, and M. Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.0277: 2022.
[38] X. Liu, C. Gong, and Q. Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.03000 2022.
[39] C. Lu and Song. Simplifying, stabilizing and scaling continuous- tatimuuous-tmee consistency models. arXiv preprint arXiv:2410. 10081, 2024.
[40] C. Lu, Y. Zhou, F. Bao, J. Chen, C. Li, and J. Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems, 35:5775-577 5787, 2022.
[41] N. Ma, M. Goldstein, M. S. Albergo, N. M. Boffi, E. Vanden-Eijnden, and S. Xie, Sit: Exploring flow and diffusion-based generative models with scalable interpolant transformers. arXiv preprint arXiv:2401.08740) 2024.
[42] T. H. Nguyen and A. Tran. Swiftbrush: One-step text-to-image diffusion model with variational score distillation. Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pages 7807-7816, 2024.
[43] OpenAl. Video generation models as world simulators. https: com/ sora/ 2024.
[44] W. Peebles and S. Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4195-4205, 2023.
[45] D. Podell, Z. English, Lacey, A. Blattmann, T. Dockhorn, J. Müller, Penna, and R. Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01192, 2023.
[46] A.-A Pooladian, H. Ben-Hamu, C. Cammingo-Enriche B. Amos, Y. Lipman, and R. T. Chen. Multisample flow matching: Straightening flows with minibatch couplings. In International Conference on Machine Learning, pages 28100-28127, PMLR, 2023.
[47] N. Rahaman, A. Baratin, D. Arpit, F. Draxler, M. Lin, F. Hamprecht, Y. Bengio, and A. Courville. On the spectral bias of neural networks. In International conference machine learning, pages 5301-5310. PMLR, 2019.
[48] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 10684-10695, 2022.
[49] C. Saharia, W. Chan, S. Saxena, L. Li, Whang, E. Denton, K. Ghasemipour, R. Gon- tijo Lopes, B. Karagol Ayan, T. Salimans, et al. Photorealistic text-to-image diffusion mod- els with deep language understanding. Advances in neural information processing systems, 35:36479-36494, 2022.
[50] K. Schwarz, Y. Liao, and A. Geiger. On the frequency bias of generative models. Advances in Neural Information Processing Systems, 14:18126-18136,, 2021.
[51] K. Schwarz, Liao, and A. Geiger. On the frequency bias generative models. Advances in Neural Information Processing Systems, 14:11126-1818 122266113666. 2021.
[52] J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. International conference on machine learning, pages 2256-2265. PMLR, 2015.
[53] Y. Song and P. Dhariwal. Improved techniques for training consistency models. arXiv preprint arXiv:2310.1 14189, 2023.
[54] Y. Song, P. Dhariwal, M. Chen, and I. Sutskever. Consistency models. 2xX1v:230..0149, 2023. 2023.
arXiv preprint
13

---

[55] Y. Song, J. Dohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arX1v:2011.13456. 2020.
[56] K. Tian, Y. Jiang, Z. Yuan, B. Peng, and L. Wang. Visual autoregressive modeling: Scalable image generation via sext-scale prediction. arXiv preprint arXiv:2404.022055 2024.
[57] Y. Tian, Z. Tu, H. Chen, J. Hu, C. Xu, and Y. Wang. U-dits: Downsample tokens in u-shaped diffusion transformers. arXiv preprint arXiv:2405.0277 2024.
[58] T. Wan, A. Wang, B. Ai, B. Wen, Mao, C.-W. Xie. D. Chen, F. Yu, H. Zhao, J. Yang. J. Zeng. J. Wang, J. Zhang, J. Zhou, J. Wang, J. Chen, K. Zhu, K. Zhao, K. Yan, L. Huang, M. Feng, N. Zhang, P. Li, P. Wu, R. Chu, R. Feng, Zhang, S. Sun, T. Fang, T. Wang. T. Gui, T. Weng,
T. Shen, W. Lin, W. Wang, W. Wang. W. Zhou, W. Wang. W. Shen, W. Yu, X. Shi, X. Huang,
X. Xu, Y. Kou, Lv, Y. Li, Y. Liu, Wang. Y. Zhang, Y. Huang, Y. Li. Y. Wu, Liu, Y. Pan.
Y. Zheng, Y. Hong, Y. Shi, Y. Feng. Z. Jiang, Z. Han, Z.-F. Wu, and Z. Liu. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv: 2503.20314 2025.
[59] Wang, Z. Li, T. Song. X. Li, T. Ge, B. Zheng, and L. Wang. Exploring DCN-like architecture for fast image generation with arbitrary resolution. In The Thirty-eighth Annual Conference Neural Information Processing Systems, 2024.
[60] S. Wang, Z. Li, T. Song, X. Li, T. Ge, B. Zheng, and L. Wang. Flowden: Exploring dcn-like ar- chitectures for fast image generation with arbitrary resolution. arXiv preprint arXiv:2410.22655. 2024.
[61] E. Xie, J. Chen, J. Chen, H. Cai, H. Tang, Y. Lin, Z. Zhang, M. Li, L. Zhu, Y. Lu, and S. Han. Sana: Efficient high-resolution image synthesis with linear diffusion transformer, 2024.
[62] Z.-Q. J. Xu, Y. Zhang, T. Luo, Y. Xiao, and Ma. Frequency principle: Fourier analysis sheds light on deep neural networks. arXiv preprint arXiv: (501.09900.06220 2019.
[63] Z.-Q.J. J. Xu, Y. Zhang, and Y. Xiao. Training behavior of deep neural network in frequency domain. In Neural Information Processing: 26th International Conference. ICONIP 2019, Sydney, NSW, Australia, December 12-15, 2019, Proceedings. Part I 26, pages 264-274. Springer, 2019.
[64] J. Yao, C. Wang, W. Liu, and X. Wang. Fasterdit: Towards faster diffusion transformers training without architecture modification. Advances in Neural Information Processing Systems, 37:56166 56189, 2024.
[65] T. Yin, M. Gharbi, T. Park, R. Zhang, E. Shechtman, F. Durand, and B. Freeman. Improved distribution matching distillation for fast image synthesis. Advances in neural information processing systems, 37:47455
2024.
[66] T. Yin, M. Gharbi, R. Zhang, E. Shechtman, F. Durand, W. T. Freeman, and T. Park. One-step diffusion with distribution matching distillation. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pages 6613-6623, 2024.
[67] S. Yu, S. Kwak, H. Jang. J. Jeong. J. Huang, J. Shin, and S. Xie. Representation alignment for generation: Training diffusion transformers is easier than you think. In International Conference on Learning Representations, 2025.
[68] Y. Zhang and B. Hooi. Hipa: enabling one-step text-to-image diffusion models via high- rrequencypromoting adaptation. arXiv preprint arXiv:2311.18158, 2023.
[69] I.. Zhou, S. Ermon, and J. Song. Inductive moment matching. arXiv preprint arXvv:2503.07565 2025.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our contributions are accurately reflected in the abstract and in the summary introduction section Section
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We explicitly mention the limitation for future works in Section g
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
15

---

N/A, because our work focuses solely on empirical evaluation to measure the effectiveness of the proposed framework.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Yes, this information can be found in Section 7 and in Appendix.
Guidelines:
The answer NA means that the paper does include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) If the contribution is new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
16

---

Answer: [Yes]
Justification: Yes, we will release our code upon acceptance.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/
wwt.Suubmisiooooiioooooooooo for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
Yes, we provided them in Section 7 and in Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: No, because the evaluation process takes hours on GPU and is very ex- pensive to compute with multiple seeds. Importantly, we conducted evaluation for several benchmarks Section 7 so running with different seeds are not feasible in our scope.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
17

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Yes, we provided them in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: Yes, totally confirm to it.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discussed the societal imparts in Section g
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
18

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification:
Yes, our code will be released with Licences.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Yes, we have properly cited the code, data and models used.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
19

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: At the time of submission, no new assets are being released. We plan to release the associated code publicly if the paper is accepted.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodll as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
20

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: The paper does not involve LLM in the core of this method.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to L.LM policy for what should or should not be described.
21

---

A Proof of Proposition 1
Given shortcut model 8e and its corresponding classifier-free guided output gä with guidance scale w. To analyze the generation process, we define sequence of intermediate points {æ', J This sequence is constructed by recursively applying N consecutive shortcut steps, each with the smallest step size d 1/N. Note that N is chosen to be power of 2. The process begins from the initial noise Σω at = O, conditioned on c. The starting point the sequence is thus defined as zg 2o. and subsequent intermediate points generated as follow:
x +
x
+ 80
x N ) d, for = NN... -.
(13)
Ideally, we assume that the model is perfectly trained, or equivalently, that the loss in Eq. (4) is minimized zero. Under this assumption, we have:
59(2 t, c, 2d) 2 [ a4 t, d) + 90 (z(+ + d, c, d)],
(14)
1
s0(2 t, 2, 2d)
2, + so t d, Ø, d)].
(15)
2
In the following derivation, we omit the time / in the network notation for simplicity, unless otherwise specified. First, we will prove by induction that:
2
g8 ( c, 2' = 2
Σ
98
105
(Iezia: c, d.
(16)
For the base case 0, we have 96 (2/ c, d) gø (a C, d,,,, which satisfies Eq. (66. Assuming
that Eq. (T6) holds for j = k, we now show that also holds for j = + 1. Using Eq. (14) and
Eq. 15
obtain:
1 L (a/s e, d) gb (2/ 22 c, dd]
80(2 2 d)
2
2 -1
2
Σ
9
(2) c,
Σ
9#
wr
(a (a
2
2
2
Σ
99
w
x+1
1=0
s0 Ø, 2 d)-
[so(a)) (2/ Ø, 2d s Ø, 2
2
2
Σ
so
s0 2.d).
2
i=0
Therefore, we have: 9% (x/, 2 d)
wss( c,,,,, 2 d) + (1 (1)
[ b+1
(2/4 c, + (1 w) ww)) 2, d,]
21+1
26+1
Σ
[w(w
sa(0 c, d) + (1 x 10660 B, + (1 w)so(#(1ea 2,
2k+1
28+1
Σ
(2/ c, + (1- n 200 8, R.d]
2k+1
2=0
Σ
(2(+id) c..
gø
2k+1
0=0
22

---

This satisfies Eq. (16 for j = k + 1. By induction, we have Eq. (16 holds for all j. Based on this result, we can rewrite the output of the shortcut model for single large step of size Nd denoted by oo C, Nd), follows:
so (o c, Nd)
98
an.
N
+ 96
N e/
2d)
)
2
2
N 2-1
N/2-1
Σ
g
10/2
F'4. (.))
+
Σ
98
(at
# c.d)
N
1=
1=0
1
N-1
Σ
(N)
(2).
x
9
N
1=0
which completes the proof.
B Multi-Level Wavelet Function
Multi-Level Wavelet Function. Algorithm 1 provides pseudo-code for our proposed multi-level wavelet objective described in Section 4
Algorithm Multi-level Wavelet Function
class hntt:Laveliaveleet..oss def __init___ssell celf.dwt DVT_2D("haes) Discrete Navelet Transforu (DWT)
self diff_func MSELosa() Distance function
def cancatenated_drtteee x): xlb, xhl, xhh (x) Decompose 4 sarelat sub-banda using DMT details corch xlh, xhl, xhh], din-1) # Concatenate the sub-bands return details
def (self, pred, target, nun num_levels): total_lose diff fune(_enep,edd target) ()ean() Calculate 1ov-level original output space
Becursively calculate loss vavelet sub-bands for each level
pred_curt target_curr pred, target
for current, level in range leun__evals):
target aub-bande using outputa previous level Derive predicted comcateeated_dorppreedd pred_ bands self. target_bands hands self concstanated_ddtttaaaee (targggttrrt
Cal falculate loss current level total_loss diff fure_fffccpprdd____o bands, target target.bande) bands) meaan)) pred_curr, target_curr pred_ bande, target_bande
Taking average from all levels loss total_loss (oun, raturn
presents qualitative comparisons illustrating the impact different levels in our multi-level wavelet function versus traditional low-level loss. Incorporating more wavelet levels yields finer details and fewer artifacts, especially in one- and few-step generation.
Results. Fig. 3
C Injecting Conditional Inputs into Network
We explore two primary strategics for incorporating conditional information including CFG scale w, current sample H,, time t, condition c, and the target step size into our network. The first strategy, similar to U-ViT (2). encodes each condition an individual token appended to the input sequence of noisy image patch tokens. The second strategy employs AdaLN-Zero blocks [44] to modulate the network with each condition; these modulations then aggregated through addition. We find that the AdaLN-Zero approach yields comparable performance to the U- VIT method but without the drawback of increased input sequence length. Given this efficiency benefit, adopt AdaLN-Zero for injecting CFG information into our network.
23

---

Navelet
Navelet,
Wavelet,
Wavelet,
Figure Effect of applying different numbers of wavelet loss layers (L) on image quality under and few-step inference. The traditional low-level objective (leftmost cnlumn) results in noticeable degradation and artifacts. contrast, using multi-level wavelet loss, especially with = 5 (green box, right), produces high-quality, consistent images across all inference settings.
D Accelerating training speed
To mitigate the high computational cost of short- cut models, we explore resuming their training from existing a flow matching weights [67]. Ta- shows that while resuming offers limited improvement for the original shorteut model, improved shortcut models (iSM) demon- strate considerable gains. Specifically, when in- corporating series of proposed methods, iSM achieves significantly better FID scores. Models
Table Performance of Improved Shortcut Models
FID v
FID x4
Shortcut Models
18
21.38
13:46
Improved Shorteut Models (iSM)
Intrinsic Guidance
9.62
3.17
Interval Guidance in Training
8.49
2.81
Function
Scaling Optimal Transport
7.97
2.23
Twin EMA
2.16
trained for 250k steps for all evaluations.
E Experiment Settings
E.1 Training & Parameterization Setting
For experiments on ImageNet, images are preprocessed to 256x256 resolution, following the protocol of ADM [2]. Our models operate in the latent space derived from the sd- vae-ft-mse autoencoder 1. These latents are subsequently normalized channel-wise us- ing means of [0.86488, -0.27787343, 0.21616915, 0.3738409] and standard deviations of [4.85503674, 5.31922414, 3.93725398, 3.9870003]. Finally, the normalized latents are scaled

---

Table 6: Experimental seltings ImageNet 256 x 256.
Parameterization Setting
Architecture
SiT-XL
GFlops
118.64
Params (M)
675 OT-FM
Flow Trajectory
Input dim.
32x32x4 4
Num. layers
24 1024
Hidden dim.
Num. heads
16
dr
Tt
Training objective
v-prediction
Training Setting
Training iteration
800K
Dropout
0 Adam
Optimizer
AdamW 31
0.9
AdamW 32
0.999
AdamW
10 8
Learning Rate
0.0001
Weight Decay
Batch Size
256
Label Dropout
0.1
Methods
CFG Scale Wmax
3.5
Interval finterval
0.3
Wavelet Levels
OT Scale K
32
Ratio of Empirical Self-comsistency Targets
0.25
EMA Parameters Used For Self-consistency Targets?
True
EMA Target Rate Dargge
0.95
EMA Parameters Used For Evaluation?
Truc
EMA Inference Rate 0 Onferr
0.9999
by a factor of 0.5, targeting approximate standard deviation of 0.5 for the network input. Detailed configurations for all experiments are provided in Table
E.2
Evaluation Details
We follow the ADM 12 evaluation setup, using the same reference batches from their official implementation and compute FID (20] over 50K generated images.
E.3
Baselines
Below, we summarize the baseline methods used our evaluation.
ADM 12
improves U-Net-based diffusion architectures and introduces classifier-guided sampling to balance sample quality and diversity.
proposes cascaded diffusion models, which generate images in a coarse-to-fine manner, similar to ProgressiveGAN 27
CDM 23
25

---

and Simple diffusion 25
leverages a diffusion model for high-resolution images by simplifying the noise schedule and model architectures.
LDM 48)
introduces the concept of operating diffusion models in compressed latent space, enhancing efficiency.
U-DiT 57
proposes a series of U-shaped DiTs based on self-attention with downsampled tokens.
U-ViT 2 adapts Vision Transformers for latent diffusion by incorporating U-Net-like long
skip connections.
DiT
44 pioneers the use of pure transformer architecture as the backbone for diffusion models, featuring AdalN-zero modules.
SiT 41 investigates how to improve DiT training by transitioning from discrete diffusion to continuous flow-based modeling.
REPA 67 accelerates the training of DiT/SIT models by regularizing network representa- tions align with features from pretrained visual encoders.
FlowDCN 60 offers fully convolutional architecture for generative modeling with linear time and memory complexity, enabling efficient high-resolution image synthesis.
iCT [53] introduces several techniques to enhance the training of CMs [54], including a log-
normal noise schedule, Pseudo- Huber loss functions, and a scheduler for total discretization steps during training.
SM [8] establishes a framework for one-step generation by combining flow matching with self-coosistency objective.
IMM [69 few-step generative models by inductively matching all moments of bootstrapped samples derived from stochastic interpolants using Maximum Mean Discrepancy (MMD), aiming for stable convergence.
Related Works
F.
Diffusion, Consistency Flow Matching models
Generative modeling has seen significant advancements with methods like diffusion models, con- sistency models, and flow matching. Diffusion models [52 55
30 are generative framework that synthesizes data by gradually transforming random noise into outputs via a stochastic denoising process. To introduce efficient generation, consistency models 54
22
39 aim for efficient gen- 53
eration by learning mapping from any point on solution trajectory directly to the data origin. Meanwhile, flow matching [37 38) proposes learning generative models by defining a target vector field between noise and data and training neural network to approximate this field. Both these methods have demonstrated strong performance when scaled to text-to-image text-to-vides
21 43
applications.
F.2
End-to-end training for one-to-many step generative models
The quest for efficient, high-quality few-step generation has explored several avenues. While early one-step models leveraged GANs [19
28 5 and MMD 36 34 the inherent instability and
complexity adversarial training limit their scalability. Consistency Models (CMs) [54] 53 39
address the instability of adversarial training without needing synthetic datasets. Unlike distillation, CMs can be trained from scratch via consistency training (CT), independent of pre-trained diffusion models. However, few-step generation with CMs usually involves discrete-time variants. These variants require meticulous timestep scheduling and are susceptible to irreducible bias accumulation due to the inherent ambiguity of discretization. Inductive Moment Matching (IMM) [69] further
advanced stable few-step training using moment matching. Shortcut Models (SMs) [8] introduced
step-size conditioning with self-bootstrapping. However, original SMs (Section 3) suffer from
inflexible guidance and guidance accumulation artifacts, which our work directly addresses.
G More Qualitative Results
26

---

Figure 4: Uncurated samples on ImageNet-256 256 with 1-step sampling.
27

---

Figure 5: Uncurated samples on ImageNet-256
256 with 2-step sampling.
28

---

Figure Uncurated samples on ImageNet-256 x 256 with 4-step sampling.
29

---

Figure 7: Uncurated samples on ImageNet-256 x 256 with 8-step sampling.
30

---

Figure 8: Uncurated samples ImageNet-256 x 256 with 128-step sampling.
31


Reviews:
[{"review_id": "zXMPDto2wE", "reviewer": "Reviewer_xmQN", "summary": "This paper presents an Improved Shortcut Model (iSM) to address key limitations of existing shortcut models in generative modeling. The identified issues include fixed classifier-free guidance (CFG) scales limiting inference flexibility, high training variance from random noise-data pairings, frequency bias toward low-frequency components, and conflicts between EMA updates and self-consistency objectives. To tackle these, the authors propose a unified framework incorporating: (1) Integrated Guidance for dynamic guidance scaling at inference; (2) Multi-Level Wavelet Loss to mitigate frequency bias; (3) Scaling Optimal Transport (sOT) Matching to reduce training variance; and (4) a Twin EMA strategy to balance self-consistency and training stability. Extensive experiments on ImageNet 256×256 demonstrate that iSM achieves significant FID improvements across on1e-step, few-step, and multi-step generation, outperforming baseline shortcut models and comparable methods.", "questions": "1. How about other metrics for comparison, such as LPIPS or IS?\n2. The Scaling OT (sOT) method is very interesting. Have you investigated the effect of the update frequency? You currently recompute it every K batches; would performance change if it were recomputed more or less frequently, and how does this interact with the scaling factor K?\n3. The accumulated guidance problem is formalized for N=128 base steps. Does this problem persist for other base step sizes (e.g., N=64 or N=256)? How does iSM’s Integrated Guidance adapt to varying N?\n4. Regarding the Twin EMA strategy, the choice of a fast decay rate of 0.95 for the target network seems relatively low compared to the slow rate of 0.999. Could you provide more intuition on how you arrived at this value? Is there a risk of the target network becoming too noisy or unstable with such a fast update?", "limitations": "1. The gap between one-step and multi-step is still significant, which is against the claim of this paper.\n2. Lack large-scale experiments to further support the claims of this paper.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The paper identifies both known (e.g., fixed CFG scales) and overlooked (e.g., accumulated guidance effects, EMA-self-consistency conflict) limitations of shortcut models, providing a clear motivation for the proposed improvements.\n2. The proposed four core components in this paper (Integrated Guidance, Multi-Level Wavelet Loss, sOT, Twin EMA) are complementary, forming a cohesive solution that addresses multiple issues simultnaeously.\n3. The authors conduct thorough experiments, including quantitative comparisons with state-of-the-art models, ablation studies for each component, and analysis of key hyper-parameters.\n4. The proposed iSM supports dynamic guidance scaling and reduces inference time by ~50% compared to standard CFG, improving real-world applicability for resource-constrained scenarios.\n\n\nWeaknesses:\n1. All experiments are conducted on ImageNet 256×256 and this paper lacks further solid experimental evaluations on larger-scale experiments, such as ImageNet 512x512, or other text-to-image experiments. It remains unknown how well these techniques and hyper-parameters perform on more complex generation tasks.\n2. Though the proposed method iSM reduces inference steps, the performance is still limited compared to other cutting-edge models, such as REPA-XL/2 (FID=1.42) (compared to iSM-128NFE, 1.88 FID). In addition, it still lags behind multi-step methods (FID 5.27 vs. 1.88 in 128 steps). The gap is significant. \n3. The proposed iSM is tested only with the SiT-XL/2 architecture; its compatibility with other backbones (e.g., DiT, U-ViT) is unproven, limiting its generalizability.", "was_revised": true, "final_justification": "The author's response has addressed all my concerns. After carefully reading the comments from other reviewers, I believe this work deserves acceptance, and therefore, I have decided to further raise my rating to a clear accept.", "rebuttal": "Thank you for your detailed review and constructive feedback. We appreciate that you found our motivation clear, our proposed solution cohesive, and our experiments thorough. Below, we provide a point-to-point response and a summary of the corresponding revisions.\n\n**Note**: References to tables, figures, or propositions without the 'Rebuttal' prefix refer to content located in the main paper.\n\n**Q1: Experiments on ImageNet 512x512, other backbones, and generalizability.**\n\nOur work introduces a robust, architecture-agnostic training methodology for shortcut models. While we report results on SiT-XL/2 to align with the Shortcut Model baseline, our technique generalizes beyond Transformer-based models. Due to limited time, we tackle both backbone and larger image size within one experiment. Specifically, we **scaled our approach to ImageNet 512×512 using XL/2 variant of FlowDCN**, a fully convolutional generative model with group-wise deformable convolution blocks instead of a DiT-based model. Given the rebuttal time, our models have not been fully trained, and we report their scores at the latest iteration (300K) in the table below. We achieved a substantial FID improvement compared to the original shortcut model training, demonstrating both **scalability and broad applicability**.\n\n**Rebuttal Table 1: ImageNet 512x512 experiment.**\n|Model(FlowDCN)|NFE|#Iter|FID↓|Precision↑|Recall↑|\n|---|---|---|---|---|---|\n|SM|1|300K|43.81|0.56|0.11|\n|**iSM(ours)**|1|300K|**37.05**|**0.60**|**0.55**|\n|SM|4|300K|12.16|**0.86**|0.19|\n|**iSM(ours)**|4|300K|**9.94**|0.78|**0.62**|\n\nIn line with common practice in diffusion model research, our work focuses on class-to-image tasks, as scaling to full text-to-image generation is challenging under current resource and time constraints. Many influential papers in this field, such as DiT, SiT, EDM, Consistency Models, and Shortcut Models, have **established their contributions through experiments on label-to-image generation**, thereby laying the groundwork for future scaling to text-to-image diffusion models.\n\n__Q2: Comparison with other cutting-edge models__\n\nA direct FID comparison between iSM and models like REPA can be misleading, as they operate in fundamentally different **inference-efficiency regimes**. REPA is a state-of-the-art **full-trajectory** diffusion model requiring 250 NFE to achieve its FID of 1.42. Our iSM achieves a competitive FID of **1.93 with just 8 NFE**. This represents a **>30x reduction in computational cost** for the trade-off in final FID. Our contribution lies in **improving shortcut models across at extreme NFE efficiency**, achieving a significant FID reduction in these settings over the original baseline and competitive performance with other model families.\n\n__Q3: Additional evaluation metrics (FD-DINOv2, IS, Precision, Recall)__\n\nWe report FD-DINOv2, Inception Score (IS), Precision, Recall for the baseline Shortcut Model (SM), Inductive Moment Matching (IMM), and our improved Shortcut Model (iSM). The results below show that iSM consistently outperforms the baseline SM across multiple NFE settings.\n\n**Rebuttal Table 2: Additional metrics for ImageNet 256x256.**\n|Model|NFE|FD-DINOv2↓|IS↑|Precision↑|Recall↑|\n|:---|:---|:---|:---|:---|:---|\n|SM|1|500.92|102.66|0.68|0.46|\n|IMM|1|247.78|128.87|0.67|**0.64**|\n|**iSM (ours)**|1|**232.31**|**223.52**|0.70|0.61|\n|SM|2|329.53|125.66|**0.77**|0.47|\n|IMM|2|152.08|173.66|0.74|**0.64**|\n|**iSM (ours)**|2|**107.63**|**302.29**|**0.77**|0.61|\n|SM|4|265.90|136.79|**0.80**|0.48|\n|IMM|4|110.88|204.95|0.77|0.62|\n|**iSM (ours)**|4|**83.70**|**298.23**|0.77|**0.62**|\n\n\n__Q4: The Scaling OT (sOT) method is very interesting. Have you investigated the effect of the update frequency?__\n\nYes, we performed an ablation over the update interval K to measure its impact (see Table 2 in main paper). We found:\n- Better training: Increasing K yields a closer match to the global transport plan, which straightens training trajectories and boosts stability.\n- Minimal overhead: Even at a larger K, the extra overhead remains modest (~4% in our final setup, K = 32).\n- Saturation point: As shown in Table 2, the FID scores improvements plateau past a certain K, so there’s little benefit in updating more frequently.\n\n__Q5: Does the accumulated guidance problem persist for other base step sizes (N)?__\n\nYes, and we emphasize that this is one of our key contributions. The accumulated guidance problem is a **systemic flaw** in the recursive formulation of shortcut models. Our **Proposition 1** provides the first formalization of this compounding error, showing that the effective guidance scales undesirably as $w^{log_2(N)}$.\n\nOur **Integrated Guidance** framework resolves this issue by making the guidance as an explicit model input `w`, we train the network to learn the mapping `(x, t, c, d, w) -> v_guided` directly. This **bypasses the flawed recursive formulation entirely**, making our solution inherently robust and independent of the base step size `N`. We’ll underscore this contribution in Section 3.2.\n\n\n__Q6: Intuition for Twin EMA decay rate (0.95)__\n\nThe selection for decay rate for fast-decay EMA ($\\theta_{target}$) needs to satisfy the following:\n- **Low enough** to keep the self-consistency target network closely aligned with the online network, avoiding the distributional lag as training progresses.\n- **High enough** to preserve the benefits of EMA, which smooths noisy mini-batch gradients and prevents training instability.\n\n\nTherefore, a much lower rate decay for $\\theta_{target}$ risks instabilility and negates the benefits of EMA, while a much higher rate negates the benefit of the twin strategy and reintroduces distributional lags. Our ablation in Table 2 empirically validates that 0.95 provides the optimal trade-off. The decay rate of 0.95 allows the self-consistency target to closely track the online network, but with just enough lag to smooth out high-frequency noise in the training signal.\n\n\n__Q7: The gap between one-step and multi-step performance__\n\nWe would like to clarify that our paper does not claim to completely eliminate the gap between one-step and multi-step generation. Instead, our central contribution is to **significantly reduce the performance degradation at lower NFEs** compared to previous shortcut models, while also providing a single flexible model that performs well across a wide range of sampling settings. We identified the degradation as primarily resulting from: (1) accumulated guidance, (2) low-frequency bias, (3) high-variance/curved trajectories, and (4) EMA-lagged self-consistency. For instance, our iSM model improves the FID from a much higher value in SM (10.60) down to **5.27 for one-step** and further down to **2.05 for four-step generation**. As noted in our conclusion (L381-383), while bridging this gap remains a challenging task, it offers an exciting and promising avenue for future research."}, {"review_id": "250BUWzXpl", "reviewer": "Reviewer_DUtt", "summary": "The authors introduce multiple numerical tricks to make class-conditional consistency models competitive with standard class-conditional generative models. The tricks are the following: a wavelet-based loss, a ¨scaled¨ optimal transport flow matching, and leveraging EMA networks with multiple decays (a slow decay for inference, and a large decay to anchor the consistency model).", "questions": "- ¨A critical, previously overlooked issue in shortcut models is the\naccumulation of guidance effects.¨ Is this a well known problem? If yes can authors provide a reference on it? if not, is it a contribution?\n\n\nAblation study.\nI love the fact authors provide a detailed ablation study, however I have some questions about it:\n- Some of the performance gain claims seems rather small and might not be significative, especially for the ¨augmented OT¨ batch parameter K. To the best of my knowledge, OT flow matching makes bring empirical improvement when one can only use a small number of inference steps (NFE<10, which makes sense in your setting), and does not bring improvement with a large number of inference steps (NFE> 50). However, computing OT pairing in the 256*256 image space should not make sens: the dimension is too large compared to the number of samples to give meaningful information. I wonder if authors have any kind of intuition on this?\n- If understand well your benchmark, GANs is still the best 1-step generation model for conditional generation?\n\n\n\n- Are Section 4-6 contribution of the paper?\n\n\nMulti-level Wavelet Function:\n- Could authors provide additional information on the ¨multi-level Wavelet Function¨? Is it mainly a modification of the loss?\n- Do you use a latent space? as in [3]?\n\n\n[1] Stein, George, et al. \"Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models.\" Advances in Neural Information Processing Systems\n\n[2] Sajjadi, Mehdi SM, et al. \"Assessing generative models via precision and recall.\" Advances in neural information processing systems 31 (2018).\n\n[3] Rombach, Robin, et al. \"High-resolution image synthesis with latent diffusion models.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- The paper is easy to follow and tackles an important problem\n\nWeakness:\n- Given the performance improvements (all methods perform very similarly), I would recommend to add other performance metrics, such as the FID with DINOv2 embedding (as recommended in [1], instead of the inception network), maybe also the precision and recall metrics [2]?", "was_revised": true, "final_justification": "The authors answered my concerns and thus I raised my score.", "rebuttal": "Thank you for your valuable feedback and questions. We are glad you found the paper easy to follow and the problem important.\n\n**Note**: References to tables, figures, or propositions without the 'Rebuttal' prefix refer to content located in the main paper.\n\n__Q1: Additional evaluation metrics (FD-DINOv2, IS, Precision, Recall)__\n\nWe report FD-DINOv2[1], Inception Score[2] (IS), Precision[3], and Recall[3] for the baseline Shortcut Model (SM), Inductive Moment Matching (IMM), and our improved Shortcut Model (iSM). The results below show that **iSM consistently outperforms the baseline SM** across multiple NFE settings.\n\n**Rebuttal Table 1: Additional evaluation metrics for ImageNet 256x256.**\n| Model | NFE | FD-DINOv2↓ | IS↑ | Precision↑ | Recall↑ |\n| :--- | :--- | :--- | :--- | :--- | :--- |\n| StyleGAN-XL | 1 |  **210.49** | **264.86** | **0.80** | 0.50 |\n| SM | 1 |  500.92 | 102.66 | 0.68 | 0.46 |\n| IMM | 1 |  247.78 | 128.87 | 0.67 | **0.64** |\n| **iSM (ours)** | 1 |  232.31 | 223.52 | 0.70 | 0.61 |\n| SM | 2 |  329.53 | 125.66 | **0.77** | 0.47 |\n| IMM | 2 | 152.08 | 173.66 | 0.74 | **0.64** |\n| **iSM (ours)** | 2 | **107.63** | **302.29** | **0.77** | 0.61 |\n| SM | 4 | 265.90 | 136.79 | **0.80** | 0.48 |\n| IMM | 4 | 110.88 | 204.95 | 0.77 | 0.62 |\n| **iSM (ours)** | 4 | **83.70** | **298.23** | 0.77 | **0.62** |\n\n\n__Q2: Is the \"accumulated guidance\" issue a well-known problem? Is it a contribution?__\n\nTo the best of our knowledge, the **explicit formalization and analysis** of the accumulation of guidance effects in shortcut models is a **novel contribution** of our paper. While CFG is well-studied, its compounding effect in shortcut models has been overlooked. We are the **first to identify, formalize (Proposition 1), and solve this critical flaw**. We’ll revise the introduction to emphasize this contribution, making its importance immediately clear.\n\n__Q3: Significance of OT gains and intuition for high-dimensional OT__\n\n1.  **OT operates in Latent Space:** We agree that OT in 256x256 pixel space may be intractable. **In fact, all our operations, including OT matching, are performed in the 32x32 latent space** of a pre-trained VAE. We apologize if this was not clear.\n2.  **Significance of Gains:** In the high-performance regime (FID < 3.0), improvements are non-linear and dropping from **2.64 to 2.23 is a major leap**, since each FID reduction in this regime becomes increasingly challenging (for example, in the paper SiT's Table 1, last two rows, when applying CFG, the improvement drop from 2.27 to 2.06)\n\n__Q4: If I understand well your benchmark, GANs is still the best 1-step generation model for conditional generation?__\n\nIt’s true that state-of-the-art GANs achieve the lowest FID in a single step, but they’re notoriously hard to train and often suffer from **diversity collapse**. In contrast, our diffusion-based iSM **trains stably** (no adversarial loss) and delivers **significantly better sample diversity** (Recall of 0.61 vs. StyleGAN-XL of 0.50), narrowing the FID gap while ensuring a **stronger diversity–fidelity trade-off**. Furthermore, iSM’s **flexible sampling schedules** and efficient guidance sampling set it apart from other diffusion models.\n\n__Q5: Are Sections 4-6 a contribution of the paper?__\n\nAbsolutely. In fact, in the Introduction (Sec. 1, lines 53–76) we explicitly identify these shortcomings of vanilla shortcut models: (1) low-frequency bias, (2) high-variance/curved trajectories, and (3) EMA-lagged self-consistency, and then state our three complementary contributions to address them:\n- Multi-Level Wavelet Loss (Section 4) to correct frequency bias;\n- Scaling Optimal Transport Matching (Section 5) to straighten and stabilize trajectories;\n- Twin EMA Strategy (Section 6) to remove the temporal lag in self-consistency targets.\n\nThese sections detail each of our contributions. As shown in Table 2, progressively adding the components from Sec. 4–6 steadily improves the model’s performance, confirming their novelty and importance.\n\n__Q6: Could authors provide additional information on the \"Multi-Level Wavelet Function\"?__\n\nCertainly. The Multi-Level Wavelet Loss functions as a **multi-scale objective** that tackles the frequency bias. Instead of naively applying L2 loss in the spatial domain, it **recursively** applies Discrete Wavelet Transform (DWT) to the prediction and target. This decomposes the error signal across various frequency bands, forcing the network to minimize errors in both low-frequency structures and high-frequency details simultaneously. For reference, Appendix B provides the pseudocode (Algorithm 1).\n\n__Q7: Do you use a latent space?__\n\nYes. Following previous works like DiT, SiT, SM, our model operates entirely in the latent space of the pre-trained `sd-vae-ft-mse` autoencoder. We will make this detail more prominent in the main text to avoid any confusion.\n\n[1] Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models, NeurIPS 2023.\n\n[2] Improved techniques for training gans, NeurIPS 2016.\n\n[3] Improved Precision and Recall Metric for Assessing Generative Models, NeurIPS 2019."}, {"review_id": "PjmXOKGr1x", "reviewer": "Reviewer_MiHU", "summary": "This paper proposes several techniques to improve training of shortcut models. Specifically, they introduce 5 improvements: (1) integrate guidance scale as an input to the network, (2) interval guidance, (3) multi-level wavelet loss, (4) twin ema that separates the model weight for inference and self-consistency target, (5) OT matching. By incorporating all of these five components, the authors show shortcut model training can be greatly improved compared with the vanilla model training.", "questions": "N/A", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n- The paper is generally well-written and easy to follow.\n- Several components looks novel, e.g., applying OT over aggregated multiple minibatches.\n- The improvement compared with the vanilla shortcut model seems quite significant.\n- The authors provide component-by-component analysis.\n\n**Weaknesses**\n- There's no detailed explanation of how sOT is performed. Could the authors provide more detailed explanation (e.g., algorithm)?\n- OT will require additional computational and memory burden. The paper mentioned it but it would be great if more explicit analysis (e.g., by numbers) were provided.\n- I wonder if Multi-Level Wavelet Function is scalable to resolution, e.g., if it still works with ImageNet 512x512.\n- Interval guidance improves the performance, but at the same moment, it also induces a new hyperparameter $t_{\\text{interval}$ that should be set at training. While the authors show 0.3 is optimal in their experiments, but experiments on other setups (e.g., other datasets or higher resolution datasets) would be make the paper more stronger by showing such values are robust to different setups.\n- The paper only uses FID for quantitative evaluation. Other metrics (e.g., IS, Precision, Recall) can be provided.", "was_revised": true, "final_justification": "I lean toward accepting the paper, though not strongly, as it focuses on technical improvements to shortcut models and limits its scope to class-conditional ImageNet generation. Considering both its empirical and academic contributions, I decide to maintain my current score.", "rebuttal": "Thank you for the positive and constructive review. We appreciate that you found the paper well-written, the components novel, and the improvements significant. We address your points below.\n\n**Note**: References to tables, figures, or propositions without the 'Rebuttal' prefix refer to content located in the main paper.\n\n__Q1: Experiments on ImageNet 512x512, other backbones, and generalizability.__\n\nOur work introduces a robust, architecture-agnostic training methodology for shortcut models. While we report results on SiT-XL/2 to align with the Shortcut Model baseline, our technique generalizes beyond Transformer-based models. Due to limited time, we tackle both backbone and larger image size within one experiment. Specifically, we **scaled our approach to ImageNet 512×512 using XL/2 variant of FlowDCN**, a fully convolutional generative model with group-wise deformable convolution blocks instead of a DiT-based model. Given the rebuttal time, our models have not been fully trained, and we report their scores at the latest iteration (300K) in the table below. We achieved a substantial FID improvement compared to the original shortcut model training, demonstrating both **scalability and broad applicability**.\n\n**Rebuttal Table 1: ImageNet 512x512 experiment.**\n|Model(FlowDCN)|NFE|#Iter|FID↓|Precision↑|Recall↑|\n|---|---|---|---|---|---|\n|SM|1|300K|43.81|0.56|0.11|\n|**iSM(ours)**|1|300K|**37.05**|**0.60**|**0.55**|\n|SM|4|300K|12.16|**0.86**|0.19|\n|**iSM(ours)**|4|300K|**9.94**|0.78|**0.62**|\n\nIn line with common practice in diffusion model research, our work focuses on class-to-image tasks, as scaling to full text-to-image generation is challenging under current resource and time constraints. Many influential papers in this field, such as DiT, SiT, EDM, Consistency Models, and Shortcut Models, have **established their contributions through experiments on label-to-image generation**, thereby laying the groundwork for future scaling to text-to-image diffusion models.\n\n__Q2: Detailed algorithm for sOT__\n\nThe procedure is as follows:\n1.  **Aggregate Batches:** For every `K` training steps, we collect and hold the noise samples and data latents from `K` separate mini-batches of size `M`.\n2.  **Compute Large-Scale OT Plan:** We compute a single OT plan over the aggregated set of `K*M` samples.\n3.  **Redistribute and Train:** The resulting `K*M` optimally paired samples are then split back into `K` mini-batches, which are used for the subsequent `K` model training steps.\n\nThis process allows the model to benefit from a large-scale transport plan without the memory burden of a large training batch size. We will add pseudo-code algorithm for Scaling Optimal Transport (sOT) into the paper.\n\n__Q3: Computational and memory overhead of sOT__\n\nThe computational overhead from sOT is a modest **~4%** increase in total training time, while the memory overhead is negligible. \n\n- **Memory Efficiency:** Our sOT implementation is memory-efficient because it **decouples the OT batch size (`K*M`) from the model's training batch size (`M`)**. We only need to store the `K*M` latent vectors, while the memory-intensive model weights and gradients are processed with the standard, smaller batch size `M`.\n- **Computational Cost:** For `K=32` and a per-GPU batch size of `M=64`, the OT plan is computed over 2048 samples every 32 steps. This adds only ~4% to the total training time on an A100 GPU, a small price for the significant gains in trajectory stability and final FID scores. We report the training iteration speed for different values of K (where K=0 indicates no OT used).\n\n**Rebuttal Table 2: Training speed with sOT in ImageNet 256x256.**\n| K | 0 | 1 | 8 | 32 |\n| :- | :- | :- | :- | :- |\n| **Training speed (s/iter)** | 0.361 | 0.362 | 0.368 | 0.376 |\n\n**Q4: I wonder if Multi-Level Wavelet Function is scalable to resolution, e.g., if it still works with ImageNet 512x512.**\n\nYes, the Multi-Level Wavelet Loss is highly scalable and particularly well-suited for higher resolutions. The Discrete Wavelet Transform (DWT) is an inherently multi-scale algorithm. Its ability to disentangle features across frequency bands becomes **even more critical** for high-resolution images, where preserving fine-grained textures (high-frequency) alongside global composition (low-frequency) is essential for quality. This makes it a robust choice ([1], [2]) for scaling to 512x512 and beyond. Our result in Rebuttal Table 1 on ImageNet 512x512 also strengthen this argument, since iSM significantly outperforms the original shortcut model.\n\n__Q5: Robustness of the interval guidance hyperparameter__\n\nWe affirm that $t_{\\text{interval}}$ is a robust hyperparameter, a finding supported by theoretical motivation, our internal ablation studies, and external validation from concurrent work.\n\n1.  **SNR-Based Principle:** The parameter is not arbitrary but is grounded in the signal-to-noise ratio (SNR) dynamics of diffusion models. Notably, previous work [1] found that guidance is ineffective or harmful at very high noise levels (low $t$), and $t_{\\text{interval}}$ simply implements this principle.\n\n2.  **Empirical Robustness (Table 2):** Our ablation study demonstrates a wide effective range, not a single brittle value.\n- Even a small cutoff at $t_{\\text{interval}} = 0.1$ substantially improves performance ($\\text{FID}_{N=1}$ drops from 9.62 to 8.58), showing the primary benefit is avoiding guidance in the earliest, highest-noise steps.\n- Performance degrades only when the cutoff is overly aggressive (e.g., $t_{\\text{interval}} = 0.5$), confirming a **stable optimal window** rather than a specific magic number.\n- Furthermore, we have trained a new model (different architecture, FlowDCN) and new dataset (ImageNet 512x512) with the same $t_{\\text{interval}}=0.3$ and the performance is still very competitive (in Rebuttal Table 1)\n\n3.  **Consistency with Prior Work:** Our finding is strongly validated by prior work [1], show this principle is critical for state-of-the-art results on diverse setups like ImageNet-512 and Stable Diffusion XL. Both our work (integrating interval guidance into the training loss) and theirs (applying it at inference) identify a similar optimal range around $t=0.3$.\n\nThis convergence of evidence confirms that $t_{\\text{interval}}=0.3$ is a robust and principled default, not a dataset-specific choice.\n\n__Q6: Additional evaluation metrics (FD-DINOv2, IS, Precision, Recall)__\n\nWe report FD-DINOv2, Inception Score (IS), Precision, and Recall for the baseline Shortcut Model (SM), Inductive Moment Matching (IMM), and our improved Shortcut Model (iSM). The results below show that **iSM consistently outperforms the baseline SM** across multiple NFE settings.\n\n**Rebuttal Table 3: Additional evaluation metrics for ImageNet 256x256.**\n| Model | NFE | FD-DINOv2↓ | IS↑ | Precision↑ | Recall↑ |\n| :--- | :--- | :--- | :--- | :--- | :--- |\n| SM | 1 |  500.92 | 102.66 | 0.68 | 0.46 |\n| IMM | 1 |  247.78 | 128.87 | 0.67 | **0.64** |\n| **iSM (ours)** | 1 |  **232.31** | **223.52** | 0.70 | 0.61 |\n| SM | 2 |  329.53 | 125.66 | **0.77** | 0.47 |\n| IMM | 2 | 152.08 | 173.66 | 0.74 | **0.64** |\n| **iSM (ours)** | 2 | **107.63** | **302.29** | **0.77** | 0.61 |\n| SM | 4 | 265.90 | 136.79 | **0.80** | 0.48 |\n| IMM | 4 | 110.88 | 204.95 | 0.77 | 0.62 |\n| **iSM (ours)** | 4 | **83.70** | **298.23** | 0.77 | **0.62** |\n\n__Q7: Regarding Formatting Concerns:__\n\nThank you for these helpful suggestions. We will them in the revision: \n*   We will cite **REPA [3]** in our component-wise analysis section as an inspiration for the evaluation structure.\n*   We will reposition **Table 2** to the top of Page 9 for improved readability and flow.\n\n[1] Applying Guidance in a Limited Interval Improves\nSample and Distribution Quality in Diffusion Models, NeurIPS 2024.\n\n[2] Diffusion-4K: Ultra-High-Resolution Image Synthesis with Latent Diffusion Models, CVPR 2025.\n\n[3] Representation alignment for generation: Training diffusion transformers is easier than you think, ICLR 2025."}, {"review_id": "ABmG4Ju9vr", "reviewer": "Reviewer_r2K6", "summary": "This paper introduces a novel framework called the Improved Shortcut Model (iSM), which aims to overcome the limitations of existing shortcut generative models. While these shortcut models offer efficient few-step generation, they often suffer from issues like fixed guidance strength, high training variance, and a bias towards low-frequency details, which can compromise flexibility, stability, and sample quality. iSM addresses these challenges through several key innovations: it incorporates dynamic guidance by making guidance strength a model input and optimizing its application; it employs a multi-level wavelet loss to better capture crucial high-frequency details; it utilizes scaled optimal transport matching to stabilize training and straighten the generation path; and finally, it implements a dual EMA (Exponential Moving Average) strategy to effectively tackle target lag issues that arise during training.", "questions": "- Can the authors provide scaling law experiments for larger models?\n- Can the authors add experiments comparing their method with SOTA models across 1 to 50 steps?\n- Can the authors provide the complete training resources and more detailed, reproducible experimental steps?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "### **Summary Of Strengths:**\n\n- The paper's core contribution lies in its proposed iSM framework's ability to significantly improve the FID scores of shortcut models when generating images in single-step or few-step (e.g., 4-step) settings. This means it achieves better image quality while maintaining high efficiency, effectively solving the performance degradation issue that shortcut models typically face at low sampling steps.\n- By introducing Integrated Guidance and interval guidance, the model allows users to more flexibly adjust guidance strength during inference to balance diversity and fidelity without retraining the model. This overcomes the limitation of previous shortcut models having fixed CFG scales.\n- This work doesn't just tackle a single problem; it systematically addresses multiple critical technical challenges that previously hindered shortcut model performance. It achieves this through various innovative methods, including multi-level wavelet loss, scaled optimal transport matching, and a dual EMA strategy, thereby concurrently resolving issues like frequency bias, high training variance, and self-consistency target lag.\n\n### **Summary Of Weaknesses:**\n\n- Experiments were primarily conducted on ImageNet 256x256. While this is a standard benchmark, a more comprehensive evaluation is needed to understand how the method performs on other types of datasets or at higher image resolutions.\n- The paper lacks a comparison with SOTA multi-step models. This would be beneficial for researchers to understand the performance gap between few-shot and traditional text-to-image models, such as Stable Diffusion 2.5.\n- It appears that the main body of this paper does not include a section on experimental setup, and there is no report on GPU resources in the appendix either. Providing more detailed setup information in the main text would be beneficial for understanding the experiments.", "was_revised": true, "final_justification": "The author has resolved my concerns regarding the model's scalability and reproducibility. I am very interested in the code, which the authors plan to open-source in the future.", "rebuttal": "Thank you for recognizing the core strengths of our work, including the significant FID improvements, enhanced flexibility, and the systematic approach to solving multiple challenges. We appreciate your suggestions for improvement.\n\n**Note**: References to tables, figures, or propositions without the 'Rebuttal' prefix refer to content located in the main paper.\n\n__Q1: ImageNet 512x512 and different backbone__\n\nOur work introduces a robust, architecture-agnostic training methodology for shortcut models. While we report results on SiT-XL/2 to align with the Shortcut Model baseline, our technique generalizes beyond Transformer-based models. Due to limited time, we tackle both backbone and larger image size within one experiment. Specifically, we **scaled our approach to ImageNet 512×512 using XL/2 variant of FlowDCN**, a fully convolutional generative model with group-wise deformable convolution blocks instead of a DiT-based model. Given the rebuttal time, our models have not been fully trained, and we report their scores at the latest iteration (300K) in the table below. We achieved a substantial FID improvement compared to the original shortcut model training, demonstrating both **scalability and broad applicability**.\n\n**Rebuttal Table 1: ImageNet 512x512 experiment.**\n| Model (FlowDCN) | NFE | #Iter | FID↓  | Precision↑ | Recall↑ |\n| --------------- | --- | ----- | -----| ---------- | ------- |\n| SM              | 1   | 300K  | 43.81  | 0.56       | 0.11    |\n| **iSM (ours)**      | 1   | 300K  | **37.05** | **0.60**        | **0.55**    |\n| SM              | 4   | 300K  | 12.16 | **0.86**       | 0.19    |\n| **iSM (ours)**      | 4   | 300K  | **9.94** | 0.78       | **0.62**    |\n\n__Q2: Text-to-image and compare multi-step model__\n\nWe thank the reviewer for pointing out the value of a broader multi-step comparison.  In fact, in Table 1 we compared iSM against the recent multi-step class-conditioned diffusion and flow models on ImageNet 256×256 (e.g. DiT-XL/2 (250 steps), SiT-XL/2 (250 steps), U-ViT-H (50 steps), SimDiff (512 steps),...).\n\nRegarding text-to-image (T2I) models such as Stable Diffusion, we note the following:\n  1.  Our work is scoped to class-conditioned ImageNet generation (as do prior methods like DiT, SiT, SM, IMM).\n  2.  Extending to T2I would require both a text encoder and retraining on large scale dataset like LAION datasets, far beyond our current compute budget.\n  3.  Unlike many text-to-image models, most class-conditional methods do not report, or perform poorly at, the 50-step sampling budget. To our knowledge, U-ViT-H is the only class-to-image model with competitive FID at 50 steps, and we included its result in Table 1 for direct comparison.\n  4.  Architecturally, we build on DiT-based backbones, which have recently been demonstrated (e.g., in PixArt-Alpha) to scale effectively to text conditioning. We therefore believe our improvements would transfer directly to the T2I setting.\n\nIn summary, Table 1 shows iSM competitive performance with respect to SOTA class-conditional generative models on ImageNet, and while a direct text-to-image model comparison is an exciting future direction, it lies outside our current class-to-image benchmark and compute resources.\n\n__Q3: Scaling law experiments for iSM__\n\nDue to limited time of rebuttal and computational budget, we couldn't scale up the model to a larger size than DiT-XL, hence we follow SM to run the scaling law experiment from smaller model (DiT-B) to our final model (DiT-XL) as a proxy for larger-scale behavior. The results in Rebuttal Table 2 clearly show that the performance gap between our iSM and the baseline SM widens significantly as model size increases, suggesting strong scaling properties.\n \n**Rebuttal Table 2: Scaling law experiment in ImageNet 256x256.**\n| Backbone   | #Params (M) | NFE | FID (SM) ↓ | FID (iSM) ↓ |\n|------------|-------------|-----|------------|------------ |\n|   DiT-B    | 130         |  1  |    40.31    |   **22.74** |\n|   DiT-B    | 130         |  4  |    28.32    |   **8.69**  |\n|   DiT-XL   | 675         |  1  |    10.60    |   **5.27**  |\n|   DiT-XL   | 675         |  4  |    7.81    |   **2.05**  |\n\n\n__Q4: Provide complete training resources and reproducible steps?__\n\nYes. We have already included our implementation details in the Appendix and a checklist in the main paper. All experimental workflows, including training and evaluation, were conducted on 8 x NVIDIA A100-40GB GPUs. We are **committed to open-sourcing our code and releasing model checkpoints** upon acceptance to ensure full reproducibility for the community."}]

Author rebuttal:

Meta-review: This paper analyzes and addresses several shortcomings of shortcut models, generative models that allow one- or few-step sampling.

Strengths:

- The paper addresses an important problem (reviewers DUtt, MiHU, r2K6).
- The paper analyses and proposes/formalizes novel solutions to several issues with shortcut models (reviewers xmQN, MiHU, r2K6).
- The experiments confirm the benefits of the proposed approach.

Weaknesses:

- At submission time, the paper only included ImageNet 256x256 (reviewers xmQN, MiHU, r2K6). Authors added ImageNet 512x512 and additional metrics to existing experiments in the rebuttal.
- This is not the end of the story: More steps are still better (reviewer r2K6), and so are GANs (reviewers DUtt). So it's not the end of the story.

All reviewers agreed that the methods and experimental evidence is significant to warrant acceptance. The authors have committed to open-sourcing code and model checkpoints.
Decision: Accept (poster)