Any-step Generation via N-th Order Recursive Consistent Velocity Field Estimation
Authors: Peng Sun
Tao Lin
Venue: ICLR 2026 Poster (2026)
Keywords: Generative Models
Primary area: generative models

Abstract: Recent advances in few-step generative models (typically $1$-$8$ steps), such as consistency models, have yielded impressive performance. However, their broader adoption is hindered by significant challenges, including substantial computational overhead, the reliance on complex multi-component loss functions, and intricate multi-stage training strategies that lack end-to-end simplicity. These limitations impede their scalability and stability, especially when applied to large-scale models.

To address these issues, we introduce **$N$-th order Recursive Consistent velocity field estimation for Generative Modeling (RCGM)**, a novel framework that unifies many existing approaches. Within this framework, we reveal that conventional one-step methods, such as consistency and MeanFlow models, are special cases of 1st-order RCGM. This insight enables a natural extension to higher-order scenarios ($N \geq 2$), which exhibit markedly improved training stability and achieve state-of-the-art (SOTA) performance.

For instance, on ImageNet $256\times256$, RCGM enables a $675\text{M}$ parameter diffusion transformer to achieve a $1.48$ FID score in just $2$ sampling steps. Crucially, RCGM facilitates the stable full-parameter training of a large-scale ($20\textrm{B}$) unified multi-modal model, attaining a $0.86$ GenEval score in $2$ steps. In contrast, conventional 1st-order approaches, such as consistency and MeanFlow models, typically suffer from training instability, model collapse, or memory constraints under comparable settings.

Code is available at: https://github.com/LINs-lab/RCGM.

Published as a conference paper at ICLR 2026
ANY-STEP GENERATION VIA N-TH ORDER RECURSIVE CONSISTENT VELOCITY FIELD ESTIMATION
Peng Sun 1.2
Tao Lin
'Zhejiang University Westlake University
sunpengewest Lake edu. en, Lintad@westlake.edd.c
ABSTRACT
Recent advances in few-step generative models (typically 1-8 steps), such as consistency models, have yielded impressive performance. However, their broader adoption is hindered by significant challenges, including substantial computational overhead, the reliance on complex multi nulli-comppnent loss functions, and intricate multi-stage training strategies that lack end-to-end simplicity. These limitations impede their scalability and stability, especially when applied to large-scale models. To address these issues, we introduce N-th order Recursive Consistent velocity field estimation for Generative Modeling (RCGM), a novel framework that unifies many existing approaches. Within this framework, we reveal that conventional one-step methods, such as consistency and MeanFlow models, are special cases of Ist-order RCGM. This insight enables a natural extension to higher-order scenarios (N 2 2), which exhibit markedly improved training stability and achieve state- of-the-art (SOTA) performance. For instance, on ImageNet 256 x 256, RCGM enables a 675M parameter diffusion transformer to achieve a 1.48 FID score in just 2 sampling steps. Crucially, RCGM facilitates the stable full-parameter training a large-scale (20B) unified multi-modal model, attaining a 0.86 GenEval score in steps. In contrast, conventional Ist-order approaches, such consistency and McanFlow models, typically suffer from training instability, model collapse, memory constraints under comparable settings. Code available at: ht https: / / thub. com/ LINs- /RCG.
INTRODUCTION
Existing PF-ODE-based generative models (Song et al., 2020b), encompassing diffusion models (Ho et al., 2020; Song et al., 2020a), flow-matching models (Lipman al., 2022; Ma et al., 2024), and consistency models (Song et al., 2023; Lu & Song, 2024). have demonstrated re- markable success in synthesizing high-fidelity data across diverse applications, including image and video genera- tion (Google, 2025a; OpenAI, 2025; Xie et al., 2024a; Ho et al., 2022: Chen et al., 2025c; Wu et al., 2025a).
Table 1: Comparison of different methods' reliance on a Ist-order objective and JVP.
Our method is independent of both.
Beyond
Method
1st Order JVP
CM (Song et al., 2023)
x
sCM (Lu Song, 2024)
MeunFlow (Geng al., 2025)
RCGM (Ours)
Within this landscape, few-step generative models (Song et al., 2023; Frans et al., 2024; Geng et al., 2025) are particularly prized for their ability to generate high-quality samples with significantly reduced computational cost, critical factor for practical deployment. However, the pursuit of this efficiency has introduced a distinct set of formidable challenges that plague current SOTA methods: (a) prohibitive computational and memory burden during training, often necessitating expensive Jacobian- vector products (JVP) (Geng et al., 2025; Lu & Song, 2024); (b) the need to combine multi- ple losses and train auxiliary models, c.g., combining consistency loss with adversarial loss (Chen et al., 2025c) or training an additional fake image generation model (Yin et al., 2024b;a; Sauer et al., 2024a); (c) fractured theoretical landscape, where highly related methods like consistency models (Song et al., 2023), shorteut models (Frans et al., 2024), and MeanFlow (Geng et al., 2025) have been developed in isolation, lacking a common theoretical foundation.
"Corresponding author.

---

Published as a conference paper at ICLR 2026
Few-stepGeneration RCGM
Simple & EaSY
AS
Figure 1: Visualization results RCGM on Dwen-Immage-20B. The images shown were generated by RCGM-tuned Aween-Image-20B model using NFE=8 (GenEval score=0.87). Please zoom in to see finer details.
These challenges restrict their broader application, particularly in generalizing large-scale models with guaranteed stability and efficiency. For instance, our experiments show that existing one-step models, such as consistency models, often suffer from training instability and high computational demands when scaled up, frequently resulting in model collapse or GPU memory exhaustion (see Tab. 4). We argue that this fragility fundamentally stems from their implicit reliance a 1st-order recursive training objective (cf. Sec. 2 and Sec. 3). Crucially, while these few-step models rely heavily on self-targeted learning, their Ist-order recursive nature is inherently incompatible with Exponential Moving Average (EMA) techniques. Consequently, they are deprived of the stable training targets and smoothed optimization dynamics that EMA typically provides, creating profound bottleneck for scaling. This critical insight leads us to the central question:
Can we develop unified and simple framework that: (a) encompasses existing few-step generative models as a special case; (b) enhances training stability and generalization to large-scale models by moving beyond the 1st-order limitation -tthrry unlocking compatibility with stabilizing techniques like EMA and obviating the need for JVP or training auxiliary models?
Problem 1
To address these challenges, propose RCGM, novel and principled framework that unifies and generalizes existing approaches. Within our framework, we show that conventional one-step models (e.g., consistency models and MeanFlow) correspond to the special case of Ist-order RCGM.
Notably, RCGM naturally supports higher-order formulations (i.c.. N > 2). These higher-order variants utilize more comprehensive trajectory information from the PF-ODE, which contributes to substantially improved training stability. This stability enables successful training in demanding large-scale settings where 1st-order models often fail, ultimately achieving SOTA performance without resorting to complex workarounds. In summary, our contributions are:
(a) We propose RCGM, unified framework that contextualizes existing few-step generative models as a specific sst-order case and generalizes them to arbitrary N-th order formulations. (b) We identify the EMA incompatibilityy paradox inherent in 1st-order self-targeted learning. We empirically verify that higher-order RCGM (e.g., 2nd-order) elegantly breaks this paradox, successfully harnessing the stabilizing benefits of aggressive EMA smoothing to exhibit superior robustness and enabling effective scaling to larger, more complex model architectures (cf. Sec. 4). (c) Our method achieves SOTA performance across a range of standard benchmarks, outperforming existing methods in few-step generation tasks while maintaining computational efficiency.
Supported by Fig. 1. Sec. 4, and App. D, our approach matches or outperforms SOTA methods across diverse datasets, architectures, and resolutions, advancing efficient, high-fidelity generation.
2

---

Published as a conference paper at ICLR 2026
2 PRELIMINARIES
Let p(x) be the data distribution for a given training set D. This distribution can also be conditional, denoted p(x|c) for given condition Diffusion-based generative models aim learn transfor- mation from simple prior distribution p(z), typically the standard Gaussian ( I), to the complex target data distribution p(x).
This often achieved by learning to reverse a forward noising process. The forward process gradually perturbs a clean data sample x ~ p(x) into a noisy intermediate sample X2 using predefined trajectory, such as X = at)) + 7t)x, where ~ (0) The time variable t spans the interval [0, 1], with the perturbation effect intensifying as increases. The scheduling functions (t)) and (t) are continuously differentiable, i.e., a(f), (t) E C* [0. 1], and satisfy the boundary conditions: (0) = 0, r(0) (yielding the data) and (<() = 1, 7(1) = (yielding pure noise).
More formally, diffusion models learn function that guides the transformation of samples along the trajectory the Probability Flow Ordinary Differential Equation (PF-ODE) (Song et al., 2020b), which connects the prior distribution p(z) to the data distribution p(x).
In this paper, we define a general prediction function -f(x, := x, X₁ that estimates the displacement from x; to target X,, with further details in (6). This function aims to predict the target point x, from the current point X; along a specific PF-ODE trajectory. In the following sections, we will introduce several prominent learning paradigms for deep generative models.
2.1 OTH-ORDER: DIFFUSION AND FLOW-MATCHING MODELS
Diffusion and Flow-Matching Models (Ho et al., 2020; Song et al., 2020b; Lipman et al., 2022; Sun et al., 2025). Recent work by Sun al. (Sun et al., 2025) established a unified framework for diffusion and flow-matching models. This framework reveals that both paradigms aim to learn the same PF-ODE (1), but they differ their underlying transport processes (i.e., their specific choices of (() and 7()) and training objectives.
Specifically, a neural network Fo is trained by minimizing a general objective of the form: Ex., [d (F (xx), á(t)z + (t)))),, where d(. denotes a distance metric. As derived in (Sun et al., 2025), the output of the trained network, F, := Fx can be used to construct the component a(t)-F.-ā(()-- and f" Xx, () = -( x(t))x.--(1)-PP. These com- a(())))))
functions: (F,,,, xx, () a()-()-(0))(
ponents, in turn, define the velocity field of the PF-ODE: dx.
da(() fF x1, + dr(0) de -d
(Fr,,,, Xe, f). The sampling process then involves numerically integrating this velocity field to solve the PF-ODE. The integration proceeds backward time, starting from a prior sample x1 ~ p(z) at t= 1 and ending t 0 to produce data sample from p(x).
Within our framework, we adopt a zeroth-order inductive learning perspective to interpret this process, view supported by Fig. 3 (a). Specifically, for sufficiently small step At, the prediction function's learning target becomes the product of the velocity field and the time step:
dx
f(x,, At)
At
as
At
0.
In essence, given the current state x;, the prediction function f directly learns to predict the displace- ment required to approximate the next state, X;-Ar, on the PF-ODE path.
2.2 1ST-ORDER: RECURSIVE CONSISTENCY MODELS
Consistency Models (Song et al., 2023; Lu & Song, 2024; Sun et al., 2025). Consistency models designed bypass the iterative nature of diffusion models. Their primary goal is to learn function that maps any noisy state x, directly to the clean data endpoint x, in single step. This is achieved by estimating the endpoint of the PF-ODE trajectory originating from x,, using the function Xy = fz (F,,, x,,
The training objective is specifically designed to instill crucial "consistency" property. This property ensures coherence between the model's predictions for the clean data, even when originating from two temporally adjacent noisy states that separated by finite time interval A/.
3

---

Published as a conference paper at ICLR 2026
EMA Decay; 0.0 MMD: 0.0066
EMA Decay; 0.9 MMD: 0.0081
ENL Decay; 0.99 MMD: 0.0088
EMA Decay: 0.999 MMD: 0.2131
Figure 2: The brittleness of Ist-order models under aggressive EMA decay rate (k).. We evaluate representative consistency-basee model (Song et al., 2023) on the Moons dataset. Gray areas: Ground truth; Red dots: One-step samples. We adopt Maximum Mean Discrepancy (MMD) as non parametric metric to measure the divergence hetween the generated and real distributions. As approaches unity (c.g.. 0.999), we observe a significant degradation in sample quality (indicated by a higher MMD and poor visual fidelity), suggesting fundamental incompatibility between Ist-order recursion and aggressive temporal smoothing.
Ex., [d (f Xx, t),
(F X, At)))] A known limitation of discrete- time consistency models is their sensitivity to the choice of At, which often requires manually tuned annealing schedules for efficient training (Song & Dhariwal, 2023; Geng et al., 2024). This challenge was later addressed by continuous consistency models, which derive their training objective by taking the limit as At + 0 (Lu & Song. 2024).
We interpret this process through 1st-order inductive learning lens, perspective supported by visualizations in Fig. 3 and theoretical analysis in App. E. This view frames the learning objective as recursive formulation:
dx. f (x,, 0) dt
At
This recursive principle-ayy apprrimmaiiooiiinninnnngao a long-range range prediction by combining an infinitesimal step with another long-range prediction is also reflected follow-up works (Frans et al., 2024; Geng et al., 2025). For instance, shortcut models (Frans et al., 2024) employ similar self-recursive formulation and generalize it to predict between arbitrary time points t and r E [0, t]: f(x, r) <- f(x,, + r). This is then combined with a flow-matching objective train one-step generative models. More recently, MeanFFlo (Geng et al., 2025) extended this idea by training a one-step model with the recursive objective f(x;,r) < dx.
A + f(x; At, r) for any target time r.
(ixt-At,")
di
In summary, while diffusion and flow-matching models inherently multi-step frameworks, consis- tency models represent a paradigm shift towards few-step or one-step generation.
The Exponential Moving Average (EMA) incompatibility paradox. Standard practice in self- targeted learning (Grill et al., 2020; He et al., 2020; Caron et al., 2021; Song et al.. 2023) involves maintaining an EMA version of the model, fo- to provide stable regression targets for the online model fa. Intuitively, one would expect high EMA decay rates (e.g., K = 0.999) to enhance training stability by filtering out optimization noise.
However, we identify a fundamental incompatibility within existing 1st-order consistency-base based models: the recursive formulation inherent to Ist-order objectives actively conflicts with the delayed updates of an EMA model, resulting in substantial performance degradation at high decay rates.
To systematically investigate this phenomenon, Fig. details an empirical analysis of how varying the EMA decay rate alters the training dynamics of 1st-order models, with large-scale evaluations provided Sec. 4.2. Our findings reveal an acute sensitivity to the decay hyperparameter. Specifically, aggressive decay rates (e.g., 0.999) severely compromise the fidelity of one-step generation, whereas relaxed rates (e.g., 0.9) fail to yield meaningful improvements over naive, non-EM EMMA telf-targe
3 METHODOLOOGY
We begin by deriving a recursive, N-th order velocity field estimator through the segmented integra- tion of the Probability Flow ODE (PF-ODE) trajectory (Sec. 3.1). Building on this formulation, introduce a unified training objective that enables any-step generation (Sec, 3.2). Finally, we discuss key practical considerations for implementing our method, RCGM (Sec. 3.3).
4

---

Published as a conference paper at ICLR 2026
ass
Finer Ad)
Bt:
1( As, 9)
dxa At dt
(e):
lbx.0)
N x internls
12g
At Ioa.t dsc
At Iic-ann 0) 00
f (x11)N++))
1 N++1
66
s(81:3)
fizs awr))
1
f(sue)
dxg
1m
thu
220
At
+
f , bn+1) + (xe, tN+1)
At J(ie-aii)) J(hee)
dt
Figure 3: A conceptual illustration of our proposed framework, RCGM, which generalizes existing generative models by formulating them within a unified higher-order structure. Trajectories map a current state target learning state. (a) Standard diffussin (Ho et al., 2020) and flow-machhing (Lipman al., 2022) models correspond to the Oth-order case (N 0) of our framework. (b-d) Prominent one-step models, including consistency models (Song et al., 2023), MeanFlow (Geng et al., 2025), and shortcut models (Frans et al., 2024), are special instances of the 1st-order case (N = 1). (e) RCGM extends this hierarchy to arbitrary orders (N 2 0), enabling the use of higher-order information for potentially more robust training dynamics.
3.1
SEGMENTED INTEGRATION ALONG THE PF-ODE TRAJECTORY
Our methodology is grounded in the PF-ODE formulation, where a trajectory from prior distribution the data distribution is defined by velocity field v(), For a diffusion process specified by X, a(l)z + (()xo, this velocity is given by (Song et al., 2020b; Sun et al., 2025):
((() x, a(T))a(T)) (x,, r) 7(T)
2((7) 2'T) a(7) T(T)
Xx lo-p,(x.).
(I)
The integral form of this ODE connects any two points X, and XtN+1 on a trajectory. We proceed by partitioning the integration interval [t. [N+1] with N intermediate points, where = > >> ... > 1N+1- This segmentation allows us to decompose the total displacement into a sum over the sub-intervals:
T)r l" v(x,, + Σ
Xtx+1
X
T)d,
(2)
The core of our approach is to approximate the first integral segment. For a sufficiently small time step At == to t1, a Ist-order Taylor approximation (i.e., a forward Euler step) is justified:
dxi
v(x,, -v(xts ())
AL.
dl.
Substituting this approximation into the exact identity from (2) yields the relationship:
dx di
tinl
X(
A
7(x,,
By rearranging (4), we obtain our final estimator. We define the recursive N- order velocity field estimation as the target derived from this multi-step formula:
N
dx:
1 A1
(XtN11 x.)
T)d7
dt
This formulation is termed recursive because the estimation of the velocity v at time depends on the integral of the same velocity field over future time steps. The "N-th order" designation refers to the N integral correction terms that refine the estimate beyond simple one-step approximation, thereby providing more accurate target for model training.
3.2 A UNIFIED TRAINING FRAMEWORK FOR ANY-STEP GENERATION
Our goal is to train an any-step generative model capable of predicting the state x, at any future time from the current state x, along a given PF-ODE trajectory. To this end, we define
5

---

Published as a conference paper at ICLR 2026
EMA Decay: 0.0 MMD: 0.0052
EMA Decay; 0.9 MMD: 0.0019
EMA Decay: 0.99 MMD: 0.0037
EMA Decay: 0.999
MMD: 0.0035
Figure 4: Robustness of 2nd-order models across varying EMA decay rates (k), We evaluate RCGM configured with decay rates (e.g.. n = 0.999) that caused significant degradation in Ist-order models, we observe that sample quality remains consistently high across all config configurations. This is evidenced by low MMD values and visually faithful generations. This empirical evidence strongly supports our hypothesis that higher-order formulations enahle effective training under aggressive temporal smoothing in self-targeted learning, therehy mitigating the associated instability issues.
under identical conditions to Fig. 2. Even when employing the aggressive EMA
displacement function rx), that maps the current state to the total displacement required to reach the target state: f(xx, r) == X, X₁ = J. ((x,T))dr, r re [0,
(6)
Using this definition, we can reformulate the recursive N-th order velocity estimator from (5) entirely in terms of displacements:
N Ext, t--1)
dx,
1 At
f(x))tNA))
dt
This identity forms the foundation of our training objective. It provides a multi-step target for the instantaneous velocity dx,/dl, which is known analytically from the PF-ODE formulation (ef. (1)).
We parameterize the displacement function with parameterized model f (x,,r) To train fe- we enforce the identity in (7). The terms f(x,; (4+1) for i 2 represent future displacements and treated as fixed targets during optimization. Following standard practice in consistency training (Song al., 2023), we use a target model fe (e.g., an exponential moving average of 0 periodically updated copy) for these terms, applying stop-gradient operator to prevent backpropagation through them. This yields the following learning objective:
dx, dt
(())= x0.2, (20)
fo fo(xx - ESo fo (xt,, t-
(8)
At
True Velocity
Model's Velocity Estimate
where x2 = attz (t)xx with z ~ (N...) time points are sampled hierarchically (e.g., t m (0, TJ, 4, 10, etc.), and .(...) a suitable metric, such as the squared fo-norm.
This unified formulation elegantly generalizes several established generative modeling paradigms:
(a) For N 0, the objective simplifies matching dx//dt with f (x,, 1,) At. This equivalent to the objectives used in score-based diffusion models (Song et al., 2020b) and flow match- ing (Lipman et al., 2022).
(b) For N = 1, the objective corresponds to those of one-step consistency models (Song et al., 2023; Lu & Song, 2024) and shortcut- based methods (Frans et al., 2024), which use a single future segment to define the training target.
By extending this framework to higher orders (N 2 2), our approach leverages multiple future steps to construct more robust and stable training signal. As demonstrate in our experiments (see Fig. 4 and Sec. 4), this generalization improves model performance and convergence across diverse generation tasks.
Notably, regardless of the setting of N, our training objective requires only a single model forward pass with gradient calculation and N forward passes without. This design avoids increased GPU

---

Published as a conference paper at ICLR 2026
memory costs during training, making it feasible for large-scale models A detailed discussion the setting of is provided in Sec. 4.2.
3.3 PRACTICAL IMPLEMENTATION OF RCGM
In this section, we detail several key aspects of the practical implementation of our method, RCGM. We discuss the parameterization of our neural network under a linear transport path, the use of an enhanced target score function to improve performance, the strategy for conditioning the model on both input and target times, and the formulation of a practical loss function for stable and effective training.
Linear transport and network parameterization. We employ the linear transport path common in flow-matching literature (Lipman et al., 2022; Ma et al., 2024; Xie et al., 2024a), defined by coefficients att) = t and (t) = 1 - t. This transport corresponds to a constant velocity field, implying that the displacement between any two states x and x, directly proportional to the time difference t - r. This property motivates our parameterization of the predictive function fx,,,t which estimates the displacement from x; to X,, as: fe((xx is a neural network designed to approximate the average displacement (x; X,, xt) r).
[x, 1, r) = Fo r) r), where Fo
Enhanced target score function. The performance of continuous generative models can be sig- nificantly improved by incorporating guidance during the training or sampling process (Ho & Salimans, 2022; Dhariwal Nichol, 2021; Karras et al., 2022). This achieved by modifying the conditional target score function from Vx, log P((x.) (defined in (1)) to enhanced version:
Vx, log Pt (xx/c) (Pe.o (x)p..x) where Ç E (0, 1) is the enhancement ratio. We follow the same implementation as previous studies (Frans et al., 2024; Sun et al., 2025).
Input time conditioning. Our method learns continuous-time model, fg(x,, r), designed to predict the state x, at target time r from an initial state x, along the probability flow ODE (PF- ODE) trajectory. To accurately map between arbitrary time points, the model must be effectively conditioned on both the input time and the target time r. Following standard practice (Ho et al., 2020; Frans et al., 2024), employ time embedding technique where and r embedded into vector representations separately. These embeddings, along with the input xt. are then fed into the neural network Fe, redefining the model S (x,, r) (x 4, r) ((-r). r).
Practical loss design. The training objective in (8) necessitates a carefully designed loss function. While the fz-norm is standard choice for the metric d(.,.) (Ho al., 2020; Song et al., 2020a), directly optimizing the original objective is suboptimal. We observe that the magnitude of the model output, f r)) z Fa- (t (6-7% r), scales linearly with the time interval implicit, scale-dependent weighting that causes optimization instability, as larger time steps dominate the gradients. To rectify this bias and stabilize training. we employ a ceraance-reduction technique inspired by Lu & Song (2024). Specifically, by leveraging the gradient identity derived in previous
r). This introduces an
studies (Lu & Song. 2024; Sun et al., 2025) (i.e., V.E(F] y) |oE Fo F + y v)()), we reformulate our objective into a regression form that decouples the gradient scale from the time interval. Substituting this into (8) yields our final training objective:
[[ t,tN+1) F. (s,,t,tN+) + E(x,, (t. (tal (())= = =Exz 1
(9)
where the target item is (xx, (t,) (tt) de [ (x,, - -N fo (xt,
dx. de
EXPERIMENTS
This section presents the experimental validation of our proposed methodology, denoted as RCGM. We begin by outlining the experimental setup, including datasets, network architectures, and imple-
'Tis is a significant advantage over conventional few-step training methods that often rely on Jacobian- vector products (JVP), which can substantially increase GPU memory consumption (Geng et al., 2025; Lu & Song, 2024). Furthermore, the use of JVP can introduce complex technical challenges when integrating with widely-used architectural optimizations like Flash-Attention (Dao al., 2022).
7

---

Published as a conference paper at ICLR 2026
mentation details. We then present comprehensive evaluation of RCGM's performance. Theoret- ically, our approach converges to conventional now-matching and diffusion models when N = 0. Consequently, to rigorously assess the unique contributions of RCGM, our empirical investigation focuses on the regime where 2 1.
4.1 EXPERIMENTAL SETUP
Datasets. Our primary evaluation is conducted on the ImageNet- IK dataset (Deng et al., 2009), utilizzig resolutions of 256 x 256 and 512 512. This choice aligns with established benchmarks in recent high-fidelity generative modeling literature (Karras et al., 2024; Song et al., 2023). We adopt the data preprocessing pipeline from ADM (Dhariwal & Nichol, 2021) to ensure consistency and comparability with prior work. All experiments performed in the latent space of pretrained autoencoders, a standard practice for efficient training of large-scale models. Specifically:
(a) For 256 256 images, we leverage widely adopted autoencoders, including the SI~VAE (Rom- bach et al., 2022) and the VA-VAE (Yao et al., 2025).
(b) For 512 x 512 images, in addition to the SD-VAE, we employ a DC-AE (Chen al., 2024c) with a higher compression ratio (f32c32) to mitigate computational demands.
Network architectures. We build upon the success of transformer-based architectures for generative modeling. Our core model is 07ra-eerre Diffusion Transformer (DiT) (Peebles Xie, 2023), a backbone widely employed in SOTA models such as SiT (Ma et al., 2024), Lightening-DiT (Yao et al., 2025), and DDT (Wang et al., 2025a).
Implementation details. Our models are implemented in PyTorch (Paszke, 2019) and trained using the AdamW optimizer (Loshchilov & Hutter, 2017) with 31 0.9, 32 = 0.95, constant learning rate of 2 x 10 10-4, and a batch size 1024. Unless otherwise specified, we train all models using a 2nd-order scheme (N = 2). The only exception is during the final 40 epochs of training on ImageNet-I KK, where we employ central difference consistency training (Sun et al., 2025) (which can be viewed special case of 2nd-order RCGM) to achieve further performance gains. For the time distribution during training, we follow the exact configurations used in Sun et al. (2025).
To evaluate the quality of generated samples, we adhere standard protocols established in the literature (Song et al., 2020b; Ho et al., 2020; Lipman et al., 2022; Brock et al., 2018). Our primary metric is the Fréchet Inception Distance (FID) (Heusel al., 2017), computed using 50, 000 generated samples (FID-50K) against the training set.
4.2 ANALYSIS OF HIGHER-ORDER TRAINING
It is widely known that training few-step models is challenging due to the instability of training (Song et al., 2023; Lu & Song, 2024), especially when using large model and a large learning rate, etc.
This issue is more severe when training few-step models in real-world applications such high- resolution text-to-image generation.
Using Exponential Moving Average (EMA) model in (8) is key technique help stabilize training and improve performance, which also evidenced in previous Ist-order methods (Song et al., 2023). For those without using EMA model, they typically require careful technical design to stabilize training. e.g., using JVP (Lu & Song, 2024) or careful hyperparameter design (Song & Dhariwal, 2023).
In this section, investigate how the order N in RCGM affects the training stability and performance under different EMA decay rates we conduct a series of ablation studies on ImageNet- 1K 256 x 256 using 675M diffusion transformer with SD-VAL VAE.
Iarge EMA decay rate x critical for 1st-order training stability. We first investigate the effect of the EMA decay rate from (8) on the stability and performance of the conventional Ist-order (N = 1) model. As illustrated in Fig. 5a, training without EMA (K = 0) is highly unstable, causing the FID score to fluctuate and fail to converge. A small decay rate (K 0.9) tempers this instability, leading to a smoother decrease in FID, yet the final performance remains suboptimal (FID of 31.70). Conversely, while large decay rates (Ns (0.99, 0.999)) effectively stabilize training dynamics, they
8

---

Published as a conference paper at ICLR 2026
70-
122
128
64
FID-50K
FID-50K
64
40
k=0.0
K= 0.0
340
K= 0.9
0.9
0.99
20
16-
0.999
16-
0.999
10
0.1
0.2 0.3 0.4 0.5 0.6
0.7 0,5
0.9
1.0
0.1
02
0.3 04 0.5
0.6
0.7
0.8
0.9
1.0
Traing Steps (K)
Traing Steps (K)
N-2
N-3
N-5
(a) Ist-order over training steps.
(b) 2nd-order over training steps.
(c) Various orders (x 0.999).
Figure 5: Ablation studies of RCGM ImageNet-1K x These studies evaluate key factors of the proposed RCGM training few-step models, i.c., the order of RCGM (N) and the EMA decay rate x. The sampling is performed using one step (I-NFE).
severely hinder convergence. This "over-stabilization" is particularly pronounced at = 0.999, where the model converges to a poor FID of 294.18. These results reveal a fundamental tension between training stability and model performance in the Ist-order setting.
Higher-order approximations resolve the stability-performancc trade-off. We next examine
whether higher-order approximations can alleviate the aforementioned tension. Fig. 5b shows the results for our second-order model (N = 2). Strikingly, the second-order model thrives under the large EMA decay rates that were detrimental in the Ist-order case. While the no- EM- (N 0) setting remains unstable and the low-EMA (K- 0.9) setting achieves modest FID of 14.94, the high-EMA regime is transformed. With N = 2, a large decay rate such as K = 0.99 no longer cripples performance but instead yields a competitive FID of 29.13 This demonstrates that higher-order models possess substantially greater robustness to the choice of K, enabling them to benefit from strong EMA stabilization without sacrificing generative quality.
Further experiments, shown in Fig. 5c, confirm that increasing the order N (with a fixed, high 0.999) can yield additional gains. Performance steadily improves N increases from to 4, which achieves the lowest FID. However, this trend reverses for N > 4, likely due to the accumulation of approximation errors in the higher-order velocity estimates in (8).
In summary, 1st-order models face a difficult trade-off: a large is needed for stability but harms final performance. Higher-order methods effectively resolve this conflict, achieving both stable convergence and strong performance with large K. Considering the balance between computational cost and performance, we adopt N- and K 0.999 our default configuration.
4.3 COMPARISON WITH SOTA FEW-STEP METHODS
As demonstrated in Tab. 2, proposed RCGM, when paired with various autoencoders, consistently outperforms or remains highly competitive with SOTA few-step generative models. The following analysis details its advantages across different VAE architectures.
(a) Performance with SD-VAE (256 x 256 and 512 x 512): When paired with a standard SD- VAE, method exhibits exceptional performance. At 256 x 256 resolution, it achieves an FID of 1.92 with 2 NFEs, outperforming IMM's best result while requiring 8 times fewer sampling steps. At 512 512 resolution, model achieves an FID of 2.25 with NFES, which highly competitive with specialized distillation models like SCD-L (2.04 FID) and sCD-M (2.26 FID), despite their significantly higher training costs (1434 and 1997 epochs vs. our 360). (b) Performance with DC-AE (512 x 512): When integrated with the DC-AE autoencoder, our model achieves a new SOTA FID score of 1.79 with only 2 NFEs. This result surpasses the leading consistency distillation model, sCD-XXL, which records an FID of 1.88 at 2 NFEs. Notably, our method achieves this superior image quality using a significantly more efficient model with only 675M parameters, compared to the 1.5B parameters of sCD-XXL.
(c) Performance with VA-VAE (256 x 256): Using the VA-VAE architecture, our method sets another benchmark, achieving remarkable FID of 1.48 in just 2 NFEs. This represents substantial improvement over the best pert-reeformin distillation method, IMM, which only reaches an FID of 1.99 after much more costly 8 16 NFEs.

---

Published as a conference paper at ICLR 2026
Table 2: System-level quality comparison for few-step generation task on class-conditional ImageNet- IK. The best results of each resolution are highlighted.
512
256
Mothod
NFE
FID
#Params
AEguchs
Method
NFR
FID | AParams
AEguchs
Diffusion & flow-matching Models
ADM-G (Dhuriaal & Nichol, 2021)
250x2
7.72
559M
355
ADM-G (Dhariwal & Nichol, 2021)
250x
4.50
559M
396
U-VIR- (Bao et al. 2025)
50xx
4.05
501M
400
U-VIE-H22 (Bao al. 2023)
50x
2.29
501M
400
DITEXI22 (Pochles& & Xic. 2023)
250x2
3.04
675M
600
DITEXT /2: (Posbles Xic. 2023)
250x2
2.37
675M
1400
XL/X
(Ma al. 2024)
250x2
2.62
675M
600
XLT (Ma et al., 2024))
250x2
2.06
675M
1400
MaskDiT
2023)
79x2
2.50
736M
MDTT
20225
250x2
1.79
675M
1300
EDM2-S (Karras et 2024)
2.56
280M
1678
REPA-XLL2 (YL al., 2024)
250x2
1.5
675M
EDM2-1. (Kamas
20144
3.06
778M
1476
REPA-NI /2, (Yu al. 2024)
250
1.43
500
EDM2 XXL (Karaa et al., 2024)
1.91
1.58
734
LightDiT (Yao el al.. 2025)
250x2
2.11
675M
61
DYF-XL-BUCAAE
250x2
2.41
675M
400
Light. D Yoo et al.. 2025)
250x2
1:35
675M
300
GANy
BigGAN (Brack et al., 2018) StyleGAN (Suner al., 2022)
8:43
160M
BigGAN (Brack ot al., 2018)
6.95
112M
1x2
2.41
168M
GigiGAN (Kang et al., 2023)
3445
568M
Masked autoregresiive usodels
MisGITT (Ching 2022)
7.32
227M
300
MikGIT (Ching 2022)
VAR-d56-s (Tiaa 21124)
10x2
2.63
2.38
350
VAR-d50-i (Tina et al., 20144
10x2
1.73
2,08
350
Ist-order cunsidennc training & distillation
#CT-M (I.s Song. 20124)
5.84
408M
1837
Shorcun-XLL/2 (Prass et al. 2024)
10.6
676M
250
5.53
498M
1837
7:80
676M
250
(TT- & Song. 2024)
5.15
778M
1274
IMM-XL/2 (7hon et ol. 2025)
1x2
7.77
675M
3840
4.65
778M
1274
2x2
5.33
675M
3810
SCT-XXL (Li Soug. 20124)
4.29
3.66
675M
3540
3.76
2.77
675M
KCD-M (1ar Song. 3024)
2.75
408M
1997
IMM (w) 1.5)
1x2
800
675M
3540
2.26
198M
1997
2x2
3.99
675M
38.00
#CIAL (Lu & Song. 2024)
2:55
778M
1454
4x2
2:51
675M
3840
2.04
778M
1434
8x2
1.99
675M
5840
4CT&XXI- (Iu & Song. 2024)
2.28
1.58
921
MensFlow-XX
Xl.2 /2 (Gesg at al, 2025)
3.41
676M
340
188
1.56
921
2.93
676M
240
LOGM-XL (Smn 2025)
2.63
McnnHlow--LL (louger training)
2:20
676M
1000
RCGM (Ours)
GSD- (Rombach ot at., 2022)
2.61
675M
@SD- VAE (Ronbacc et at. 2022)
2.13
675M
BSD- VAE
2:35
675M
360
SSD VAE
1.92
675M
424
@UCABB (Chen et al., 2024c)
2.45
675M
300
SAA VAE (Yan al., 2025)
2:25
675M
424
SDC
1.79
675M
500
VA VAE
1.48
675M
424
summary, across multiple autoencoder architectures, our RCGM consistently delivers a superior trade-off between sample quality, sumpling speed, and model parameter efficiency. It establishes new SOTA results while substantially reducing the computational overhead required for high-fudelity image generation.
Validating RCGM on real-world applications. To assess its practical efficacy, we evaluate RCGM on two demanding real-world tasks: text-to-image generation (App. D.1) and the training of few-step unified multimodal models (App. D.2). Our results demonstrate that RCGM exhibits remarkable performance and versatility across these diverse settings, substantially outperforming existing methods in the computationally constrained, few-step sampling regime.
For instance, in text-to-image synthesis, RCGM attains a GenEval score of 0.85 with only NFE= 2. This marks significant advance over the previous SOTA, SANA-Sprint (Chen al., 2025c), which achieves a score of 0.77, thereby establishing a new benchmark for highly efficient generation.
5 CONCLUSION AND LIMITATIONS
In this paper, we introduced RCGM, a unified framework for continuous generative modeling that bridges the gap between multi-step and few-step synthesis. Our key innovation is a novel N-th order flow matching objective that improves training stability and significantly boosts performance, especially in few-step regimes. Through extensive experiments on ImageNet- IK, we demonstrated that RCGM establishes new state of the art across a spectrum of few-step generation settings.
Despite its strong performance, RCGM shares a limitation with contemporary generative models: achieving high-fidelity synthesis in extreme few-step regimes (e.g., 1-NFE) remains an open challenge, particularly for high-resolution imagery. We conjecture that this partly attributable to the absence of an adversarial objective, which has proven effective for enhancing perceptual quality in other generative paradigms. Consequently, a promising direction for future research is the integration of adversarial training into the RCGM framework to further push the boundaries of sample quality in this challenging setting. We leave this promising avenue for future work.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENT
This work was supported in part by the National Science and Technology Major Project (No. 00222D0115101), NSFC under No. 62576285, the Research Center for Industries the Future (RCIF) Westlake University, and the Westlake Education Foundation.
We thank the anonymous reviewers for their insightful comments and suggestions, which significantly improved this paper. We also express gratitude to Xinyi Shang, Yansen Han, and Zhenglin Cheng for their valuable discussions and feedback.
ETHICS STATEMENT
This research adheres to the ICLR Code of Ethics and is committed to the principles of responsible and transparent scientific inquiry. The study involves no human participants, personal or sensitive data, or any activities requiring approval from an institutional ethics review board. All datasets used are publicly accessible under appropriate licenses, with proper attribution given to their original sources. To promote openness and reproducibility, we provide our implementation code and experimental settings for verification and further development by the research community. We also declare that conflicts interest or external funding have influenced the design, execution, or presentation of this work.
REPRODUCIBILITY STATEMENT
Comprchensive details regarding the datasets, model architectures, optimization settings, and training procedures are provided in Sec. 4.1 of the main paper and App. D. These materials are designed to facilitate the reliable and transparent reproduction of our results. Additionally, source code will be made publicly available upon acceptance of the paper.
REFERENCES
Fan Bao, Shen Nic, Kaiwen Xue, Yue Cao, Chongxuan Li, Hang Su, and Jun Zhu, All are worth words: A vit backbone for diffusion models. In Proceedings the IEEE/CVF conference computer vision and pattern recognition, pp. (2669-22679 2023.
Andrew Brock, Jeff Donahue, and Karen Simonyan, Large scale gan training for high fidelity natural image synthesis. arXiv preprint 1XXv::8009.1196 2018.
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties self-supervised vision transformers. Proceedings of the IEEE/CVF international conference computer vision, pp. 9650-9660, 2021.
Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T Freeman. Maskgit: Masked generative image transformer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11315-13322 2022.
Jiuhai Chen. Zhiyang Xu, Xichen Pan. Yushi Hu, Can Qin, Tom Goldstein, Lifu Huang, Tianyi Zhou, Saining Xie, Silvio Savarese, Le Xue, Caiming Xiong, and Ran Xu. Blip3-o: A Family of Fully Open Unified Multimodal Mocells-rcciieecture, Training and Dataset. arXivorr,, 000/2505.00568, 2025a.
Jiuhai Chen, Zhiyang Xu, Xichen Pan, Yushi Hu, Can Qin, Tom Goldstein, Lifu Huang, Tianyi Zhou, Saining Xie, Silvio Savarese, et al. Blip3-o: family fully open unified multimodal models-architeete training and dataset. arXiv preprint arX1v:2505.09568, 2025b.
Junsong Chen, Chongjian Ge, Enze Xie, Yue Wu, Lewei Yao, Xiaozhe Ren, Zhongdao Wang, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart-o: Weak-to-strong training of diffusion transformer for 4k text-to-image generation. arXiv preprint NMW:22403.04692 2024a.

---

Published as a conference paper at ICLR 2026
Junsong Chen, Yue Wu, Simian Luo, Enze Xie, Sayak Paul, Ping Luo, Hang Zhao, and Zhenguo Li. {delta): Pixart-((eelta); Fast and controllable image generation with latent consistency models. arXiv preprint (rXiv:2401.05252, 2024b.
Pixart-(
Junsong Chen, Shuchen Xue, Yuyang Zhao, Jincheng Yu, Sayak Paul, Junyu Chen, Han Cai, Song Han, and Enze Xie. Sana-sprint: One-step diffusion with continuous-time consistency distillation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 16185-16195, 2025c.
Junying Chen, Zhenyang Cai, Pengcheng Chen, Shunian Chen, Ke Ji, Xidong Wang, Yunjin Yang, and Benyou Wang. Shareggt--4olmmgee Aligning Multimodal Models with GPT-4o-Level Image Generation. arXiv.org, abs/2506. 18095, 2025d.
Junyu Chen, Han Cai, Junsong Chen, Enze Xie, Shang Yang. Haotian Tang, Muyang Li, Yao Lu, and Song Han. Deep compression autoencoder for efficient high-resolution diffusion models. arXiv preprint arXiv:2410. 10733, 2024c.
Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. Janus-pro: Unified multimodal understanding and generation with data and model scaling, arXiv preprint arXiv:2501. 17811, 2025e.
Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashattention: Fast and memory- efficient exact attention with io-awareness. Advances in neural information processing systems, 35: 16344-16359, 2022.
Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiaonan Nie, Ziang Song, et al. Emerging properties in unified multimodal pretraining. arXiv preprint arXiv:2505. 14683, 2025.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference computer vision and pattern recognition, pp. 248-2482-55 255. leee, 2009.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-8794, 2021.
Runpei Dong, Chunrui Han, Yuang Peng, Zekun Qi, Zheng Ge, Jinrong Yang, Liang Zhao, Jianjian Sun, Hongyu Zhou, Haoran Wei, Xiangwen Kong, Xiangyu Zhang, Kaisheng Ma, and Li Yi. Dreamllm: Synergistic Multimodal Comprehension and Creation. In The Twelfth International Conference Learning Representations, 2024.
Patrick Esser. Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference on machine learning. 2024a.
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first International Conference on Machine Learning. 2024b.
Kevin Frans, Danijar Hafner, Sergey Levine, and Pieter Abbeel. One step diffusion via shortcut models. arXiv preprint arXiv:2410. 12557 2024.
Shanghua Gao, Pan Zhou, Ming-Ming Cheng, and Shuicheng Yan. Masked diffusion transformer is a strong image synthesizer. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 23173-23173, 2023.
Zhengyang Geng, Ashwini Pokle, William Luo, Justin Lin, and J Zico Kolter. Consistency models made easy. arXiv preprint arXiv:2406. 14548, 2024.
Zhengyang Geng, Mingyang Deng. Xingjian Bai, Zico Kolter, and Kaiming He. Mean flows for one-step generative modeling. arXiv preprint arXiv:2505. 1..9005.3440 /3447, 2025.
12

---

Published as a conference paper at ICLR 2026
Google.
Experiment with
gemini 2.0 flash native image
genera-
tion,
2025a.
URL
gow.leebo.gggglboo..oo.ooo com/en/ -ea---alaa-attt--------------ee-
Google.
Gemini 2.5 flash image: High-consistency image generation and editing, 8 2025b. URL https: /twstuasstuuiioooooo a google com com/modes htt///ool/ss//eem s //gmminiiiiii -2-5 5- flash- image. Official model page on Google Al Studio. Internal development code name: nano-banana.
Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent-a new approach to self-supervised learning. Advances in neural information processing systems, 33:21271-21288 2020.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pallern recognition, pp. 9729-9738, 2020.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12530 2022.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 15:6840-6851 2020.
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David Fleet. Video diffusion models. Advances in Neural Information Processing Systems, 35:8633-8646 2022.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu Yuanzhi Li, Shean Wang. Lu Wang, Weizhu Chen, al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022.
Minguk Kang, Jun-Yan Zhu, Richard Zhang. Jaesik Park, Eli Shechtman, Sylvain Paris, and Taesung Park. Scaling up gans for text-to-image synthesis. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 10124-10134, 2023.
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. Advances in neural information processing systems, 35:26565-2657 2022.
Tero Karras, Miika Aittala, Jaakko Lehtinen, Janne Hellsten, Timo Aila, and Samuli Laine. Analyzing and improving the training dynamics of diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 24174-24184 2024.
Black Forest Labs. Flux. https: httpss:/gitaein htggtgttttbbb. httpchh...ooomaeccee com/b hatnmbblckkooo foresk-foortttaee flux, 2024.
Bin Lin, Zongjian Li, Xinhua Cheng, Yuwei Niu, Yang Ye, Xianyi He, Shenghai Yuan, Wangbo Yu, Shaodong Wang, Yunyang Ge, et al. Uniworld: High-resolution semantic encoders for unified visual understanding and generation. arXiv preprint arXiv:2506.03147, 2025.
Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747, 2022.
Bingchen Liu, Ehsan Akhgari, Alexander Visheratin, Aleks Kamko, Linmiao Xu, Shivam Shrirao, Joao Souza, Suhail Doshi, and Daiqing Li. Playground v3: Improving text-to-image alignment with deep-fusion large language models. arXiv preprint arXiv:2409. 2024.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arxiv preprint arXiv:1711.05101, 2017.
Cheng Lu and Yang Song. Simplifying, stabilizing and scaling continuous-timg time consistency models. arXiv preprint 140iv:2410.1 1081, 2024.
13

---

Published as a conference paper at ICLR 2026
Simian Luo, Yiqin Tan, Longbo Huang. Jian Li, and Hang Zhao. Latent consistency models: Synthesizing high-resolution images with few-step inference. arXiv preprint arXiv:2310.04378, 2023.
Nanye Ma, Mark Goldstein, Michael S Albergo, Nicholas M Boffi, Eric Vanden- BainennEijnde,, and Saining Xie. Sit: Exploring flow and diffusion-based generative models with scalable interpolant transformers. In European Conference on Computer Vision, pp. 23-40. Springer, 2024.
ModelTC.
Owen-image-lighninng Swen-Image Lightning, 2025.
https: https://github.. Mode TC/
OpenAl. Introducing 4o image generation, 2025. URL https: pettps://opeaateee htm///ppenaa.ccmmmm/eee/////
introducing-4es mmage-generation/
Xichen Pan, Satya Narayan Shukla, Aashu Singh. Zhuokai Zhao, Shlok Kumar Mishra, Jialiang Wang, Zhiyang Xu, Jiuhai Chen, Kunpeng Li, Felix Juefei-Xu, Ji Hou, and Saining Xie, Transfer between Modalities with MetaQueries. arXiv.org. abs/2504.06256, 2025.
A Paszke. Pytorch: An imperative style, high-performance deep learning library. arXiv preprint 1rXiv:1912.00703, 2019.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 4195-4205, 2023.
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01155, 2023.
Qi Qin, Le Zhuo, Yi Xin, Ruoyi Du, Zhen Li, Bin Fu, Yiting Lu, Xinyue Li, Dongyang Liu, Xiangyang Zhu, Will Beddow, Erwann Millon, Wenhai Wang Victor Perez, Yu Qiao, Bo Zhang, Xiaohong Liu, Hongsheng Li, Chang Xu, and Peng Gao. Lumina-image 2.0: A unified and efficient image generative framework, 2025. URL https: / warxiv.vvvv//////////.//////aava .rg
wwtp//////////////////oo/////e./ http//dff///// 250.3 21758.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Axel Sauer, Katja Schwarz, and Andreas Geiger. Stylegan-xl: Scaling stylegan to large diverse datasets. In ACM SIGGRAPH 2022 conference proceedings, pp. 1-10, 2022.
Axel Sauer, Frederic Boesel, Tim Dockhorn, Andreas Blattmann, Patrick Esser, and Robin Rombach. Fast high high-resolttion image synthesis with latent adversarial diffusion distillation. In SIGGRAPH Asia 2024 Conference Papers, pp. 1-11, 2024a.
Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion distillation. European Conference on Computer Vision, pp. 87-103. Springer, 2024b.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02500: 2020a.
Yang Song and Prafulla Dhariwal. Improved techniques for training consistency models. arXiv preprint arXiv:2310. 14189, 2023.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020b..
Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. arXiv preprint arXiv:2303.01469, 2023.
Peng Sun, Yi Jiang, and Tao Lin. Unified continuous generative models. arXiv:2505.07447, 2025.
arXiv preprint
14

---

Published as a conference paper at ICLR 2026
Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng. and Liwei Wang. Visual autoregressive modeling: Scalable image generation via scxte prediction. Advances in neural information processing systems, 37:84839-884888 2024.
Fu- Yun Wang, Zhaoyang Huang, Alexander William Bergman, Dazhong Shen, Peng Gao, Michael Lingelbach, Keqiang Sun, Weikang Bian, Guanglu Song, Yu Liu, et al. Phased consistency model. arXiv preprint urXiv:2405. 0224005.8477 /8407, 2024.
Shuai Wang. Zhi Tian, Weilin Huang. and Limin Wang. Ddt: Decoupled diffusion transformer. arXiv preprint ar arXih:2504.05741, 2025a.
Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu, Pro- lificdreamer: high-fidelity and diverse text-to-3d generation with variational score distillation. Advances in neural information processing systems, 36:8406- 1441-4411,11 2023.
Zidong Wang, Yiyuan Zhang, Xiaoyu Yue, Xiangyu Yue, Yangguang Li, Wanli Ouyang, and Lei Bai. Transition models: Rethinking the generative learning objective. 2025b.
Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan. Sheng ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, Yuxiang Chen, Zecheng Tang, Zekai Zhang, Zhengyi Wang, An Yang, Bowen Yu, Chen Cheng. Dayiheng Liu, Deqing Li, Hang Zhang, Hao Meng, Hu Wei, Jingyuan Ni, Kai Chen, Kuan Cao, Liang Peng, Lin Qu, Minggang Wu, Peng Wang, Shuting Yu, Tingkun Wen, Wensen Feng, Xiaoxiao Xu, Yi Wang, Yichang Zhang, Yongqiang Zhu, Yujia Wu, Yuxuan Cai, and Zenan Liu. Qwen-image technical report, 2025a. URL https: //tp//aa.v.../////////.../.aaa.l org.orgg/ooo...o /bss/ 2508. 02324.
Chenyuan Wu, Pengfei Zheng, Ruiran Yan, Shitao Xiao, Xin Luo, Yueze Wang. Wanli Li, Xiyan Jiang, Yexin Liu, Junjie Zhou, al. Omnigen2: Exploration advanced multimodal generation. arXiv preprint arXiv:2506 250.:2871,...8,1,, 2025b.
Size Wu, Zhonghua Wu, Zerui Gong, Qingyi Tao, Sheng Jin, Qinyue Li, Wei Li, and Chen Change Loy. Openuni: A simple baseline for unified multimodal understanding and generation. 2025c.
URL https:://aaat: org/ abs/ 25005/505... 23661.
Shitao Xiao, Yueze Wang, Junjie Zhou, Huaying Yuan, Xingrun Xing, Ruiran Yan, Shuting Wang, Tiejun Huang, and Zheng Liu. Omnigen: Unified image generation. arXiv preprint arXiv:2409.1 13340, 2024.
Enze Xie, Junsong Chen, Junyu Chen, Han Cai, Haotian Tang, Yujun Lin, Zhekai Zhang, Muyang Li, Ligeng Zhu, Yao Lu, et al. Sana: Efficient nighrresolution image synthesis with linear diffusion transformers. arXiv preprint arXiv:2410. 10629, 2024a.
Enze Xie, Junsong Chen, Yuyang Zhao, Jincheng Yu, Ligeng Zhu, Yujun Lin, Zhekai Zhang, Muyang Li, Junyu Chen, Han Cai, al. Sana 1.5: Efficient scaling of training-time and inference-time compute in linear diffusion transformer. arXiv preprint arXiv:2501. 18427, 2025a.
Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation. arXiv preprint arXiv:2408. 12528, 2024b.
Jinheng Xie, Zhenheng Yang, and Mike Zheng Shou. Show-o2: Improved native unified multimodal
models. arXiv preprint arXiv:2506. /5564, 2025b.
Jingfeng Yao. Bin Yang, and Xinggang Wang. Reconstruction generation: Taming optimization dilemma in latent diffusion models. arXiv preprint arXiv:2501.01423, 2025.
Junyan Ye. Dongzhi Jiang. Zihao Wang, Leqi Zhu, Zhenghao Hu, Zilong Huang, Jun He, Zhiyuan Yan, Jinghua Yu, Hongsheng Li, Conghui He, and Weijia Li. Echo-4o: Harnessing the Power of GPT-4o Synthetic Images for Improved Image Generation. arXiv, 2025.
Tianwei Yin, Michaël Gharbi, Taesung Park, Richard Zhang. Eli Shechtman, Fredo Durand, and William T Freeman. Improved distribution matching distillation for fast image synthesis. arXiv preprint arXiv:240..1 14867, 2024a.
15

---

Published as a conference paper at ICLR 2026
Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Fredo Durand, William T Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 6613-6623 2024b.
Sihyun Yu, Sangkyung Kwak, Huiwon Jang, Jongheon Jeong, Jonathan Huang, Jinwoo Shin, and Saining Xie. Representation alignment for generation: Training diffusion transformers is easier than you think. arXiv preprint arXiv:2410.06940. 2024.
Yanli Zhao, Andrew Gu, Rohan Varma, Liang Luo, Chien-Chin Huang, Min Xu, Less Wright, Hamid Shojanazeri, Myle Ott, Sam Shleifer, et al. Pytorch fsdp: experiences scaling fully sharded data parallel. arXiv preprint arXiv:2304. 1 1277, 2023.
Hongkai Zheng, Weili Nie, Arash Vahdat, and Anima Anandkumar. Fast training of diffusion models with masked transformers. arXiv preprint ww..h::3000093000 2023.
Kaiwen Zheng, Yongxin Chen, Huayu Chen, Guande He, Ming-Yo Yu Liu, Jun Zhu, and Qinsheng Zhang. Direct discriminative optimization: Your likelihood-based visual generative model is secretly a gan discriminator. arXiv preprint arXiv:2503.01103 2025.
Linqi Zhou, Stefano Ermon, and Jiaming Song. arXiv:2503.07565, 2025.
Inductive moment matching. arXiv preprint
Mingyuan Zhou, Huangjie Zheng, Zhendong Wang. Mingzhang Yin, and Hai Huang. Score identity distillation: Exponentially fast distillation of pretrained diffusion models for one-step generation. In Forty-first International Conference Machine Learning, 2024.
16

---

Published as a conference paper at ICLR 2026
CONTENTS
1
Introduction
2 Preliminaries
2.1 Oth-order: Diffusion and Flow-Matching Models
2.2
1st-order: Recursive Consistency Models
3
Methodology
3.1
Segmented Integration along the PF-ODE Trajectory
3.2
A Unified Training Framework for Any-Step Generation
3.3
Practical Implementation of RCGM
4
Experiments
4.1
Experimental Setup
4.2
Analysis of Higher-Ore Order Training
4.3
Comparison with SOTA Few-step Methods
5
Conclusion and Limitations
10
A Utilization of Large Language Models (LLMs)
18
Implementation Details
18
B.1 Qualitative Analysis of RCGM Training Dynamics
18
C Related Work
19
C.I Foundations: Multi-Step Integration Instantancous Fields
19
C.2 Interval-Based Consistency for Few-Step Generation
................................
19
C33 Adversarial Refinement for One-Step Synthesis
20
Detailed Experiment
20
D.I Comparison with Text-to-image Models
20
D.2 Comparison with Unified Multimodal Models
................................
D.3 Quantitative Analysis on Qwen-Image-20B
23
Theoretical Analysis
23
E.1 A Recursive Learning Perspective of Consistency Models
17

---

Published as a conference paper at ICLR 2026
A UTILIZATION OF LARGE LANGUAGE MODELS (LLMS)
In this study, Large Language Models (LLMs) are employed at the sentence level to assist in linguistic refinement. Their use was strictly confined to improving grammatical accuracy and overall readability of the manuscript. All research concepts, methodological designs, experimental processes, and analytical findings remain entirely original and have been solely contributed by the authors.
B IMPLEMENTATION DETAILS
Algorithm 1 N-th Order RCGM Training Step
Require: Data distribution p(x), order N 2 0, initial time step AL (we set A/ = 0.01 in this paper),
EMA decay rate K.
Require: Initial parameters 0, stop gradient operator sg, EMA parameters 0
1: Preparation Phase:
2: Sample data p(x), noise (O,
3: Sample current time ~ (0 1) and target time 4N+1 ~ (((,
4: Construct perturbed state
iz (1 l)x. 5: Compute target velocity v = Z X.
6: Define time trajectory: set la = 1, 1 = to At, and let {4 be linearly spaced between 4 and N++1
7: Recursive Velocity Estimation:
8: Initialize accumulated displacement
9: Estimate state at f1: Xt, = x; våt.
for to do
11: Estimate segment displacement: d, = Fo- (Xt, (i41) (l+41 12).
Update state: Xt,1 +- Xt; + d;.
Accumulate: A + A d, (Note: A approximates Eimee plite v(x,, T)dT)
13:
14: end for
15: Loss Calculation:
16: Compute model output: Fo tN+1).
17: Update 0 by minimizing objective:
1 [sg(u) (N-1N+1) + -v/
(())
u sg( (()
Al
18: Update EMA parameters: 0 + k0 + (1-k)0.
During the inference stage, our sampling strategy is designed for simplicity and independence from hyperparameter tuning. For one-step generation, follow the direct prediction procedure described in Sec. 2, requiring no external hyperparameter configuration. For few-step and multi-step generation, adopt the UCGM sampler (Sun al., 2025) using its default settings without any additional manual tuning.
B. QUALITATIVE ANALYSIS OF RCGM TRAINING DYNAMICS
We investigate the convergence properties of the proposed N-th Order RCGM by modeling the training process as a dynamic interplay between a Ground-Truth Anchor and a Recursive Bootstrap Tail. The student model aims to match hybrid target displacement Dunget constructed over the interval [t, tN41). Based on Alg. 1, this target can be conceptually decomposed as:
N
D Durger R
A
Fo (x, (t, tit1)
(10)
Anchor (Low Bins, High Variance)
(High Low Variance)
18

---

Published as a conference paper at ICLR 2026
Here, the Anchor is derived from the real data distribution (providing unbiased instantaneous direction), while the Tail is generated by the EMA teacher 0 (providing smooth but potentially biased manifold approximations). The stability of this system depends critically on the interaction between the recursive order and the EMA decay rate k. We identify two distinct operating regimes:
The Error Accumulation Regime (N 2 2). When the recursive order N is high, the target trajectory is dominated by the model's own predictions (the Tail). In this regime, the primary challenge is recursive variance amplification. Since the estimation at step depends on the state Xt, predicted at step 1, small perturbations in the teacher's parameters 0 exponentially through the recursive chain. This phenomenon is analogous to eπror propagation in numerical integration. If the teacher evolves too quickly (i.e., low k), the target values become non-stationary and noisy, preventing the student from converging. Consequently, high-order training requires strong stabilization: we must employ a high EMA decay rate (e.g.. 0.999) to effectively "freeze" the teacher, minimizing the temporal variance of the target and ensuring that the recursive tail provides a consistent guidance signal.
Regime I:
compounded
Regime The Bias Correction Regime (N = 1). When N 1, the recursive tail is short, and the dominant eπror source shifts from variance to geometric bias. A single-step linear approximation inevitably undershoots the curvature of the true data manifold. If the teacher is too stable (i.e., k Rs 1), the model risks converging spurious fixed point where the student simply mimics the teacher's biased linear prediction, ignoring the curvature information. Crucially, the Anchor term v At contains the necessary first-order derivative information correct this bias. To effectively incorporate this correction, the system requires "plasticity": the teacher must update rapidly to reflect the rectified trajectory suggested by the Anchor. Thus, low-order training necessitates lower EMA decay rate (e.g., K ze 0.90). This increases the system's 's responsiveness, allowing the Anchor to actively correct the Tail's geometric errors and preventing the solidification of incorrect linear assumptions.
Synthesis. Our analysis reveals fundamental trade-off between stability and plasticity. High N enables better long-range approximation but introduces instability, necessitating a rigid teacher (High k). Low N is stable but geometrically biased, necessitating an adaptive teacher (Low x) drive correction. This informs our hyperparameter selection strategy: к should be positively correlated with the recursive order N.
C RELATED WORK
The landscape of continuous-time generative models has evolved from multi-step integration towards high-fidelity, few-step synthesis. Our work builds upon this trajectory by addressing the limitations of existing paradigms. We contextualize our contributions by surveying the two dominant research thrusts that enable rapid generation: interval-based consistency and adversarial refinement.
C.I FOUNDATIONS: MULTI-STEP INTEGRATION OF INSTANTANEOUS FIELDS
The dominant paradigm in continuous generative modeling, including Denoising Diffusion Mod- els (Ho et al., 2020; Song et al., 2020b) and Flow-Matching (Lipman et al., 2022), is the learning of an instantaneous velocity field. These models train a neural network to approximate the local dynamics dx. of Probability Flow Ordinary Differential Equation (PF-ODE). To generate sample, one must numerically integrate this field, typically requiring hundreds or thousands of steps to ensure fidelity. The core limitation of this approach is its sensitivity to coarse discretization; when using few steps, large truncation errors accumulate, particularly for trajectories with high curvature, leading to a significant degradation in sample quality (Karras et al., 2022). This challenge has catalyzed the development of methods designed for the few-step regime.
de
C.2 INTERVAL-BASER CONSISTENCY FOR FEW-STEP GENERATION
A major research thrust aims to overcome this limitation by enforcing consistency over finite time intervals, effectively teaching the model about the integrated structure the ODE path. Consistency Models (CMs) (Song al., 2023) pioneered this approach by enforcing relative constraint: the
19

---

Published as a conference paper at ICLR 2026
model's prediction of the trajectory's 's endpoint (Xo) should be consistent across different starting points (X,, X on the same path. This concept was extended by methods like MeanFlow (Geng et al., 2025), which directly model their proposed average velocity to predict other points beyond the endpoint along the PF-ODE.
However, critical implementation challenge emerged: the need to compute time derivatives en- force these interval-based objectives. Early methods relied on Jacobian- Vector Products (JVP) (Geng al., 2025; Lu & Song, 2024), which introduced a severe scalability bottleneck. JVP is computa- tionally intensive and, more importantly, incompatible with essential modern training optimizations like FlashAttention (Dao et al., 2022) and FSDP-based distributed training (Zhao et al., 2023), hindering its application to callion-paramett models. Consequently, the field has pivoted to using finiteddifference estimators as a scalable and hardware-friendly alternative (Sun et al., 2025). These estimators, which rely only on forward passes, ensure compatibility with contemporary large-scale training infrastructures.
C.3 ADVERSARIAL REFINEMENT FOR ONE-STEP SYNTHESIS
parallel and complementary approach achieves figh-fidelity, one-step generation by incorporating external, adversarial signals. This is motivated by the fact that relative consistency constraints do not explicitly guarantee that the final output lies on the true data manifold. Adversarial objectives provide absolute anchor to the data distribution.
Methods in this family, such as distillation techniques like DMD/DMD2 (Yin et al., 2024b;a) and other GAN-based refiners (Sauer et al., 2024b;a; Zheng et al., 2025), employ an auxiliary discriminator to sharpen model outputs. This adversarial pressure can be powerful enough to enable a distilled "student" model to surpass the performance of its "teacher." However, this reliance is double-edged sword. It often introduces training instability and increases computational overhead due to the auxiliary network. Critically, these frameworks typically depend on frozen, pre-trained teacher to generate a large dataset of target samples. For ultra-large models, the cost of generating this dataset can be prohibitive, in some cases exceeding the cost of training the student model itself (Yin et al., 2024a). This trade-off between sample fidelity and training complexity remains key challenge.
D DETAILED EXPERIMENT
D.1 COMPARISON WITH TEXT-TO-IMAGE MODELS
To validate the real-world applicability of our approach, we benchmarked RCGM on the text-to-image synthesis task, presenting detailed results in Tab. 3. For this evaluation, we fine-tuned the SANA-0.6B and SANA-1.6B backbones for 30, 000 steps, using batch sizes of 128 and 64, respectively. The experimental results clearly demonstrate that RCGM achieves SOTA performance while operating with extremely low NFE. We conduct all experiments on publicly available datasets (Chen et al., 2025d; Ye et al., 2025) and models (Chen et al., 2025c; Xie et al., 2024a) to ensure reproducibility and transparency.
(a) SOTA quality at 2-NFE: With the addition of a second inference step, RCGM's generative quality further enhanced, reaching a GenEval score of 0.85 for the 0.6B model and 0.84 for the 1.6B version. This level of performance surpasses not only the leading few-step models but also powerful multi-step architectures such SANA-1.5 (0.81) and Playground v3 (0.76). This top-tier output is delivered with a highly competitive throughput of 6.50 samples/s and a latency of just 0.26s.
(b) Superiority in the 1-NFE setting: When constrained to single inference step challenging setting for generative models- RCGM markedly outperforms its peers. Our 0.6B variant achieves a GenEval score of 0.80, placing it ahead of strong contenders like SANA-Sprint-LLd. (0.76) and FLUX-Schnell (0.69). Crucially, this high-quality output is paired with unmatched efficiency; at 7.30 samples/s, RCGM-0066 stands as the fastest model in this category.
The success of RCGM is especially compelling given its fundamental simplicity. Powerful baselines like SANA-Sprint employ sophisticated hybrid loss, integrating sCM (Lu & Song. 2024) with LADD (Sauer et al., 2024a)-an an adversarial technique requiring a dedicated discriminator. Our approach, however, relies solely on the straightforward objective in (8). The fact that this minimalist
20

---

Published as a conference paper at ICLR 2026
Table System-level benchmark of RCGM against text-to-image models. Throughput (batch=10) and
latency (batch=1) were measured single A100 (BF16). The best and second-best results among few-step models are highlighted. 'Our evaluation.
Throughput (samples/s)
Method
NFK.
Latency (8)
#Params
GenEval
DPG-Bench 1
Multi-step models
SDXL (Podell et al., 2023)
50x2
0.15
6.5
2.6B
0.55
74.7
PixArt-Y: (Chen et 2024a)
20x2
0.40
2.7
0.6B
0.54
80.5
SD3-Medium (Esser al., 2024b)
28x2
0.28
4.4
2.0B
0.62
84.1
FLUX-Dev (Labs, 2024)
50x2
0.04
23.0
12.0B
0.67
84.0
Playground v3 (Liu
2024)
0.06
15.0
24B
0.76
87.0
(Xie SANA-0.6B
2024a)
20x2
1.7
0.9
0.6B
0.64
83.6
SANA- (Xie et al., 2024a)
20x2
1.0
1.2
1.68
0.66
84.8
SANA-1.5 (Xe al., 2025a)
20x2
0.26
4.2
4.8B
0.81
84.7
Lumina-Image-2.00 (Qin 2025)
18x2
2.6B
0.73
87.2
Few-step models
SDXL-DMD2 (Yin al., 2024a)
2.89
0.40
0.9B
0.58
FLUX- (Labs, 2024)
0.92
1.15
12.0B
0.71
SANA-Sprint-Oain 0.66 (Chen et al., 2025c)
6.46
0,25
0.68
0.76
81.51
SANA-Sprint- 1.6B (Chen et al., 2025c)
5.68
0.24
1.6B
0.77
82.17
SDXL-UCM (Luo et al., 2023)
2.89
PixArt-LCM (Chen al., 2024b)
3.52
0.31
0.6B
0.42
PCM (Wang al., 2024)
2.62
0.56
0.98
0.55
SD3.5-Tubo (Esser
2024a)
1.61
0.68
8.0B
0.53
PixArt-MMD
2024a)
4.26
SDXL-DMD2 (Yin al., 2024a)
3.36
0.32
0.9B
0.59
FLUX Schnell (Lahs, 2024)
1.58
0.68
12.08
0.69
SANA-Sprint-0.6B (Chen al., 2025c)
7.22
0.6B
0.72
SANA-Sprint-I 1.6B (Chen al., 2025c)
6.71
SDXL-LLCM (Iuio al., 2023)
3.36
0.32
0.98
0.28
PixArt-LCM (Chen et al., 2024b)
4.26
0,25
0.6B
0.41
PCM (Wang ol., 2024)
3.16
0.40
0.9B
SD3.5.Turho (Esser
2024a)
2.48
0.45
8.0B
0.51
TIM (Wang et al., 2025b)
0.8B
0.67
75.0
RCGM-0.6B (Ours)
6.50
0,26
0.6B
0.85
80.3
RCGM-1.6B (Ours)
5.71
79.1
RCGM-066 (Ours)
7.30
77.2
RCGM-1.6B (Ours)
6.75
0.22
1.6B
0.78
76.5
framework yields SOTA results demonstrates that RCGM offers a more elegant and direct solution to the enduring conflict between sampling speed and visual fidelity.
D.2 COMPARISON WITH UNITIED MULTIMODAL MODELS
The development of Unified Multimodal Models (UMM), which are capable of both profound comprehension (typically yielding textual outputs) and sophisticated generation (resulting visual outputs), represents significant frontier in artificial intelligence. Such integrated systems hold the potential unlock synergistic capabilities, where understanding informs generation and vice versa. leading to more intelligent and versatile applications (Google, 2025a;b; OpenAI, 2025).
Recent advancements UMMs have showcased their considerable potential across diverse range of applications, including high-fidelity text-to- text-to-image generation and intricate image editing ( (Wu et al., 2025a; Pan et al., 2025). These models have been lauded within the research community for their powerful generative abilities (Chen et al., 2025a; Dong et al., 2024).
However, primary obstacle to the widespread adoption of these models is their prohibitive com- putational cost. This inefficiency stems from their reliance on iterative, diffusion-based generation processes, which incur significant overhead and lead slow inference times. To address this critical efficiency bottleneck, integrate proposed method, RCGM, into SOTA UMMs.
We demonstrate this by tin-ttuning two prominent open-source models: first, conducting full- parameter tuning OpenUni-L-512 (Wu et al., 2025c) for 60, 000 steps with batch size of 128; and second, applying Owane-Imaee200 20B Wu al. (2025a) for 000 steps with batch size of 64. The experimental results
LoRA (Hu et al., 2022) tuning (with r = 64 and = 64) to
21

---

Published as a conference paper at ICLR 2026
Table 4: System-level comparison of RCGM with unified multimodal models generation tasks. Results compare inference efficiency (NFE) and performance across three benchmarks. Best and second-bo -bstt scores are highlighted as bold and underline. respectively. indicates results using LLM-rewritten prompts on GenEval. All our experiments were conducted on 8x NVIDIA H800 GPUs.
Image Generation
Method
NFE.
GenEval f
DPG-Benchh
WISE
Show-o2-7B (Xic al., 2025b)
50x2
0.76
86.14
0.39
OmniGen (Niao et al., 2024)
30x2
0.70
81.16
OmniGen2 (Wu al. 2025b)
50x2
0.80 0.86'
83.57
Show-o (Xic ct al., 2024b) 2024b)
50x2
0.68
67.27
0.35
Janus Pro (Chen at., 2025e)
0.80
84.19
0.35
MetaQuery-XL
al. 2025)
30x2
0.78/ 0.80*
81.10
0.55
BLIP33---BB (Chen et al., 2025b)
30 + 50x
0.84
81.60
0.62
UniWorld-VI (Lin et al., 2025)
28x2
0.80 / 0.84
0.55
OpenUni-L-S12 (Wu al., 2025c)
20x2
0.85
81.54
0.52
Bagel (Deng et al., 2025)
0.88
0.52
Qwen-Image-208 (Wa al., 2025a)
50x2
0.87
88.32
0.62
OpenUaiiL-512@@CM (Song et al., 2023) (medel collapse)
0.0
OpenUni-L-S12@CMM (Song et al., 2023) (model collapse)
0.0
Qwen-Image-20B (Song et al., 2023) (model collapse) Owen-Image-20Be3 2CM (Song et al., 2023) (model collapse)
0.0
OwennJmugge22URBMMMuettt (Geng al,, 2025) (out memory)
Qwen-Image 200 (Geng al., 2025) (out of memory)
OpenUaaiL-511eeRCCMM (ours)
0.85
80.15
0.50
OpenUni---512BRRCM (ours)
0.80
76.40
0.45
Owet--magge22BBBRRCOO (ours)
0.87
87.39
0.58
Qwenn-mugge-20-B (ours)
0.82
84.09
0.50
Wwea-Image-20BBRRCOO (ours)
0,52
59.50
0.30
presented in Tab, clearly demonstrate our method's efficacy. Specifically, we observe the following key outcomes:
(a) Significant reduction in computational cost: Our method dramatically reduces the NFE to just or 2. stark contrast to the 40 to 100 NFEs required by the original models. This represents reduction of over 95% in computational workload, thereby enabling faster and more efficient image generation.
(b) Competitive performance with fewer steps: When applied to OpenUniiL-5122 our method with an NFE of 2 achieves GenEval score of 0.85, matching the performance of the original model which requires 40 steps. While there a slight decrease in the DPG-Bench and WISE scores, the performance remains highly competitive. Even with single step (NFE=1), our model maintains a strong GenEval score of 0.80.
(c) Effective application larger models: With the more powerful Qwen-Image-20B: our method at 2-NFE achieves a GenEval score of 0.82 and a DPG-Bench score of 84.09. Although these scores slightly lower than the original model's 100-NFE process, they are still comparable other leading UMMs that require significantly more computational resources. This demonstrates the scalability and effectiveness of our approach on larger, more capable models.
In summary, our proposed method provides a compelling solution to the efficiency challenges inherent in diffusion-based UMMs. By substantially decreasing the required number of generation steps while preserving a high level of performance, RCGM paves the way for more practical and accessible applications of these powerful multimodal systems.
Discussion on open-source community efforts. To the best of our knowledge, Qwen-Image- Lightning (ModelTC, 2025) represents the sole open-source initiative focused on training a few-step variant of a large-scale UMM. This method is based on the Distribution Matching Distillation (DMD2) framework (Yin et al., 2024a); however, it notably omits its generative adversarial network (GAN) loss component. This crucial omission, however, directly leads to a significant and widely acknowledged problem: generation pattern collapse. Specifically, Loven-mmaee-iigtniig is known to suffer from generating highly similar, or even nearly identical, images across diverse input prompts, severely limiting its generative diversity and overall practical utility.
22

---

Published as a conference paper at ICLR 2026
Table 5: Quantitative comparison of RCGM against baselines 2wen-Image-20B for text-to-image generation. raw denotes full-patte parameeerrr implementations the generator and scoring networks, which incur excessive GPU memory costs. Consequendly, to enable baseline methods (VSD, SiD, and DMD) to run within memory constraints, we implement their fake score networks using LoRA (r=64).
Image Generation
Method
NFE 4
GenEval f
DPG-Bench
WISE 1
VSD (Wang et al., 2023) (raw, out of GPU memory)
N/A
DMD (Yin et al., 2024b) (raw, out of GPU memory)
N/A
DMD2 (Yin et al., 2024a) (raw, out of GPU memory)
N/A
SiD (Zhou et al., 2024) (raw, of GPU memory)
N/A
0.67
84.44
0.22
VSD (Wang et al., 2023)
0.73
86.16
0.34
0.81
84.31
0.47
DMD (Yin et 2024b)
0.80
84.08
0.46
0.77
87.05
0.42
SiD (Zhou al., 2024)
0.78
86.94
0.41
0.55
79.12
0.21
sCM Song, 2024) (JVP-free)
2
0.64
83,91
0.46
0.49
82.39
0.33
McanFlow (Geng al., 2025) (JVP-free)
0.57
85.09
0.40
0.47
74.31
0.19
RCGM (N=1)
0.71
81.48
0.42
0.55
75.56
0.33
RCGM (N=2)
0.78
85.27
0.51
D.3
QUANTITATIVE ANALYSTS ON QWEN-IMAGE-20B
Tab. 5 presents comprehensive quantitative comparison of RCGM against SOTA acceleration methods on the Owen-Image-2099 model. The experiments were conducted on 8 GPUs with local batch size of 4. Unless otherwise specified, models were trained for 3,000 steps with an AdamW learning rate of x 10-5, Our analysis highlights three key observations regarding scalability, order capability, and compatibility with adversarial objectives.
Scalability to Full-Parameter Training on 20B Models. A significant challenge in accelerating large-scale diffusion models is the excessive memory consumption and training instability associated with Consistency Model (CM) based approaches. As shown in the top section of Tab. 5, baseline methods such as VSD, DMD, and SiD fail run in the "raw" pallmearrmeter setting due Out-Of- Memory (OOM) errors, forcing them to rely on low-rank approximations (LoRA, r = 64) to fit within memory constraints. In contrast, RCGM demonstrates superior memory efficiency and stability, successfully enabling full-parameter training on the 20B parameter model. This result underscores the robustness method, overcoming the well-known instability issues often plaguing CM-series methods in high-parameter regimes.
Impact Higher-Order Trajectory Approximation. We observe a clear performance correlation with the approximation order N. While the first-order variant (N = 1) provides baseline accel- cration, increasing the order to N yields a substantial performance leap. Specifically, RCGM (N = 2) significantly outperforms the first-order counterpart across all metries. For instance, at NFE=2, the second-order model achieves a GenEval score of 0.78 compared to 0.71 1 for =
E THEORETICAL ANALYSIS
E.1 A RECURSIVE LEARNING PERSPECTIVE OF CONSISTENCY MODELS
The consistency model training objective enforces self-consistency along the sampling trajectory. Given the parameterization (x Xt, == a(t) = the objective is formulated as:
a()))
Ex, [d (F
Fr, x,, (f X Art At)))]
23

---

Published as a conference paper at ICLR 2026
We focus the specific case of flow matching (Lipman et al., 2022), where a(t) t, 7(t) 1 t, á(t) = 1, and (t) -1
Under these conditions, the training loss LCM (0) simplifies to:
LCM (0) = d (t F (x,) x,, (t - At) F (x:-a1) X(-
(11)
This objective minimizes the distance between the current model prediction and the target prediction derived from the preceding time step.
We can express the & loss explicitly:
(O) = ||t F (x+) - (t At) Fa-((x--a) + x, x
(12)
To analyze the training dynamics, we consider the limit At - and apply a Taylor expansion, which yields:
LCM( - -11t- Fo-F (x,) - (t At) Fo- (x,
dx:
(13)
dt
Minimizing this loss corresponds to the following update rule: By induction, the model learns the integrated velocity field:
dx.
Fx- + (t At) Fø- (x:-ar) dt
At.
(14)
t. Fo (x1) t S/" 'x, dr = Xt XD.
(15)
dr
Let us define the prediction function as f(x,, 0) == X1 - Xo. Substituting this into (14), we obtain the following recursive relationship:
dx: dt
f f(x, 0) + f(x 0)
At
(16)
This result confirms that the consistency model training objective is equivalent to recursively learning the velocity field of the underlying ODE.
24


Reviews:
[{"review_id": "aRWlHmFF9P", "reviewer": "Reviewer_91YB", "summary": "The paper proposes RCGM (N-th order Recursive Consistent velocity field estimation for Generative Modeling), a unified framework for any-step generation that generalizes diffusion/flow-matching (0-th order) and one-step consistency/MeanFlow models (1-st order) to higher orders (N≥2). The key idea is to derive a multi-step target for the PF-ODE velocity via segmented integration and use it to train a displacement predictor, with a practical loss that avoids JVP and scales to large models. Empirical results indicates that higher-order training improves stability and quality in the few-step regime, achieving strong ImageNet FID at 2 NFEs and competitive multimodal text-to-image results.", "questions": "- The experiments on RCGM rely on specific autoencoders (VA-VAE and DC-AE). Can the method be adapted to other, possibly weaker, encoders?\n- Corollary 1 shows that as N increases, the uniform approximation error decreases. Could the authors provide some intuition behind this result?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The segmented-integration derivation for the recursive estimator is neat and connects cleanly to known objectives; the displacement parameterization under a linear transport path is well-motivated.\n- Systematic experiments demonstrate RCGM’s performance for different orders N, including an instability analysis across EMA decay factors and thorough comparisons against alternative methods.\n- The presentation is clear and easy to follow, and RCGM is introduced with well-motivated explanations and helpful illustrations.", "weaknesses": "- It would be useful to include experiments on additional datasets beyond ImageNet.\n- Figure 3(c) only compares FID scores for different N under a fixed k. It would be interesting to see whether the current results hold for varying k.\n- It would be helpful to report and compare the computational cost of the method for different N.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uj9JH9Rn9j", "reviewer": "Reviewer_jXWE", "summary": "The paper introduces a unified framework for few-step generative modeling that generalizes existing methods like Consistency Models and MeanFlow to an N-th order recursive formulation. By leveraging higher-order trajectory information from the probability flow ODE, RCGM improves training stability, scalability, and sample quality without requiring jacobian-vector products or auxiliary networks. The method achieves strong results: FID of 1.48 on ImageNet 256×256 in two sampling steps, demonstrating effectiveness in few-step generation.", "questions": "Please consider addressing the weaknesses noted above.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1) Clear motivation: few‐step generative models are valuable but face obstacles (instability, complexity). RCGM directly addresses this.\n\n2) Good theoretical framing: the segmentation of the PF-ODE trajectory, the definition of N^th order recursive velocity field estimator gives a compelling narrative and unifies prior methods.\n\n3) Reported FID of ~1.48 on ImageNet 256×256 with 2 steps using a 675 M parameter model is highly competitive. The ablation that shows N > 1 improves stability under large EMA decay rates is particularly valuable (shows the method has practical benefits). \n\n4) The fact that the method does not require Jacobian-vector products (JVP) is a good plus for memory efficiency.", "weaknesses": "1) While the theoretical derivation is sound, the paper does not fully characterise the conditions/assumptions under which the higher‐order recursion yields gains (e.g., error bounds, effect of Δt, effect of segmentation size, how approximation error accumulates).\n\n2) The paper claims simplicity and memory‐efficiency, but limited quantitative reporting is provided on computational cost, memory usage, training time, inference time, etc., especially compared to baseline few‐step methods.\n\n3) The title says “Any-step Generation” but experiments for wide range of steps beyond 2 are less emphasized.\n\n4) The paper shows that for N > 4 the performance reversed (due to approximation error) which is good, but the discussion of why this happens (and what one should do) is limited. Also does the method’s advantage hold in ultra low‐step regimes (e.g., 1 step) or does it degrade rapidly?\n\n5) How sensitive is RCGM to the choice of the time segmentation sequence {t₀, t₁, …, tₙ₊₁}? For instance, how does performance vary if the intermediate times are uneven, or chosen adaptively rather than uniform?\n\n6) You note that for N > 4 the performance trend reverses due to “accumulation of approximation errors” (Sec 4.2). Could you provide more quantitative analysis of how approximation error behaves as N increases? Are there theoretical bounds or heuristics for choosing N?\n\n7) The theoretical formulation considers the PF‐ODE velocity field and segments the integral (Eq 2, (3) etc). Does the method assume that the underlying forward noising schedule and PF‐ODE satisfy certain smoothness or Lipschitz conditions? If so, it would be good to state them explicitly.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Vo8XbLhWLB", "reviewer": "Reviewer_6g9s", "summary": "The paper proposes a novel method for training one or few-step generative models by using recurrent consecutive flow maps as target. The argument is that increasing the number of these steps N, with respect to other few steps methods such as consistency models and MeanFlow where N=1, increases training stability. The method results in SOTA generative performance on common benchmarks.", "questions": "- 1) From my understanding, in the training loss there are mostly three terms: the prediction term for the displacement, which will receive gradients; the ground truth velocity, which is the montecarlo estimate (x1 - x0) commonly used in rectified flow; the N step predictions for the small consecutive displacements, which are also used as a target from t_1 to t_N. If that's the case, I wonder how one can ensure stability, as the smaller jumps and the flow matching style 0 order term receive less or no training.\n- 2) If the argument holds for N=2, then is it correct to expect the results to improve also with N=3? Have the authors experimented with this? It would be interesting to see an ablation for different N.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The method is conceptually simple, and is effective on the proposed benchmarks. It does not rely on JVP, which can simplify implementation and usage with different frameworks and architectures.", "weaknesses": "While the results are encouraging, I think the method section of the paper is a bit rushed, which makes it hard to understand exactly the training procedure. There is no pseudocode or code, and I could not properly grasp how the training works (see questions). Furthermore, there are important details missing, such as batch size used and other hyperparameters. Some choices are not well motivated, such as how the authors arrived to Eq.9.\n\nnother thing that seems unusual is the 1-st order model not working well without EMA target. This is somewhat in disagreement with the literature. It would have been great to see the effect of EMA rate on the target network with other 1st order methods such as MeanFlow or Shortcut Models, to verify the claim that using EMA weights is helpful.\n\nOverall, the fact that using a higher order N improves performance seems counterintuitive, as I would expect more error from the target model to accumulate. While the great results obtained prove my intuition wrong, I would appreciate a clearer exposition of the why this is the case, as well as a way to verify what is actually done in practice (e.g. sharing the code).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HgBbrJygUt", "reviewer": "Reviewer_948b", "summary": "This paper proposes RCGM (Recursive Consistent velocity field estimation for Generative Modeling), a framework that extends consistency models to N-th order formulations. The authors claim their method unifies existing few-step generative models and achieves improved training stability and SOTA performance on ImageNet generation tasks.", "questions": "- Can you provide training curves, loss plots, or other evidence for the claimed \"model collapse\" of baseline methods in Table 4?\n- What is the actual wall-clock training time comparison between N=1 and N=2, accounting for the additional forward passes?\n- How does the method perform with error bars over multiple random seeds? Generative models are known to be sensitive to initialization.\n- Can you provide ablation studies showing N>1 helps across different model scales (not just 675M) and datasets (not just ImageNet)?\n- Why not compare against adversarial methods (DMD2, LADD) experimentally rather than only dismissing them in the related work?\n- In Equation (8), what is the theoretical justification for the asymmetric stop-gradient operation?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- Strong empirical results: The method achieves competitive FID scores on ImageNet (1.48 FID at 256×256 with 2 steps, 1.79 FID at 512×512 with DC-AE).\n- Comprehensive experimental coverage: The paper includes experiments on class-conditional generation, text-to-image synthesis, and unified multimodal models.", "weaknesses": "**Limited Technical Novelty**\n\nThe core contribution is an incremental extension of existing consistency models using standard numerical ODE techniques. The segmented integration in Equations (2)-(5) is a straightforward application of multi-step numerical integration (similar to predictor-corrector methods). The claim of \"unification\" is overstated—showing that N=0 recovers diffusion models and N=1 recovers consistency models is mathematical verification rather than novel insight.\n\n**Table 2 concerns:**\n\n- Compares methods with vastly different training budgets (RCGM: 424 epochs vs. IMM: 3840 epochs), making direct comparison problematic\n- Missing recent baselines: FLUX-Schnell, SD3-Turbo are mentioned in Table 3 but absent from Table 2\n- No ablation on the choice of autoencoder (SD-VAE vs. DC-AE vs. VA-VAE)\n\n**Missing ablations on:**\n\n- Time point sampling strategies for {t_i}\n- Sensitivity to N across different model scales and datasets\n\n**Weak Theoretical Contribution**\n\n- Appendix D provides standard numerical analysis (Taylor expansion error bounds)\n- Theorem 1 gives O(Δt²) truncation error + O((N+1)²ε²) approximation error—this is expected from numerical ODE theory\n- No theoretical explanation for why higher-order methods improve training stability (only convergence rate is analyzed)\n- The \"optimal N\" in Corollary 1 depends on unknown problem-dependent constants, providing limited practical guidance\n- Figure 3b shows N=2 still requires very large EMA (κ=0.999) for best performance, which contradicts claims of resolving the stability-performance trade-off\n- The stability comparison is confounded by EMA rate—N=1 with better hyperparameter tuning might achieve similar results\n\n\n## Minor Weaknesses\n\n - Code not available for review (promised \"upon acceptance\")\n- Many implementation details missing (hierarchical time sampling procedure, exact schedule)\n- Table 3 includes \"†Our evaluation\" for baselines—without code, these cannot be verified\n- No error bars or multiple runs reported for any experiments", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)