Stable Consistency Tuning: Understanding and Improving Consistency Models

## Abstract

Abstract Diffusion models achieve superior generation quality but suffer from slow generation speed due to the iterative nature of denoising. In contrast, consistency models, a new generative family, achieve competitive performance with significantly faster sampling.
These models are trained either through consistency distillation, which leverages pretrained diffusion models, or consistency training/tuning directly from raw data.
In this work, we propose a novel framework for understanding consistency models by modeling the denoising process of the diffusion model as a Markov Decision Process (MDP) and framing consistency model training as the value estimation through Temporal Difference (TD) Learning.
More importantly, this framework allows us to analyze the limitations of current consistency training/tuning strategies.
Built upon Easy Consistency Tuning (ECT), we propose Stable Consistency Tuning (SCT), which incorporates variance-reduced learning using the score identity.
SCT leads to significant performance improvements on benchmarks such as CIFAR-10 and ImageNet-64. On ImageNet-64, SCT achieves 1-step FID 2.42 and 2-step FID 1.55, a new SoTA for consistency models.

## 1 Introduction

Diffusion models have significantly advanced the field of visual generation, delivering state-of-the-art performance in images (Dhariwal & Nichol, ; Rombach et al., ; Song & Ermon, ; Karras et al., ; ), videos (Shi et al., ; Blattmann et al., ; Singer et al., ; Brooks et al., ; Bao et al., ), 3D (Gao et al., ; Shi et al., ), and 4D data (Ling et al., ).
The core principle of diffusion models is the iterative transformation of pure noise into clean samples.
However, this iterative nature comes with a tradeoff: while it enables superior generation quality and training stability compared to traditional methods (Goodfellow et al., ; Sauer et al., ), it requires substantial computational resources and longer sampling time (Song et al., ; Ho et al., ). This limitation becomes a substantial bottleneck when generating high-dimensional data, such as high-resolution images and videos, where the increased generation cost slows practical application.

Consistency models (Song et al., ), an emerging generative family, largely address these challenges by enabling high-quality, one-step generation without adversarial training.
Recent studies (Song & Dhariwal, ; Geng et al., ) have shown that one-step and two-step performance of consistency models can rival that of leading diffusion models, which typically require dozens or even hundreds of inference steps, underscoring the tremendous potential of consistency models.
The primary training objective of consistency models is to enforce the self-consistency condition (Song et al., ), where predictions for any two points along the same trajectory of the probability flow ODE (PF-ODE) converge to the same solution.
To achieve this, consistency models adopt two training methods: consistency distillation (CD) and consistency training/tuning (CT).
Consistency distillation leverages a frozen pretrained diffusion model to simulate the PF-ODE, while consistency training/tuning directly learns from real data with no need for extra teacher models.

The motivation of this paper is to propose a novel understanding of consistency models from the perspective of bootstrapping.
Specifically, we first frame the numerical solving process of the PF-ODE (i.e., the reverse diffusion process) as a Markov Decision Process (MDP), also indicated in prior works (Black et al., ; Fan et al., ).
The initial state of the MDP is randomly sampled from Gaussian. The intermediate state consists of the denoised sample $\mathbf{x}_{t}$ and the corresponding conditional information, including the timestep $t$.
The policy function of the MDP corresponds to the action of applying the ODE solver to perform single-step denoising, resulting in the transition to the new state.
Building on this MDP, we show that the training of consistency models, including consistency distillation, consistency training/tuning, and their variants, can be interpreted as Temporal difference (TD) learning (Sutton & Barto, ), with specific reward and value functions aligned with the PF-ODE.
From this viewpoint, we can derive, as we will elaborate later, that the key difference between consistency distillation and consistency learning lies in how the ground-truth reward is estimated.
The difference leads to distinct behaviors: Consistency distillation has a lower performance upper bound (being limited by the performance of the pretrained diffusion model) but exhibits lower variance and greater training stability. Conversely, consistency training/tuning offers a higher performance upper bound but suffers from a larger variance in reward estimation, which can lead to unstable training. Additionally, for both CD and CT, smaller ODE steps (i.e., $\Delta t=t-r$) can improve the performance ceiling but complicate the optimization.

Building upon the foundation of Easy Consistency Tuning (ECT), we introduce Stable Consistency Tuning (SCT), which incorporates several enhancements for variance reduction and faster convergence:
1) We introduce a variance-reduced training target for consistency training/tuning via the score identity (Vincent, ; Xu et al., ), which provides a better approximation of the ground truth score. This helps improve training stability and facilitates better performance and convergence. Additionally, we show that variance-reduced estimation can be applied to conditional generation settings for the first time.
2) Our method adopts a smoother progressive training schedule that facilitates training dynamics and reduces discretization error.
3) We extend the scope of ECT to multistep settings, allowing for deterministic multistep sampling. Additionally, we investigate the potential capacity and optimization challenges of multistep consistency models and propose an edge-skipping multistep inference strategy to improve the performance of multistep consistency models.
4) We validate the effectiveness of classifier-free guidance in consistency models, where generation is guided by a sub-optimal version of the consistency model itself.

Figure: Figure 1: Stable consistency tuning (SCT) with variance reduced training target. SCT provides a unifying perspective to understand different training strategies of consistency models.
Refer to caption: x1.png

## 2 Preliminaries on Consistency Models

In this section, we present the essential background on consistency models to ensure a more self-contained explanation.

Diffusion Models define a forward stochastic process with the intermediate distributions $\mathbb{P}_{t}(\mathbf{x}_{t}|\mathbf{x}_{0})$ conditioned on the initial data $\mathbf{x}_{0}\sim\mathbb{P}_{0}$ (Lipman et al., ; Kingma et al., ). The intermediate states follow a general form $\mathbf{x}_{t}=\alpha_{t}\mathbf{x}_{0}+\sigma_{t}\bm{\epsilon}$ with $\mathbf{x}_{1}\approx\bm{\epsilon}\sim\mathcal{N}(0,\mathbf{I})$. The forward process can be described with the following stochastic differential equation (SDE):

$$ $\mathrm{d}\mathbf{x}_{t}=f_{t}\mathbf{x}_{0}\mathrm{d}t+g_{t}\mathrm{d}\bm{w}_ {t},$ (1) $$

where $\bm{w}_{t}$ is the standard Wiener process, $f_{t}=\frac{\mathrm{d}\log\alpha_{t}}{\mathrm{d}t}$, and $g_{t}^{2}=\frac{\mathrm{d}\sigma_{t}^{2}}{\mathrm{d}t}-2\frac{\mathrm{d}\log
\alpha_{t}}{\mathrm{d}t}\sigma_{t}^{2}$. For the above forward SDE, a remarkable property is that there exists a reverse-time ODE trajectory for data sampling, which is termed as probability flow ODE (PF-ODE) (Song et al., ) That is,

$$ $\mathrm{d}\mathbf{x}=\left[f_{t}-\frac{g_{t}^{2}}{2}\nabla_{\mathbf{x}}\log \mathbb{P}_{t}(\mathbf{x})\right]\mathrm{d}t.$ (2) $$

It allows for data sampling without introducing additional stochasticity while satisfying the pre-defined marginal distributions $\mathbb{P}_{t}(\mathbf{x}_{t})=\mathbb{E}_{\mathbb{P}(\mathbf{x}_{0}\mid
\mathbf{x}_{t})}[\mathbb{P}(\mathbf{x}_{t}\mid\mathbf{x}_{0})]$. In diffusion models, a neural network $\bm{\phi}$ is typically trained to approximate the score function $\bm{s}_{\bm{\phi}}(\mathbf{x}_{t},t)\approx\nabla_{\mathbf{x}}\log\mathbb{P}_{
t}(\mathbf{x}_{t})$, enabling us to apply numerical solver to approximately solving the PF-ODE for sampling. Many works apply epsilon-prediction $\bm{\epsilon}_{\bm{\phi}}(\mathbf{x}_{t},t)=-\sigma_{t}\nabla_{\mathbf{x}}\log
\mathbb{P}_{t}(\mathbf{x}_{t})$ form for training.

Consistency Models propose a training approach that teaches the model to directly predict the solution point of the PF-ODE, thus enabling 1-step generation. Specifically, for a given trajectory $\{\mathbf{x}_{t}\}_{t\in[0,1]}$, the consistency model $\bm{f}_{\bm{\theta}}(\mathbf{x}_{t},t)$ is trained to satisfy $\bm{f}_{\bm{\theta}}(\mathbf{x}_{t},t)=\mathbf{x}_{0},\forall t\in[0,1]$, where $\mathbf{x}_{0}$ is the solution point on the same PF-ODE with $\mathbf{x}_{t}$. The training strategies of consistency models can be categorized into consistency distillation and consistency training. But they share the same training loss design,

$$ $d(\bm{f}_{\bm{\theta}}(\mathbf{x}_{t},t),\bm{f}_{{\bm{\theta}}^{-}}(\mathbf{x} _{r},r))\,,$ (3) $$

where $d(\cdot,\cdot)$ is the loss function, $0\leq r<t\leq 1$, $\bm{\theta}^{-}$ is the EMA weight of $\bm{\theta}$ or simply set to $\bm{\theta}$ with gradient disabled for backpropagation. Both $\mathbf{x}_{t}$ and $\mathbf{x}_{r}$ should be approximately on the same PF-ODE trajectory.

## 3 Understanding Consistency Models

Consistency model as bootstrapping. For a general form of diffusion $\mathbf{x}_{t}=\alpha_{t}\mathbf{x}_{0}+\sigma_{t}\bm{\epsilon}$, there exists an exact solution form of PF-ODE as shown in previous work (Song et al., ; Lu et al., ),

$$ $\displaystyle\mathbf{x}_{s}=\frac{\alpha_{s}}{\alpha_{t}}\mathbf{x}_{t}-\alpha _{s}\int_{\lambda_{t}}^{\lambda_{s}}e^{-\lambda}\bm{\epsilon}(\mathbf{x}_{t_{ \lambda}},t_{\lambda})\mathrm{d}\lambda\,,$ (4) $$

where $\lambda_{t}=\ln(\alpha_{t}/\sigma_{t})$, $t_{\lambda}$ is the reverse function of $t_{\lambda}$, and $\bm{\epsilon}(\mathbf{x}_{t_{\lambda},t})=-\sigma_{t_{\lambda}}\nabla\log
\mathbb{P}_{t_{\lambda}}(\mathbf{x}_{t_{\lambda}})$ is the scaled score function. Consistency models aim to learn a $\mathbf{x}_{0}$ predictor with only the information from $\mathbf{x}_{t},\forall t\in[0,1]$. The left term is already known with $\mathbf{x}_{t}$, and thereby we can write the consistency model-based $\mathbf{x}_{0}$ prediction as

$$ $\displaystyle\hat{\mathbf{x}}_{0}(\mathbf{x}_{t},t;{\bm{\theta}})=\frac{1}{ \alpha_{t}}\mathbf{x}_{t}-\bm{h}_{\bm{\theta}}(\mathbf{x}_{t},t),$ (5) $$

where $s$ is set to 0 0 with $\alpha_{s}=1$, $\bm{\theta}$ is the model weights, and $\bm{h}_{\bm{\theta}}$ is applied to approximate the weighted integral of $\bm{\epsilon}$ from $t$ to $s=0$.

The loss of consistency models penalize the $\mathbf{x}_{0}$ prediction distance between $\mathbf{x}_{t}$ and $\mathbf{x}_{r}$ at adjacent timesteps,

$$ $\displaystyle\hat{\mathbf{x}}_{0}(\mathbf{x}_{t},t;\bm{\theta})\overset{\text{ fit}}{\longleftarrow}\hat{\mathbf{x}}_{0}(\mathbf{x}_{r},r;\bm{\theta^{-}})\,,$ (6) $$

where $0\leq r<t$ and $\bm{\theta}^{-}$ is the EMA weighg of $\bm{\theta}$.
Therefore, we have the following learning target

$$ $\displaystyle\frac{1}{\alpha_{t}}\mathbf{x}_{t}-\bm{h}_{\bm{\theta}}(\mathbf{x }_{t},t)\overset{\text{fit}}{\longleftarrow}\frac{1}{\alpha_{s}}\mathbf{x}_{r} -\bm{h}_{\bm{\theta}^{-}}(\mathbf{x}_{r},r)$ (7) $$

It is noting that $\mathbf{x}_{r}=\frac{\alpha_{r}}{\alpha_{t}}\mathbf{x}_{t}-\alpha_{r}\int_{
\lambda_{t}}^{\lambda_{r}}e^{-\lambda}\bm{\epsilon}(\mathbf{x}_{t_{\lambda}},t
_{\lambda})\mathrm{d}\lambda$, and hence we replace the $\mathbf{x}_{r}$ in the above equation and have

$$ $\displaystyle{\color[rgb]{0.12109375,0.46484375,0.70703125}\definecolor[named] {pgfstrokecolor}{rgb}{0.12109375,0.46484375,0.70703125}\bm{h}_{\bm{\theta}}( \mathbf{x}_{t},t)}\overset{\text{fit}}{\longleftarrow}{\color[rgb]{1,0,0} \definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\bm{r}}+{\color[rgb]{.75,0,.25} \definecolor[named]{pgfstrokecolor}{rgb}{.75,0,.25}\bm{h}_{\bm{\theta}^{-}}( \mathbf{x}_{r},r)}\,,$ (8) $$

where ${\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\bm{r}=\int_
{\lambda_{t}}^{\lambda_{s}}e^{-\lambda}\bm{\epsilon}(\mathbf{x}_{t_{\lambda}},
t_{\lambda})\mathrm{d}\lambda}$. The above equation is a Bellman Equation. ${\color[rgb]{0.12109375,0.46484375,0.70703125}\definecolor[named]{
pgfstrokecolor}{rgb}{0.12109375,0.46484375,0.70703125}\bm{h}_{\bm{\theta}}(
\mathbf{x}_{t},t)}$ is the value estimation at state $\mathbf{x}_{t}$, ${\color[rgb]{.75,0,.25}\definecolor[named]{pgfstrokecolor}{rgb}{.75,0,.25}\bm{
h}_{\bm{\theta}^{-}}(\mathbf{x}_{r},r)}$ is the value estimation at state $\mathbf{x}_{r}$, and ${\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\bm{r}}$ is the step ‘reward’.

Standard formulation. It is known that the diffusion generation process can be modeled as a Markov Decision Process (MDP) (Black et al., ; Fan et al., ), and here we show that the training of consistency models can be viewed as a value estimation learning process, which is also known as Temporal Difference Learning (TD-Learning), in the equivalent MDP. We show the standard formulation in Table 1.

In Table 1, $s_{t_{n}}$ and $a_{t_{n}}$ are the state and action at timestep $t_{n}$, $P_{0}$ and $P$ are the initial state distribution and state transition distribution, $\Phi(\mathbf{x}_{t_{N-n}},t_{N-n},t_{N-n-1})$ is the ODE solver, $\pi$ is the policy following the PF-ODE, reward $R$ is equivalent to the ${\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}r}$ defined above and value function $V_{\bm{\theta}}$ is corresponding to $\bm{h}_{\bm{\theta}}$. $\pi$ is the Dirac distribution $\delta$ due to the deterministic nature of PF-ODE.

From this perspective, we can have a unifying understanding of consistency model variants and their behaviors. Fig. 1 provides a straightforward illustration of our insight. One of the most important factors of the consistency model performance is how we estimate ${\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\bm{r}}$ in the equation.

**Table 1: The definition of symbols in the value estimation of the PF-ODE equivalent MDP.**
| MDP symbols | Definition |
| --- | --- |
| $s_{t_{n}}$ | $(t_{N-n},\mathbf{x}_{t_{N-n}})$ |
| $a_{t_{n}}$ | $\mathbf{x}_{t_{N-n-1}}:=\Phi(\mathbf{x}_{t_{N-n}},t_{N-n},t_{N-n-1})$ |
| $P_{0}(s_{0})$ | $(t_{N},\mathcal{N}(\bm{0},\mathbf{I}))$ |
| $P(s_{t_{n+1}}\mid s_{t_{n}},a_{t_{n}})$ | $(\delta_{t_{N-n-1}},\delta_{\mathbf{x}_{t_{N-n-1}}})$ |
| $\pi(a_{t_{n}}\mid s_{t_{n}})$ | $\delta_{\mathbf{x}_{t_{N-n-1}}}$ |
| $R(s_{t_{n}},a_{t_{n}})$ | $\int_{\lambda_{t_{N-n}}}^{\lambda_{t_{N-n-1}}}e^{-\lambda}\bm{\epsilon}(<br>\mathbf{x}_{t_{\lambda}},t_{\lambda})\mathrm{d}\lambda$ |
| $V_{\bm{\theta}}(s_{t_{n}})$ | $\bm{h}_{\bm{\theta}}(\mathbf{x}_{t_{N-n}},t_{N-n})$ |

Understanding consistency distillation. For consistency distillation, the approximation of ${\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\bm{r}}$ is depent on the pretrained diffusion model $\bm{\epsilon}_{\bm{\phi}}$ and the ODE solver applied. For instance, if the first-order DDIM (Song et al., ) is applied, then the approximation is formulated as,

$$ $\displaystyle{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0 }\bm{r}}\approx\bm{\epsilon}_{\bm{\phi}}(\mathbf{x}_{t},t)\int_{\lambda_{t}}^{ \lambda_{r}}e^{-\lambda}\mathrm{d}\lambda+\mathcal{O}((\lambda_{r}-\lambda_{t} )^{2})\,.$ (9) $$

We can observe that the error comes from two aspects: one is the prediction error between the pretrained diffusion model $\bm{\epsilon}_{\bm{\phi}}(\mathbf{x}_{t},t)$ and the ground truth $\bm{\epsilon}(\mathbf{x}_{t},t)$; the other is the first-order assumption that $\bm{\epsilon}(\mathbf{x}_{t_{\lambda}},t_{\lambda})\approx\bm{\epsilon}(
\mathbf{x}_{t},t),\forall t_{\lambda}\in[t,r]$. The first error indicates a better pretrained diffusion model can lead to better performance of consistency distillation. The second error indicates that the distance between $t$ and $s$ should be small eough to eliminate errors caused by low-order approximation. This perspective also connects the n-step TD algorithm with the consistency distllation. The n-step TD is equivalent to apply multistep (n-step) ODE solver to compute the $\mathbf{x}_{r}$ from $\mathbf{x}_{t}$.

Understanding consistency training/tuning. For consistency training/tuning, the approximation of ${\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\bm{r}}$ is achieved through approximating the groudtruth $\bm{\epsilon}(\mathbf{x}_{t},t)$ with the conditional $\bm{\epsilon}(\mathbf{x}_{t},t;\mathbf{x}_{0})$, where $\mathbf{x}_{0}$ is sampled from the dataset $\mathcal{D}$ and $\mathbf{x}_{t}=\alpha_{t}\mathbf{x}_{0}+\sigma_{t}\bm{\epsilon}$ with $\bm{\epsilon}\sim\mathcal{N}(\bm{0},\mathbf{I})$. It is konwn that the groudtruth $\bm{\epsilon}(\mathbf{x}_{t},t)$ is equivalent to

$$ $\begin{split}\bm{\epsilon}(\mathbf{x}_{t},t)&=-\sigma_{t}\nabla_{\mathbf{x}_{t }}\log\mathbb{P}_{t}(\mathbf{x}_{t})\\ &=-\sigma_{t}\mathbb{E}_{\mathbb{P}_{t}(\mathbf{x}_{0}\mid\mathbf{x}_{t})} \left[\nabla_{\mathbf{x}_{t}}\log\mathbb{P}(\mathbf{x}_{t}\mid\mathbf{x}_{0}) \right]\\ &=-\sigma_{t}\mathbb{E}_{\mathbb{P}(\mathbf{x}_{0}\mid\mathbf{x}_{t})}\left[- \frac{\mathbf{x}_{t}-\alpha_{t}\mathbf{x}_{0}}{\sigma_{t}^{2}}\right]=\mathbb{ E}_{\mathbb{P}(\mathbf{x}_{0}\mid\mathbf{x}_{t})}\left[\frac{\mathbf{x}_{t}- \alpha_{t}\mathbf{x}_{0}}{\sigma_{t}}\right]=\mathbb{E}_{\mathbb{P}(\mathbf{x} _{0}\mid\mathbf{x}_{t})}\left[\bm{\epsilon}(\mathbf{x}_{t},t;\mathbf{x}_{0}) \right]\end{split}\,,$ (10) $$

where $\bm{\epsilon}(\mathbf{x}_{t},t;\mathbf{x}_{0})=\frac{\mathbf{x}_{t}-\alpha_{t}
\mathbf{x}_{0}}{\sigma_{t}}$. In simple terms, the ground truth $\bm{\epsilon}(\mathbf{x}_{t},t)$ is the expectation of all possible conditional epsilon $\bm{\epsilon}(\mathbf{x}_{t},t;\mathbf{x}_{0}),\forall\mathbf{x}_{0}\in
\mathcal{D}$. Instead, previous work on consistency training/tuning apply the conditional epsilon to approximate the ground truth epsilon, which can be regarded as a one-shot MCMC approximation. The approximation is formulated as,

$$ $\displaystyle{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0 }\bm{r}}\approx\bm{\epsilon}(\mathbf{x}_{t},t;\mathbf{x}_{0})\int_{\lambda_{t} }^{\lambda_{r}}e^{-\lambda}\mathrm{d}\lambda+\mathcal{O}((\lambda_{r}-\lambda_ {t})^{2})$ (11) $$

Similarly, the error comes from two aspects: one is the difference between conditional epsilon $\bm{\epsilon}(\mathbf{x}_{t},t;\mathbf{x}_{0})$ and groudtruth epsilon $\bm{\epsilon}(\mathbf{x}_{t},t)$; the other is the first-order approximation error. Even though, it is shown by previous work that the final learning objective will converge to the gourd truth under minor assumptions (e.g, $L$-Lipschitz continuity) (Song et al., ).

$$ $\displaystyle\bm{h}_{\bm{\theta}}(\mathbf{x}_{t},t)=\mathbb{E}_{\mathbb{P}( \mathbf{x}_{0}|\mathbf{x}_{t})}\left[\bm{\epsilon}(\mathbf{x}_{t},t;\mathbf{x} _{0})\right]\int_{\lambda_{t}}^{\lambda_{r}}e^{-\lambda}\mathrm{d}\lambda+h_{ \bm{\theta}}(\mathbf{x}_{r},r)\,.$ (12) $$

However, the variance of one-shot MCMC is large. This causes the consistency training/tuning is not as stable as distillation methods even though it has a better upper bound.

Summary.
In summary, the main performance bottlenecks in improving consistency training/tuning can be attributed to two factors:

- 1.
Training Variance: This refers to the gap between the conditional epsilon $\bm{\epsilon}(\mathbf{x}_{t},t;\mathbf{x}_{0})$ and the ground truth epsilon $\bm{\epsilon}(\mathbf{x}_{t},t)$. Although, in theory, the conditional epsilon is expected to match the ground truth epsilon on average, it exhibits higher variance, which introduces instability and deviations during training.
- 2.
Discretization Error: In the numerical solving of the ODE for consistency training/tuning, only first-order solvers can be approximated. To push performance to its upper limit, the time intervals between sampled points, $t$ and $r$, must be minimized, i.e., $\mathrm{d}t=\lim(t-r)\to 0$. However, smaller $\mathrm{d}t$ results in a longer information propagation process (with large $N$). If the training process lacks stability, error accumulation through bootstrapping may occur, potentially causing training failure.

Figure: Figure 2: Phasing the ODE path along the time axis for consistency training. We visualize both training and inference techniques in discrete form for easier understanding.
Refer to caption: x2.png

## 4 Stable Consistency Tuning

Our method builds upon Easy Consistency Tuning (ECT) (Geng et al., ), chosen for its efficiency in prototyping. Given our analysis of consistency models from the bootstrapping perspective, we introduce several techniques to enhance performance.

### 4.1 Reducing the Training Variance

Previous research has shown that reducing the variance for diffusion training can lead to improved training stability and performance (Xu et al., ).
However, this technique has only been applied to unconditional generation and diffusion model training.
We generalize this technique to both conditional/unconditional generation and consistency training/tuning for variance reduction.
Let $\bm{c}$ represent the conditional inputs (e.g., class labels).
We begin with

$$ $\displaystyle\begin{aligned} \nabla_{\mathbf{x}_{t}}\log\mathbb{P}_{t}(\mathbf {x}_{t}\mid\bm{c})&=\mathbb{E}_{\mathbb{P}(\mathbf{x}_{0}\mid\mathbf{x}_{t}, \bm{c})}\left[\nabla_{\mathbf{x}_{t}}\log\mathbb{P}_{t}(\mathbf{x}_{t}\mid \mathbf{x}_{0},\bm{c})\right]\\ &=\mathbb{E}_{\mathbb{P}(\mathbf{x}_{0}\mid\bm{c})}\left[\frac{\mathbb{P}( \mathbf{x}_{0}\mid\mathbf{x}_{t},\bm{c})}{\mathbb{P}(\mathbf{x}_{0}\mid\bm{c}) }\nabla_{\mathbf{x}_{t}}\log\mathbb{P}_{t}(\mathbf{x}_{t}\mid\mathbf{x}_{0}, \bm{c})\right]\\ &=\mathbb{E}_{\mathbb{P}(\mathbf{x}_{0}\mid\bm{c})}\left[\frac{\mathbb{P}( \mathbf{x}_{t}\mid\mathbf{x}_{0},\bm{c})}{\mathbb{P}(\mathbf{x}_{t}\mid\bm{c}) }\nabla_{\mathbf{x}_{t}}\log\mathbb{P}_{t}(\mathbf{x}_{t}\mid\mathbf{x}_{0}, \bm{c})\right]\\ &=\mathbb{E}_{\mathbb{P}(\mathbf{x}_{0}\mid\bm{c})}\left[\frac{\mathbb{P}( \mathbf{x}_{t}\mid\mathbf{x}_{0})}{\mathbb{P}(\mathbf{x}_{t}\mid\bm{c})}\nabla _{\mathbf{x}_{t}}\log\mathbb{P}_{t}(\mathbf{x}_{t}\mid\mathbf{x}_{0})\right]\\ &\approx\frac{1}{n}\sum_{\{\mathbf{x}_{0}^{(i)}\}\sim\mathbb{P}(\mathbf{x}_{0} \mid c)}^{i=0,\dots n-1}\frac{\mathbb{P}(\mathbf{x}_{t}\mid\mathbf{x}_{0}^{(i) })}{\mathbb{P}(\mathbf{x}_{t}\mid\bm{c})}\nabla_{\mathbf{x}_{t}}\log\mathbb{P} _{t}(\mathbf{x}_{t}\mid\mathbf{x}_{0}^{(i)})\\ &\approx\frac{1}{n}\sum_{\{\mathbf{x}_{0}^{(i)}\}\sim\mathbb{P}(\mathbf{x}_{0} \mid c)}^{i=0,\dots n-1}\frac{\mathbb{P}(\mathbf{x}_{t}\mid\mathbf{x}_{0}^{(i) })}{\sum_{\mathbf{x}_{0}^{(j)}\in\{\mathbf{x}_{0}^{(i)}\}}\mathbb{P}(\mathbf{x }_{t}\mid\mathbf{x}_{0}^{(j)},\bm{c})}\nabla_{\mathbf{x}_{t}}\log\mathbb{P}_{t }(\mathbf{x}_{t}\mid\mathbf{x}_{0}^{(i)})\\ &=\frac{1}{n}\sum_{\{\mathbf{x}_{0}^{(i)}\}\sim\mathbb{P}(\mathbf{x}_{0}\mid c )}^{i=0,\dots n-1}\frac{\mathbb{P}(\mathbf{x}_{t}\mid\mathbf{x}_{0}^{(i)})}{ \sum_{\mathbf{x}_{0}^{(j)}\in\{\mathbf{x}_{0}^{(i)}\}}\mathbb{P}(\mathbf{x}_{t }\mid\mathbf{x}_{0}^{(j)})}\nabla_{\mathbf{x}_{t}}\log\mathbb{P}_{t}(\mathbf{x }_{t}\mid\mathbf{x}_{0}^{(i)})\end{aligned}$ (13) $$

The key difference between the variance-reduced score estimation of conditional generation and unconditional generation is whether the samples utilized for computing the variance-reduced target are sampled from the conditional distribution $\mathbb{P}(\mathbf{x}_{0}\mid\bm{c})$ or not.
In the class-conditional generation, this means we compute stable targets only within each class cluster. For text-to-image generation, we might estimate probabilities using CLIP (Radford et al., ) text-image similarity, though we leave this for future study.
Therefore, the conditional epsilon estimation adopted in previous consistency training/tuning can be replaced by our variance-reduced estimation:

$$ $\begin{split}\bm{\epsilon}(\mathbf{x}_{t},t)&=-\sigma_{t}\nabla_{\mathbf{x}_{t }}\log\mathbb{P}_{t}(\mathbf{x}_{t})\\ &\approx\frac{1}{n}\sum_{\{\mathbf{x}_{0}^{(i)}\}\sim\mathbb{P}(\mathbf{x}_{0} \mid c)}^{i=0,\dots n-1}\frac{\mathbb{P}(\mathbf{x}_{t}\mid\mathbf{x}_{0}^{(i) })}{\sum_{\mathbf{x}_{0}^{(j)}\in\{\mathbf{x}_{0}^{(i)}\}}\mathbb{P}(\mathbf{x }_{t}\mid\mathbf{x}_{0}^{(j)})}(-\sigma_{t}\nabla_{\mathbf{x}_{t}}\log\mathbb{ P}_{t}(\mathbf{x}_{t}\mid\mathbf{x}_{0}^{(i)}))\\ &=\frac{1}{n}\sum_{i=0}^{n-1}W_{i}\bm{\epsilon}(\mathbf{x}_{t},t;\mathbf{x}_{0 }^{(i)}))\,,\end{split}$ (14) $$

where $W_{i}=\frac{\mathbb{P}(\mathbf{x}_{t}\mid\mathbf{x}_{0}^{(i)})}{\sum_{\mathbf{
x}_{0}^{(j)}\in\{\mathbf{x}_{0}^{(i)}\}}\mathbb{P}(\mathbf{x}_{t}\mid\mathbf{x
}_{0}^{(j)})}$ is the weight of conditional $\bm{\epsilon}(\mathbf{x}_{t},t;\mathbf{x}_{0}^{(i)})$.

### 4.2 Reducing the Discretization Error

As discussed earlier, to achieve higher performance, we need to minimize $\Delta t=(t-r)$. On one hand, when $\Delta t$ is relatively large, the model suffers from increased discretization errors. On the other hand, when $\Delta t$ is too small, it may lead to error accumulation or even training failure.
Previous works (Song et al., ; Song & Dhariwal, ; Geng et al., ) employ a progressive training strategy, which has consistently been shown to be effective. The model is initially trained with a relatively large $\Delta t$, and as training progresses, $\Delta t$ is gradually reduced. Although a larger $\Delta t$ introduces higher discretization errors, it allows for faster optimization, enabling the model to quickly learn a coarse solution. Gradually decreasing $\Delta t$ allows the model to learn more fine-grained results, ultimately improving performance. In the ECT, the training schedule is determined by

$$ $t\sim\text{LogNormal}(P_{\text{mean}},P_{\text{std}}),\quad r:=\text{ReLU} \left(1-\frac{1}{q^{\lfloor\text{iter}/d\rfloor}}n(t)\right)t$ (15) $$

,
where $q$ is to determine the shrinking speed, $d$ is to determine the shrinking frequency, ReLU is equivalent to $\max(\cdot,0)$, and $n(t)$ is a pre-defined monotonic function. We note that it is beneficial to apply a smoother shrinking process. That is, we reduce both $q$ and $d$ to obtain a smoother shrinking process than the original ECT settings. This provides our method with a faster and smoother training process. In addition to the training schedule, training weight is important to balance the training across different timesteps. We apply the weighting $1/(t-r)$ following previous work (Song & Dhariwal, ; Geng et al., ). Suppose $r=\alpha t$, the weighting can be decomposed into $\frac{1}{t}\times\frac{1}{(1-\alpha)}$. The weighting scheme has two key effects: First, $1/t$ assigns higher weights to smaller timesteps, where uncertainty is lower. Predictions at smaller timesteps serve as teacher models for larger timesteps, making stable training at these smaller steps crucial. Second, $1/(1-\alpha)$ ensures that as $\Delta t$ decreases, the weight dynamically increases, preventing gradient vanishing during training. We apply a smooth term $\delta>0$ in the weighting function $1/(t-r+\delta)\leq\frac{1}{\delta}$ to avoid potential numerical issues and instability when the $\Delta t$ becomes too tiny.

### 4.3 Phasing the ODE for Consistency Tuning

Previous works (Heek et al., ; Wang et al., ) propose dividing the ODE path along the time axis into multiple segments during training, enabling consistency models to support deterministic multi-step sampling with improved performance. We test our method in this scenario and find that, while this training approach increases the minimum required sampling steps, it improves the fidelity and stability of the generated results. We apply the Euler solver to achieve multistep re-parameterization, formulated as:

$$ $\mathbf{x}_{s}=D_{\bm{\theta}}(\mathbf{x}_{t})+\frac{s}{t}(\mathbf{x}_{t}-D_{ \bm{\theta}}(\mathbf{x}_{t})),$ (16) $$

where $D_{\bm{\theta}}$ denotes the original consistency model, predicting the ODE solution point $\mathbf{x}_{0}$, and $s$ is the edge timestep. We propose a new training schedule to adapt to the multistep training setting.

$$ $t\sim\text{LogNormal}(P_{\text{mean}},P_{\text{std}}),\quad r:=\text{ReLU} \left(1-\frac{1}{q^{\lfloor\text{iter}/d\rfloor}}n(t)\right)(t-s)+s$ (17) $$

### 4.4 Exploring Better Inference for Consistency Model

Guiding consistency models with a bad version of itself.
Previous work (Karras et al., ) demonstrates that even unconditional diffusion models can benefit from classifier-free guidance (Ho & Salimans, ).
It suggests that the unconditional outputs in classifier-free guidance can be replaced with outputs from a sub-optimal version of the same diffusion model, thus extending the applicability of classifier-free guidance.

$$ $\nabla_{\mathbf{x}_{t}}\log\mathbb{P}_{\bm{\theta}}(\mathbf{x}_{t}|\bm{c};t)+ \nabla_{\mathbf{x}_{t}}\log\left[\frac{\mathbb{P}_{\bm{\theta}}(\mathbf{x}_{t} |\bm{c};t)}{\mathbb{P}_{{\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor} {rgb}{1,0,0}\bm{\theta}^{\star}}}(\mathbf{x}_{t}|\bm{c};t)}\right]^{\omega}\,,$ (18) $$

where $\omega$ is the guidance strength, ${\color[rgb]{1,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,0}\bm{\theta^{
\star}}}$ is a sub-optimal version of $\bm{\theta}$, and $\bm{c}$ represents the optional label conditions.
Our empirical investigations confirm that this strategy can be applied to consistency models, resulting in enhanced sample quality.

Edge-skipping inference for multistep consistency model.
While segmenting the ODE path to train a multistep consistency model can enhance generation quality, it may encounter optimization challenges, especially around the edge timesteps $\{s_{i}\}_{i=1}^{n}$ with $s_{1}=1>\cdots>s_{i}>\cdots>s_{n}=0$. For timesteps $s_{i-1}\geq t>s_{i}$, the consistency model learns to predict $\mathbf{x}_{s_{i}}$ from $\mathbf{x}_{t}$. However, for $s_{i}\geq t^{\prime}>s_{i+1}$, the model learns to predict $\mathbf{x}_{s_{i+1}}$ from $\mathbf{x}_{t^{\prime}}$. When $t$ and $t^{\prime}$ are very close to $s_{i}$, denoted as $t=s_{i}^{+}$ and $t^{\prime}=s_{i}^{-}$, it is apparent that $\mathbf{x}_{s_{i}^{+}}$ and $\mathbf{x}_{s_{i}^{-}}$ can be very similar. However, the model is expected to predict two distinct results ($\mathbf{x}_{s_{i}}$ and $\mathbf{x}_{s_{i+1}}$) from very similar inputs ($\mathbf{x}_{s_{i}^{+}}$ and $\mathbf{x}_{s_{i}^{-}}$).

Neural networks typically follow $L$-Lipschitz continuity, where small input changes result in small output changes. This property conflicts with the requirement to produce distinct outputs from similar inputs near edge timesteps, potentially leading to insufficient training, particularly near $s_{i}^{-}$.
To address this, we propose skipping the edge timesteps during multistep sampling. Specifically, even though we aim for the model to perform sampling through the timesteps

$$ $s_{1}:=1\to s_{2}\to s_{3}\to\cdots\to s_{n}:=0\,,$ (19) $$

we instead achieve multistep sampling via

$$ $s_{1}:=1\to\eta s_{2}\to\eta s_{3}\to\cdots\to\eta s_{n}:=0\,,$ (20) $$

where $\eta>0$ is a scaling factor. When $\eta$ is set to 1, the process reverts to normal multistep sampling. This method works because the predictions of $\mathbf{x}_{s_{i}}$ and $\mathbf{x}_{\eta s_{i}}$ are close when $\eta$ is near 1, allowing for a tolerable degree of approximation error.
Fig. 2 illustrates this concept with a discrete example.
The model is designed to sample via the sequence $\mathbf{x}_{1}\to\mathbf{x}_{3/6}\to\mathbf{x}_{0}$; however, it instead samples through the sequence $\mathbf{x}_{1}\to\mathbf{x}_{2/6}\to\mathbf{x}_{0}$.

Figure: (a) 1-step
Refer to caption: x3.png

## 5 Experiments

### 5.1 Experiment Setups

Evaluation Benchmarks. Following the evaluation protocols of iCT (Song & Dhariwal, ) and ECT (Geng et al., ), we validate the effectiveness of SCT on CIFAR-10 (unconditional and conditional) (Krizhevsky et al., ) and ImageNet-64 (conditional) (Deng et al., ). Performance is measured using Frechet Inception Distance (FID, lower is better) (Heusel et al., ) consistent with recent studies (Geng et al., ; Karras et al., ).

Compared baselines.
We compare our method against
accelerated samplers (Lu et al., ; Zhao et al., ),
state-of-the-art diffusion-based methods (Ho et al., ; Song & Ermon, ; ; Karras et al., ),
distillation methods (Zhou et al., ; Salimans & Ho, ),
alongside consistency training and tuning approaches.
Among these models, consistency training and tuning methods serve as key baselines, including CT (LIPIPS) (Song et al., ), iCT (Song & Dhariwal, ), ECT (Geng et al., ), and MCM (CT) (Heek et al., ).
CT introduces the first consistency training algorithm, utilizing LIPIPS loss to improve FID performance. iCT presents an improved training strategy over CT, making the performance of consistency training comparable to state-of-the-art diffusion models for the first time. MCM (CT) proposes segmenting the ODE path for consistency training, while ECT introduces the concept of consistency tuning along with a continuous-time training strategy, achieving notable results with significantly reduced training costs.

Model Architectures and Training Configurations From a model perspective, iCT is based on the ADM (Dhariwal & Nichol, ), ECT is built on EDM2 (Karras et al., ), and MCM follows the UViTs of Simple Diffusion (Hoogeboom et al., ). The model size of ECT is similar to that of iCT, while MCM does not explicitly specify the model size. The iCT model is randomly initialized, whereas both ECT and MCM use pretrained diffusion models for initialization. In terms of training costs, iCT uses a batch size of 4096 across 800,000 iterations, MCM employs a batch size of 2048 for 200,000 iterations, and ECT utilizes a batch size of 128 for 100,000 iterations.
SCT follows ECT’s model architecture and training configuration.

### 5.2 Results and Analysis

Training efficiency and efficacy. In Fig. 3(b), we plot 1-step FID and 2-step FID for SCT and ECT along the number of training epochs, under the same training configuration. From the figure, we observe that SCT significantly improves convergence speed compared to ECT, demonstrating the efficiency and efficacy of SCT training. Additionally, the performance comparisons in Tables 4 and 4 also show that SCT outperforms ECT across different settings.

Quantitative evaluation.
We present results in Table 4 and Table 4. Our approach consistently outperforms ECT across various scenarios, achieving results comparable to advanced distillation strategies and diffusion/score-based models.

Figure: Figure 4: The effectiveness of variance reduced training target.
Refer to caption: x5.png

Figure: Figure 6: The effectiveness of classifier-free guidance on consistency models.
Refer to caption: x7.png

The effectiveness of training variance reduction. It is worth noting that SCT and ECT employ different progressive training schedules. To exclude this effect, we adopt ECT’s fixed training schedule, in which the 2-step FID surpasses Consistency Distillation within a single A100 GPU hour.
We use $\Delta t=t/256$ as a fixed partition, with a batch size of 128, over 16k iterations on CIFAR-10, while keeping all other settings unchanged.
For SCT models on CIFAR-10, we calculate the variance-reduced target only within the training batch, which is also the default setting of all our experiments on CIFAR-10.
To further showcase the effectiveness of the variance-reduced target, we use all 50,000 training samples as a reference to compute the target.
Although more reference samples are used, they do not directly influence the model’s computations; they are solely utilized for calculating the training target.
Fig. 5 presents a comparison of these three methods, showing that our approach achieves notable improvements in both 1-step and 2-step FID. Notably, when using the entire sample set as the reference batch, the improvement becomes more pronounced, with the 1-step FID dropping from 5.61 to 4.56.

Table: Table 2: Comparing the quality of samples on CIFAR-10.

The Effectiveness of CFG. Inspired by prior work Karras et al. (), we adopt the outputs of the sub-optimal version of the model as the negative part in classifier-free guidance (CFG). We set the CFG strength as 1.2 and the sub-optimal version as the ema weight with half training iterations by default. We investigate the influence of the two factors on SCT-S models on ImageNet. As illustrated in Fig. 6, an appropriate CFG setting can significantly enhance the quality of generation.

Edge-skipping Multistep Sampling. To demonstrate the effectiveness of our method, we record the 4-step FID curve at various training stages, utilizing different $\eta$ values for edge-skipping multistep inference. We find that a smaller $\eta$ at the beginning of training yields superior performance. As training progresses, the model’s estimates of multi-stage results become increasingly accurate, and larger $\eta$ values gradually enhance performance. However, as previously analyzed, the multistep model struggles to achieve perfect multistep training, leading to better overall performance for $\eta=0.9$ compared to $\eta=1.0$ (the default method).

## 6 Conclusion

In this work, we propose Stable Consistency Tuning (SCT), a novel approach that unifies and improves consistency models. By addressing the challenges in training variance and discretization errors, SCT achieves faster convergence and offers insights for further improvements.
Our experiments demonstrate state-of-the-art 1-step and few-step generative performance on both CIFAR-10 and ImageNet-64$\times$64, offering a new perspective for future studies on consistency models.

## Appendix

[appendices]

## Appendix I Related Works

Diffusion Models. Diffusion models (Ho et al., ; Song et al., ; Karras et al., ) have emerged as leading foundational models in image synthesis. Recent studies have developed their theoretical foundations (Lipman et al., ; Chen & Lipman, ; Song et al., ; Kingma et al., ) and sought to expand and improve the sampling and design space of these models (Song et al., ; Karras et al., ; Kingma et al., ). Other research has explored architectural innovations for diffusion models (Dhariwal & Nichol, ; Peebles & Xie, ), while some have focused on scaling these models for text-conditioned image synthesis and various real-world applications (Shi et al., ; Rombach et al., ; Podell et al., ). Efforts to accelerate the sampling process include approaches at the scheduler level (Karras et al., ; Lu et al., ; Song et al., ) and the training level (Meng et al., ; Song et al., ), with the former often aiming to improve the approximation of the probability flow ODE (Lu et al., ; Song et al., ). The latter primarily involves distillation techniques (Meng et al., ; Salimans & Ho, ; Wang et al., ) or initializing diffusion model weights for GAN training (Sauer et al., ; Lin et al., ).

Consistency Models. Consistency models are an emerging class of generative models (Song et al., ; Song & Dhariwal, ) for fast high-quality generation. It can be trained through either consistency distillation or consistency training. Advanced methods have demonstrated that consistency training can surpass diffusion model training in performance (Song & Dhariwal, ; Geng et al., ). Several studies propose different strategies for segmenting the ODE (Kim et al., ; Heek et al., ; Wang et al., ), while others explore combining consistency training with GANs to enhance training efficiency (Kong et al., ). Additionally, the consistency model framework has been applied to video generation (Wang et al., ; Mao et al., ), language modeling (Kou et al., ) and policy learning (Prasad et al., ).

## Appendix II Limitations

The work is limited to traditional benchmarks with CIFAR-10 and ImageNet-64 to validate the effectiveness of unconditional generation and class-conditional generation. However, previous works, including iCT (Song & Dhariwal, ) and ECT (Geng et al., ), only validate their effectiveness on these two benchmarks. We hope future research explores consistency training/tuning at larger scales such as text-to-image generation.

## Appendix III Qualitative Results

Figure: Figure 7: 1-step samples from class-conditional SCT trained on CIFAR-10. Each row corresponds to a different class.
Refer to caption: extracted/5952320/sec/samples/sample-1step-cond.png

Figure: Figure 8: 2-step samples from class-conditional SCT trained on CIFAR-10. Each row corresponds to a different class.
Refer to caption: extracted/5952320/sec/samples/sample-2step-cond.png

Figure: Figure 9: 1-step samples from unconditional SCT trained on CIFAR-10.
Refer to caption: extracted/5952320/sec/samples/sample-1step-uncond.png

Figure: Figure 10: 2-step samples from unconditional SCT trained on CIFAR-10.
Refer to caption: extracted/5952320/sec/samples/sample-2step-uncond.png

Figure: Figure 11: 4-step samples from unconditional SCT trained on CIFAR-10.
Refer to caption: extracted/5952320/sec/samples/sample-4step-uncond.png

Figure: Figure 12: 8-step samples from unconditional SCT trained on CIFAR-10.
Refer to caption: extracted/5952320/sec/samples/sample-8step-uncond.png

Figure: Figure 13: 1-step samples from class-conditional SCT trained on ImageNet-64 (FID 2.23). Each row corresponds to a different class.
Refer to caption: extracted/5952320/sec/samples/imgnet-1step-m.png

Figure: Figure 14: 2-step samples from class-conditional SCT trained on ImageNet-64 (FID 1.47). Each row corresponds to a different class.
Refer to caption: extracted/5952320/sec/samples/imgnet-2step-m.png

Figure: Figure 15: 4-step samples from class-conditional SCT trained on ImageNet-64 (FID 1.78). Each row corresponds to a different class.
Refer to caption: extracted/5952320/sec/samples/imgnet-4step-m.png

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: - What's the relationship between the MDP in section 3 and the methods in section 4?
- What's the most significant part for stable training?
soundness: 3
strengths: - The paper explores several techniques in the recent progress of diffusion models and improves the empirical performance of consistency models.
- The methods are technically correct and easy to follow.
- The experiments are solid.
summary: This paper analyzes the training bottlenecks of consistency models and proposes several improvements for better training of consistency models which are based on previous works of diffusion models, including a bias-variance trade-off training objective for consistency training; a smooth weighting function and reduced schedule of time spacing; a multistep version with edge-skipping and a better guidance techniques. Though all of these methods are not novel, the combination shows promising empirical results on CIFAR10 and ImageNet64.
weaknesses: Though this paper proposes several techniques, overall it is kind of incremental and strongly increases the complexity of the algorithm. It is hard to tell which part is important and which part is not; and it is hard to tell which part plays a crucial role for the "stable" claim.

- The analysis framework and the final conclusions in section 3 are well-known in the community, and the training variance and discretization errors are not new understanding aspects. The proposed MDP framework seems to be unrelated to the contents in section 4.
- The proposed methods in section 4 are based on previous techniques in diffusion models, which all bring small improvements and cannot fundamentally "stabilize" the training of consistency models.

---

confidence: 5
contribution: 2
initial_rating: 3
presentation: 2
questions: I have some questions to the authors:

1. SCT has faster convergence speed and better performance than ECT. Can you show the additional computation cost of SCT compared to ECT, for example in GFLOPs, since SCT requires the conditional epsilon estimation (Equation (14))?

2. To clarify: Figure 6 illustrates the effective of CFG in three difference choices of the sub-optimal version of EMA weights (at 25%, 50%, and 75% of the total iterations). Does it mean that the guiding technique is only applied starting from one of these pivot iterations?
soundness: 2
strengths: The proposed Edge-skipping mechanism (4.4) in inference for multistep CM explained through L-Lipschitz continuity of neuron networks sounds interesting.
summary: The paper views Consistency Models (CMs) as a Markov Decision Process (MDP) with learning objective as Temporal Difference (TD) comes from Reinforcement Learning (RL). Proposed method incorporates several techniques: variance-reduced training target (from STF[1]), smoothen timestep choices to reduce discretization errors (from ECT[2]), splitting time axis into multiple segments during training (from MCM[3], PCM[4]), guiding mechanism from [5], and edge-skipping inference for multistep model. In overall, the results have quite significant improvement in 1-step and 2-step evaluation.

[1] SFT: Stable target field for reduced variance score estimation in diffusion models

[2] ECT: Consistency Models Made Easy

[3] MCM: Multistep consistency models

[4] PCM: Phased Consistency Model

[5] Guiding a Diffusion Model with a Bad Version of Itself
weaknesses: While the paper shows improvements in results, it seems to lack novelty. As mentioned above, most factors incorporated into the final method are drawn from relevant works, with little or no modification.

1. Reducing the training variance (4.1) basically comes from SFT[1]

2. Reducing the discretization error (4.2) has already been presented in ECT[2]

3. Guiding consistency models with a bad version of itself (4.4) from [3] is applied with modification in choice of the sub-optimal version of network.

Regarding the experiments:

4. Edge-skipping inference (4.4) is an interesting technique to enhance sample quality in multistep CMs, comparison with previous works (MCM[4], PCM[5]) also conducted in this settings is required to demonstrate its advantages and potential.

Furthermore:

5. Consistency tuning initializes the model from a pretrained Diffusion Model (DM), providing it with a good starting point. In contrast, for consistency training, a pretrained DM is not available, which may lead to greater training variance. Experiments demonstrating the advantage of variance reduction in consistency training could enhance the paper.

[1] SFT: Stable target field for reduced variance score estimation in diffusion models

[2] ECT: Consistency Models Made Easy

[3] Guiding a Diffusion Model with a Bad Version of Itself

[4] MCM: Multistep consistency models

[5] PCM: Phased Consistency Model

---

confidence: 5
contribution: 2
initial_rating: 3
presentation: 2
questions: My biggest concern is the novelty of this work. More experiments are needed to verify the effectiveness of STF for the conditional consistency model. 
Since the author identifies training variance and discretization error as two main bottlenecks for consistency training/finetuning, the experiment only focuses on consistency tuning, which requires the pretrained diffusion model. There is no evidence that the proposed methods will work with consistent training. Furthermore, if the consistency model is initialized with a pretrained diffusion model, it already has low training variance. Therefore, it would be more interesting to see if STF works with consistency training.
soundness: 2
strengths: 1. Viewing the consistency model as MDP is interesting. 
2. Edge-skipping inference for MCM improves sampling FID.
summary: This paper shows the correspondence between the MDP and the consistency model, as they use the same temporal learning training objective. From the MDP view, they identify the performance bottlenecks in consistency tuning and consistency training, which are training variance and discretization error. For training variance, the paper adopts STF [1]. To reduce discretization error, the author uses a training scheduler from ECT [2] and a weighting technique from iCT [3]. Several sampling techniques, such as guided sampling and edge-skipping inference, are used in this paper.

[1]: Stable Target Field for Reduced Variance Score Estimation in Diffusion Models

[2]: ECT: Consistency Models Made Easy

[3]: Improved Techniques for Training Consistency Models
weaknesses: 1. The paper's novelty is limited. Since understanding CM as MDP is interesting, training variance and discretization error are already mentioned in previous work [1, 2].
3. Using STF for consistency training to tackle training variance seems straightforward. STF seems very hard to apply for text-to-image, limiting the application of this technique.
4. Techniques to reduce discretization error are also introduced from previous work [1,2].
5. Guiding consistency models from its bad version is from [3].
6. Lack of comparison between edge-skipping vs aDDIM from MCM [4].
7. The background of MCM should be provided since improved MCM is part of the method. The lack of proper background makes the paper hard to read.
8. Minor: there are typos in lines 217 and 218 (ground truth).

[1]: ECT: Consistency Models Made Easy

[2]: Improved Techniques for Training Consistency Models

[3]: Guiding a Diffusion Model with a Bad Version of Itself

[4]: Multistep Consistency Models

---

confidence: 4
contribution: 1
initial_rating: 1
presentation: 1
questions: I have no specific question -- cf. the above weaknesses for potential improvements of the paper. I recommend a "strong reject" and suggest the authors to submit a major revision to a later conference.
soundness: 1
strengths: The idea of **leveraging a noise estimator with lower variance**, taken from the literature of diffusion models, to improve consistency training is interesting. It tackles a well identified problem of consistency models and, based on the presented ablation study, seems to yield valuable results.
summary: This paper frames the learning of consistency models as state value learning in a Markov decision process (MDP), deducing two main performance bottlenecks in consistency training: the variance of the score/noise estimator and the temporal discretization error. The authors then deploy several strategies to tackle these two bottlenecks, notably leveraging a better noise estimator and changing the temporal schedule. The resulting consistency model outperforms the state of the art on standard benchmarks.
weaknesses: The paper presents too many important flaws, described below, to be considered for ICLR. Therefore, I strongly recommend rejecting the paper.

## Relevance of the MDP framework

The developed MDP framework is **disconnected from the rest of the contributions**. In the paper, the MDP view of consistency models provides no other insight related to the rest of the paper then than the identification of two main improvements directions -- variance reduction and alleviation of discretization errors. However, **this MDP view is not necessary** to identify these directions: it has already been clearly identified in prior work via more direct means.
- The discretization error is already identified in the original paper of Song et al. (2023, Theorem 2).
- Variance is already discussed by Song et al. (2023, Section 5), Song & Dhariwal (2023, Section 3.4) and Dou et al. (2023), for example. The variance problem can directly be identified as originating in the one-sample estimation of the score as already noted by Song et al. (2023, Section 5). It is the starting point of several research papers aiming to reduce this variance (Xu et al., 2023; Pooladian et al., 2023).

Dou et al. A Unified Framework for Consistency Generative Modeling. [ICLR 2024 submission](https://openreview.net/forum?id=Qfqb8ueIdy).\
Pooladian et al. Multisample flow matching: Straightening flows with minibatch couplings. ICML 2023.

## Limited novelty

The proposed solutions to the variance and discretization problems (Section 4) are **orthogonal and either already exist in the literature or are adaptions of existing techniques**. While combinations of existing methods can be acceptable, the contributions should be more developed, better motivated, and further supported by more experiments.

The most valuable novelty is the idea to use an already existing lower-variance estimator of the score/noise (Section 4.1) for consistency training. But this is insufficiently developed and tested. On the same topic, the paper highlights the adaption of this technique to conditional generation, but it is **straightforward**; furthermore, as the authors notice, it is not directly applicable to non-categorical conditioning.

## Unclear writing and unsupported claims

The writing of the paper is lacking in many aspects, even beyond the many typos (cf. next weakness) and leads several claims to be unsupported. **Section 3 is particularly unclear**, hurting the soundness of the MDP view.
- While Eq. (8) might be seen as a Bellman equation in a relevant setting, it is unclear to me why $h_\theta$ is a value function and $r$ a reward, especially as the latter is never optimized anywhere.
- The MDP was only developed for continuous-time consistency distillation at the beginning of the section, but is discussed in other settings (training and discretization) afterwards.
- Eq. (9), (11) and (12) lack context and development to be properly understood. They could be encompassed in small lemmas or propositions.
- The remark on the link with $n$-step temporal difference (line 198) is insufficiently explained.
- The introduction mentions performance bounds (lines 77 to 80), but they are nowhere to be found.

## Lacking finalization

Firstly, given the low significance of the contributions, **the experimental part of the paper deserves more focus**. Besides developing it, I would advice the authors to **release the source code** to ensure the paper's reproducibility.

Secondly, the paper suffers from too many mistakes or typos, making it hard to read, as shown in the following (possibly non-exhaustive) list.
- Mathematical problems:
  - $t$ and $r$ are not defined on line 81;
  - On line 114, the expectation should be over $x_0$ instead of $x_0 | x_t$;
  - On line 136, $t_\lambda$ should be the reverse function of $\lambda_t$, and $\mathbf{\epsilon}(\mathbf{x}\_{t\_\lambda, t})$ should be $\mathbf{\epsilon}(\mathbf{x}\_{t\_\lambda}, t\_\lambda)$;
  - In Eq. (7), $\alpha_s$ should be $\alpha_r$.
- Formatting:
  - the dataset is not specified for Figure 3's experiment;
  - I suspect that most of the figures are not properly readable in grayscale;
  - there should be no line between points in Figure 4's plot;
  - Figure 5 should be referenced in the corresponding paragraph ("Edge-skipping Multistep Sampling");
  - Tables 2 and 3's captions almost overlap;
  - lines 523 to 526 should be written using the correct font size;
  - it is not specified whether all CIFAR-10 experiments are conditional or unconditional;
  - a few references are duplicated (Song et al., 2020; Song et al., 2023).
- Typos/language:
  - on line 149: "weighg";
  - on line 154, "it is noting" is not correct English;
  - on line 188, "depent";
  - on line 196, "eough";
  - on line 202, "konwn";
  - on line 218, "gourd";
  - on lines 413 and 415, "LIPIPS";
  - on line 523, "a previous papers".