Bidirectional Consistency Models

## Abstract

Abstract Diffusion models (DMs) are capable of generating remarkably high-quality samples by iteratively denoising a random vector, a process that corresponds to moving along the probability flow ordinary differential equation (PF ODE).
Interestingly, DMs can also invert an input image to noise by moving backward along the PF ODE, a key operation for downstream tasks such as interpolation and image editing.
However, the iterative nature of this process restricts its speed, hindering its broader application.
Recently, Consistency Models (CMs) have emerged to address this challenge by approximating the integral of the PF ODE, largely reducing the number of iterations.
Yet, the absence of an explicit ODE solver complicates the inversion process.
To resolve this, we introduce Bidirectional Consistency Model (BCM), which learns a single neural network that enables both forward and backward traversal along the PF ODE, efficiently unifying generation and inversion tasks within one framework.
We can train BCM from scratch or tune it using a pretrained consistency model, wh ich reduces the training cost and increases scalability.
We demonstrate that BCM enables one-step generation and inversion while also allowing the use of additional steps to enhance generation quality or reduce reconstruction error.
We further showcase BCM’s capability in downstream tasks, such as interpolation, inpainting, and blind restoration of compressed images.
Notably, when the number of function evaluations (NFE) is constrained, BCM surpasses domain-specific restoration methods, such as I 2 SB and Palette, in a fully zero-shot manner, offering an efficient alternative for inversion problems.
Our code and weights are available at https://github.com/Mosasaur5526/BCM-iCT-torch .

## 1 Introduction

Two key components in image generation and manipulation are *generation* and its *inversion*.
Specifically, *generation* aims to learn a mapping from simple noise distributions, such as Gaussian, to complex ones, like the distribution encompassing all real-world images.
In contrast, *inversion* seeks to find the reverse mapping, transforming real data back into the corresponding noise (^1^11The inversion problem also refers to restoring a high-quality image from its degraded version. However, in this paper, we define it more narrowly as the task of finding the corresponding noise for an input image.).
Recent breakthroughs in deep generative models (Goodfellow et al., ; Kingma & Welling, ; Dinh et al., ; Song & Ermon, ; Ho et al., ; Song et al., ; ) have revolutionized this field.
These models have not only achieved remarkable success in synthesizing high-fidelity samples across various modalities (Karras et al., ; Rombach et al., ; Kong et al., ; OpenAI, ), but have proven effective in downstream applications, such as image editing, by leveraging the inversion process (Mokady et al., ; Huberman-Spiegelglas et al., ; Hertz et al., ).

Particularly, score-based diffusion models (DMs) (Song & Ermon, ; Ho et al., ; Song et al., ; ; Karras et al., ) have stood out for generation (Dhariwal & Nichol, ).
Starting from random initial noise, DMs progressively remove noise through a process akin to a numerical ODE solver operating over the probability flow ordinary differential equation (PF ODE), as outlined by Song et al. ().
However, it typically takes hundreds of iterations to produce high-quality generation results.
This slow generation process limits broader applications.

This issue has been recently addressed by consistency models (CMs) (Song et al., ; Song & Dhariwal, ), which directly compute the integral of PF ODE trajectory from any time step to zero.
Similar to CMs, Kim et al. () introduced the Consistency Trajectory Model (CTM), which estimates the integral between any two time steps along the trajectory towards the denoising direction.
Through these approaches, the consistency model family enables image generation with only a single Number of Function Evaluation (NFE, i.e., number of network forward passes) while offering a trade-off between speed and quality.

On the other hand, unfortunately, the inversion tasks remain challenging for DMs.
First, the generation process in many DMs (Ho et al., ; Karras et al., ) is stochastic and hence is non-invertible;
second, even for methods that employ a deterministic sampling process (Song et al., ), inverting the ODE solver necessitates at least hundreds of iterations for a small reconstruction error;
third, although CMs and CTM accelerate generation by learning the integral directly, this integration is strongly non-linear, making the inversion process even harder.

Therefore, in this work, we aim to bridge this gap through natural yet non-trivial extensions to CMs and CTM.
Specifically, they possess a key feature of self-consistency: points along the same trajectory map back to the same initial point.
Inspired by this property, we pose the questions:
{adjustwidth}2em1.5em
*Is there a form of stronger consistency where points on the same trajectory can map to each other, regardless of their time steps’ order?
Can we train a network to learn this mapping?*
In the following sections of this paper, we affirmatively answer these questions with our proposed *Bidirectional Consistency Model* (BCM).
Specifically:

- 1.
We train a single neural network that enables both forward and backward traversal (i.e., integration) along the PF ODE, unifying the generation and inversion tasks into one framework.
We can either train the model from scratch or fine-tune it from a pretrained consistency model to reduce training cost and increase scalability.
- 2.
We demonstrate BCM can generate images or invert a given image with a single NFE, and can achieve improved sample quality or lower reconstruction error by chaining multiple time steps.
- 3.
Leveraging BCM’s capability to navigate both forward and backward along the PF ODE trajectory, we introduce
two sampling schemes and a combined approach that has empirically demonstrated superior performance.
- 4.
We apply BCM for image interpolation, inpainting, and blind restoration of compressed images, showcasing its potential versatile downstream tasks.

## 2 Background and Preliminary

Figure: (a) DM
Refer to caption: x1.png

Before launching into the details of the Bidirectional Consistency Model (BCM), we first describe some preliminaries, including a brief introduction to Score-based Diffusion Models (DMs), Consistency Models (CMs), and Consistency Trajectory Model (CTM) in the following.
We also illustrate these models, along with our proposed method, in Figure 1.

Score-based Diffusion Models. Score-based Diffusion Models (DMs, Song et al., ) sample from the target data distribution by progressively removing noise from a random ${\mathbf{x}}_{T}\sim N(0,T^{2}{\bm{I}})$.
To achieve this, DMs first diffuse the data distribution $p_{\text{data}}({\mathbf{x}})$ through a stochastic differential equation (SDE):
$d{\mathbf{x}}_{t}=\bm{\mu}({\mathbf{x}}_{t},t)dt+\sigma(t)d{\mathbf{w}}_{t},$
which has a reverse-time SDE, as described by
Anderson ():
$d{\mathbf{x}}_{t}=\left[\bm{\mu}({\mathbf{x}}_{t},t)-\frac{1}{2}\sigma^{2}(t)
\nabla\log p_{t}({\mathbf{x}}_{t})\right]dt+\sigma(t)d\bar{{\mathbf{w}}}_{t},$
where $t\in[0,T]$(^2^22To avoid numerical issues, we always set $t$ in $[0.002,T]$ in practice. However, to keep the notation simple, we will ignore this small initial value $0.002$ when describing our methods in this paper.), $p_{t}$ is the marginal density of ${\mathbf{x}}_{t}$, and ${\mathbf{w}}_{t}$ and $\bar{{\mathbf{w}}}_{t}$ represents the standard Wiener process in forward and reverse time respectively.
Note, that $p_{0}$ is our desired data distribution $p_{\text{data}}$.
Remarkably, Song et al. () showed that there exists an ordinary differential equation, dubbed the Probability Flow (PF) ODE, whose solution trajectories have the same marginal density $p_{t}$ at time $t$:

$$ $\displaystyle{d{\mathbf{x}}_{t}}/{dt}=\bm{\mu}({\mathbf{x}}_{t},t)-1/2\sigma^{ 2}(t)\nabla\log p_{t}({\mathbf{x}}_{t}).$ (1) $$

During training, DMs learn to estimate $\nabla\log p_{t}({\mathbf{x}}_{t})$ with a score model $\bm{s}({\mathbf{x}}_{t},t)$ with score matching (Hyvärinen & Dayan, ; Song et al., ; Karras et al., ).
And during sampling, DMs solve the empirical PF ODE from time $T$ to 0 0 with a numerical ODE solver.
Following Karras et al. (), we set $\mu=0,\sigma=\sqrt{2t},T=80$.

Consistency Models, Consistency Training, and improved Consistency Training.
However, while the above procedure has proven effective in generating high-quality samples, the numerical ODE solver typically requires iterative evaluations of the network $\bm{s}({\mathbf{x}}_{t},t)$ and hence bottlenecks the generation speed.
To this end, Song et al. () proposed Consistency Models (CMs) that train a network to estimate the solution to PF ODE directly, i.e.,

$$ $\displaystyle\bm{f}_{\bm{\theta}}({\mathbf{x}}_{t},t)\approx{\mathbf{x}}_{0}={ \mathbf{x}}_{t}+{\textstyle\int_{t}^{0}}\left(\nicefrac{{d{\mathbf{x}}_{s}}}{{ ds}}\right)ds$ (2) $$

The network $\bm{f}_{\bm{\theta}}(\cdot,\cdot)$ can be either trained by distillation or from scratch with Consistency Training (CT).
Here, we describe the consistency training in more detail since it lays the foundation of our proposed method:
to begin, consistency training first discretizes the time horizon $[0,T]$ into $N-1$ sub-intervals, with boundaries $0=t_{1}<t_{2}<\dots<t_{N}=T$.
The training objective is defined as

$$ $\displaystyle\mathcal{L}_{CT}^{N}(\bm{\theta},\bm{\theta}^{-})=\mathbb{E}_{{ \mathbf{z}},{\mathbf{x}},n}[\lambda(t_{n})d(\bm{f}_{\bm{\theta}}({\mathbf{x}}+ {t_{n+1}}{\mathbf{z}},t_{n+1}),\bm{f}_{\bm{\theta}^{-}}({\mathbf{x}}+{t_{n}}{ \mathbf{z}},t_{n}))].$ (3) $$

$\bm{\theta}$ and $\bm{\theta}^{-}$ represents the parameters of the online network and a target network, respectively.
The target network is obtained by $\bm{\theta}^{-}\leftarrow\texttt{stopgrad}\left(\mu\bm{\theta}^{-}+(1-\mu)\bm{
\theta}\right)$ at each iteration.
$\lambda(\cdot)$ is a reweighting function, ${\mathbf{x}}$ represents the training data sample, and
${\mathbf{z}}\sim\mathcal{N}(0,{\bm{I}})$ is a random Gaussian noise.
During training, $N$ is gradually increased, allowing the model to learn self-consistency in an incremental manner.
Additionally, Song et al. () proposed to keep track of an exponential moving average (EMA) of the online parameter $\bm{\theta}$.
Specifically, after optimizing $\bm{\theta}$ in each iteration, CMs update the EMA by
$\bm{\theta}_{\text{EMA}}\leftarrow\mu_{\text{EMA}}\bm{\theta}_{\text{EMA}}+(1-
\mu_{\text{EMA}})\bm{\theta}.$
After training, CMs discard the online parameters and generate samples with $\bm{\theta}_{\text{EMA}}$.

Besides, in a follow-up work, Song & Dhariwal () suggested to use the Pseudo-Huber loss for $d$, along with other techniques that include setting $\mu=0$ (i.e., $\bm{\theta}^{-}\leftarrow\texttt{stopgrad}\left(\bm{\theta}\right)$), proposing a better scheduler function for $N$, adapting a better reweighting function $\lambda(t_{n})=\nicefrac{{1}}{{|t_{n}-t_{n+1}|}}$.
Dubbed improved Consistency Training (iCT), these modifications significantly improve the performance.
Therefore, unless otherwise stated, we inherit these improving techniques in our work.

Consistency Trajectory Model.
While CMs learn the integral from an arbitrary starting time to 0 0, Consistency Trajectory Model (CTM) (Kim et al., ) learns the integral between any two time steps along the PF ODE trajectory towards the denoising direction.
More specifically, CTM learns

$$ $\displaystyle\bm{f}_{\bm{\theta}}({\mathbf{x}}_{t},t,u)\approx{\mathbf{x}}_{u} ={\mathbf{x}}_{t}+{\textstyle\int_{t}^{u}}\left(\nicefrac{{d{\mathbf{x}}_{s}}} {{ds}}\right)ds,\quad\text{where }u\leq t.$ (4) $$

CTM demonstrates that it is possible to learn a stronger consistency:
two points ${\mathbf{x}}_{t}$ and ${\mathbf{x}}_{u}$ along the same trajectory not only can map back to the same initial point ${\mathbf{x}}_{0}$, but also can map from ${\mathbf{x}}_{t}$ to ${\mathbf{x}}_{u}$, provided $u\leq t$.
This inspires us for a stronger consistency with a bijection between ${\mathbf{x}}_{t}$ and ${\mathbf{x}}_{u}$.

However, it is worth noting that, while sharing some motivation, our methodology is foundationally different from that in CTM.
Our network adopts a different parameterization, which is a natural extension of that by EDM (Karras et al., ).
Moreover, CTM largely relies on adversarial loss (Goodfellow et al., ), while our proposed method, following Song & Dhariwal (), is free from LPIPS (Zhang et al., ) and adversarial training.

## 3 Methods

In this section, we describe details of Bidirectional Consistency Model (BCM).
From a high-level perspective, we train a network $\bm{f}_{\bm{\theta}}({\mathbf{x}}_{t},t,u)$ that traverses along the probability flow (PF) ODE from time $t$ to time $u$, i.e.,
$\bm{f}_{\bm{\theta}}({\mathbf{x}}_{t},t,u)\approx{\mathbf{x}}_{u}={\mathbf{x}}
_{t}+\int_{t}^{u}\frac{d{\mathbf{x}}_{s}}{ds}ds$.
This is similar to Equation 4, but since we aim to learn both generation and inversion, we do not set constraints on $t$ and $u$, except for $t\neq u$.
To this end, we adjust the network parameterization and the training objective of consistency training (Song et al., ; Song & Dhariwal, ).
These modifications are detailed in Sections 3.1 and 3.2.
Besides, we introduce new sampling schemes leveraging our model’s invertibility, which are described in Section 3.3.
Finally, we discuss BCM’s inversion in Section 3.4.

Figure: Algorithm 1 Bidirectional Consistency Training (Red indicates differences from CT/iCT (Song et al., ; Song & Dhariwal, )

### 3.1 Network Parameterization

We first describe the network parameterization.
Our network takes in three arguments: 1) the sample ${\mathbf{x}}_{t}$ at time $t$, 2) the current time step $t$, and 3) the target time step $u$, and outputs the sample at time $u$, i.e., ${\mathbf{x}}_{u}$.
To achieve this, we directly expand the models used in Consistency Models (CMs) (Song et al., ; Song & Dhariwal, ) with an extra argument $u$.
In CMs, the networks first calculate Fourier embeddings (Tancik et al., ) or positional embeddings (Vaswani et al., ) for the time step $t$, followed by two dense layers.
Here, we simply concatenate the embeddings of $t$ and $u$, and double the dimensionality of the dense layers correspondingly.

Similar to CMs (Song et al., ) and EDM (Karras et al., ), instead of directly learning $\bm{f}_{\bm{\theta}}$, we train $\bm{F}_{\bm{\theta}}$ and let $\bm{f}_{\bm{\theta}}({\mathbf{x}}_{t},t,u)=c_{\text{skip}}(t,u){\mathbf{x}}_{t
}+c_{\text{out}}(t,u)F_{\bm{\theta}}(c_{\text{in}}(t,u){\mathbf{x}}_{t},t,u)$, where

$$ $\displaystyle c_{\text{in}}(t,u)=\frac{1}{\sqrt{\sigma_{\text{data}}^{2}+t^{2} }},\quad c_{\text{out}}(t,u)=\frac{\sigma_{\text{data}}(t-u)}{\sqrt{\sigma_{ \text{data}}^{2}+t^{2}}},\quad c_{\text{skip}}(t,u)=\frac{\sigma_{\text{data}} ^{2}+tu}{\sigma_{\text{data}}^{2}+t^{2}}.$ (5) $$

Note that $c_{\text{skip}}(t,t)=1$ and $c_{\text{out}}(t,t)=0$, which explicitly enforce the boundary condition $\bm{f}_{\bm{\theta}}({\mathbf{x}}_{t},t,t)={\mathbf{x}}_{t}$.
We detail our motivation and derivations in Appendix D.

### 3.2 Bidirectional Consistency Training

We now discuss the training of BCM, which we dub as Bidirectional Consistency Training (BCT).
Following Song et al. (); Song & Dhariwal (), we discretize the time horizon $[0,T]$ into $N-1$ intervals with boundaries $0=t_{1}<t_{2}<\dots<t_{N}=T$.

Our training objective has two terms.
The first term takes the same form as Equation 3, enforcing the consistency between any points on the trajectory and the starting point.
We restate it with our new parameterization for easier reference:

$$ $\displaystyle\mathcal{L}_{CT}^{N}(\bm{\theta})=\mathbb{E}_{{\mathbf{z}},{ \mathbf{x}},t_{n}}[\lambda(t_{n})d(\bm{f}_{\bm{\theta}}({\mathbf{x}}+{t_{n+1}} {\mathbf{z}},t_{n+1},0),\bm{f}_{\bar{\bm{\theta}}}({\mathbf{x}}+{t_{n}}{ \mathbf{z}},t_{n},0))],$ (6) $$

where ${\mathbf{x}}$ is one training sample, ${\mathbf{z}}\sim\mathcal{N}(0,{\bm{I}})$, and $\bar{\bm{\theta}}$ is the stop gradient operation on $\bm{\theta}$, and $\lambda(t_{n})=\nicefrac{{1}}{{|t_{n}-t_{n+1}|}}$.
We replace $\bm{\theta}^{-}$ in Equation 3 with $\bar{\bm{\theta}}$ according to Song & Dhariwal ().

The second term explicitly sets constraints between any two points on the trajectory.
Specifically, given a training example ${\mathbf{x}}$, we randomly sample two time steps $t$ and $u$, and want to construct a mapping from ${\mathbf{x}}_{t}$ to ${\mathbf{x}}_{u}$, where ${\mathbf{x}}_{t}$ and ${\mathbf{x}}_{u}$ represent the results at time $t$ and $u$ along the Probability Flow (PF) ODE trajectory, respectively.
Note, that the model learns to denoise (i.e., generate) when $u<t$, and to add noise (i.e., inverse) when $u>t$.
Therefore, this single term unifies the generative and inverse tasks within one framework,
and with more $t$ and $u$ sampled during training, we achieve consistency over the entire trajectory.
To construct such a mapping, we minimize the distance

$$ $\displaystyle d\left(\bm{f}_{\bm{\theta}}({\mathbf{x}}_{t},t,u),{\mathbf{x}}_{ u}\right).$ (7) $$

However, Equation 7 will have different scales for different $u$ values, leading to a high variance during training.
Therefore, inspired by Kim et al. (), we map both $\bm{f}_{\bm{\theta}}({\mathbf{x}}_{t},t,u)$ and ${\mathbf{x}}_{u}$ to time 0 0, and minimize the distances between these two back-mapped images, i.e.,

$$ $\displaystyle d\left(\bm{f}_{\bar{\bm{\theta}}}(\bm{f}_{\bm{\theta}}({\mathbf{ x}}_{t},t,u),u,0)),\bm{f}_{\bar{\bm{\theta}}}({\mathbf{x}}_{u},u,0)\right),$ (8) $$

where $\bar{\bm{\theta}}$ is the same $\bm{\theta}$ with stop gradient operation.
We denote this as a “soft” trajectory constraint.
Unfortunately, directly minimizing Equation 8 without a pretrained DM is still problematic.
This is because, without a pretrained DM, we can only generate ${\mathbf{x}}_{t}$ and ${\mathbf{x}}_{u}$ from the diffusion SDE, i.e.,
by adding Gaussian noise to ${\mathbf{x}}$.
However, ${\mathbf{x}}_{t}$ and ${\mathbf{x}}_{u}$ generated by the diffusion SDE do not necessarily lie on the same PF ODE trajectory, and hence Equation 8 still fails to build the desired bidirectional consistency.
Instead, we notice that when the CT loss defined in Equation 6 converges, we have $\bm{f}_{\bar{\bm{\theta}}}({\mathbf{x}}_{u},u,0)\approx\bm{f}_{\bar{\bm{\theta
}}}({\mathbf{x}}_{t},t,0)\approx{\mathbf{x}}$.
We therefore optimize:

$$ $\displaystyle d\left(\bm{f}_{\bar{\bm{\theta}}}(\bm{f}_{\bm{\theta}}({\mathbf{ x}}_{t},t,u),u,0)),\bm{f}_{\bar{\bm{\theta}}}({\mathbf{x}}_{t},t,0)\right).$ (9) $$

Empirically, we found Equation 9 plays a crucial role in ensuring accurate inversion performance.
We provide experimental evidence for this loss choice in Section B.2.1.

Finally, we recognize that the term $\bm{f}_{\bar{\bm{\theta}}}({\mathbf{x}}+{t_{n}}{\mathbf{z}},t_{n},0)$ in Equation 6 and the term $\bm{f}_{\bar{\bm{\theta}}}({\mathbf{x}}_{t},t,0)$ in Equation 9 have exactly the same form.
Therefore, we set $t=t_{n}$ to reduce one forward pass.
Putting together, we define our objective as

$$ $\displaystyle\begin{split}\mathcal{L}_{BCT}^{N}(\bm{\theta})=\mathbb{E}_{{ \mathbf{z}},{\mathbf{x}},t_{n},t_{n^{\prime}}}\Big{[}\lambda(t_{n})&d(\bm{f}_{ \bm{\theta}}({\mathbf{x}}+{t_{n+1}}{\mathbf{z}},t_{n+1},0),\bm{f}_{\bar{\bm{ \theta}}}({\mathbf{x}}+{t_{n}}{\mathbf{z}},t_{n},0))\\ +\lambda^{\prime}(t_{n},t_{n^{\prime}})&d\left(\bm{f}_{\bar{\bm{\theta}}}(\bm{ f}_{\bm{\theta}}({\mathbf{x}}+t_{n}{\mathbf{z}},t_{n},t_{n^{\prime}}),t_{n^{ \prime}},0)),\bm{f}_{\bar{\bm{\theta}}}({\mathbf{x}}+t_{n}{\mathbf{z}},t_{n},0 )\right)\Big{]}\end{split}$ (10) $$

where we set reweighting as $\lambda^{\prime}(t_{n},t_{n^{\prime}})=\nicefrac{{1}}{{|t_{n}-t_{n^{\prime}}|}}$ to keep the loss scale consistent.

Training from scratch. We can train BCM from scratch using Equation 10.
We empirically found that all the training settings used for iCT (Song & Dhariwal, ), including the scheduler function for $N$, the sampling probability for $t_{n}$ (aka the noise schedule $p(n)$ in (Song et al., ; Song & Dhariwal, )), the EMA rate, and more, also works well for BCM.
Please refer to Section C.1 for more details on these settings and hyperparameters.
We summarize the training process in Algorithm 1 and compare the training of CT, CTM, and BCT in Table 2 in Appendix E.

Bidirectional Consistency Fine-tuning. To reduce the training cost and increase scalability, we can also initialize BCM with a pretrained consistency model and fine-tune it using Equation 10 to incorporate bidirectional consistency.
Recall that BCM takes in three arguments:
1) the sample ${\mathbf{x}}_{t}$ at time $t$, 2) the current time step $t$, and 3) the target time step $u$, and outputs the sample at time $u$, i.e., ${\mathbf{x}}_{u}$.
In contrast, the standard consistency model only takes the first two inputs and can be viewed as a special case of BCM when the target time step $u$ is set to 0 0.
Therefore, we aim to initialize BCM such that it preserves the performance of the pretrained CM when $u=0$.
To achieve this, we concatenate the embeddings of $t$ and $u$, then pass the result through a linear layer (without bias vector) to reduce the dimensionality by half.
The weight matrix is initialized as $[{\mathbf{I}},\bm{0}]$.
The embedding layers for $t$ and $u$ are initialized using the pretrained CM’s embedding layer, but they are not tied together during fine-tuning.
Other layers are initialized identically to those of the CM.
It is easy to check that this initialization effectively ignores $u$, preserving the CM’s generation performance without fine-tuning. See Section C.1 for more details.

### 3.3 Sampling

In this section, we describe the sampling schemes of BCM.
Similar to CMs and CTM, BCM supports 1-step sampling naturally.
However,
BCM’s capability to navigate both forward and backward along the PF ODE trajectory allows us to design more complicated multi-step sampling strategies to improve the sample quality.
We present two schemes and a combined approach that has empirically demonstrated superior performance in the following.

Ancestral Sampling.
The most straightforward way for multi-step sampling is to remove noise sequentially.
Specifically, we first divide the time horizon $[0,T]$ into $N$ sub-intervals with boundaries $0=t_{0}<t_{1}<\dots<t_{N}=T$.
Then, we sample a noise image ${\mathbf{x}}_{T}\sim\mathcal{N}(0,T^{2}{\bm{I}})$, and sequentially remove noise with the network:
${\mathbf{x}}_{t_{n-1}}\leftarrow\bm{f}_{\bm{\theta}}({\mathbf{x}}_{t_{n}},t_{n
},t_{n-1}),\quad n=N,N-1,...,1$.
We note that the discretization strategy may differ from the one used during the training of BCM.
Since this sampling procedure can be viewed as drawing samples from the conditional density
$p_{{\mathbf{x}}_{t_{n-1}}|{\mathbf{x}}_{t_{n}}}({\mathbf{x}}_{t_{n-1}}|{
\mathbf{x}}_{t_{n}})$, we dub it as ancestral sampling, and summarize it in Algorithm 2.
We can also view 1-step sampling as ancestral sampling, where we only divide the time horizon into a single interval.

Zigzag Sampling.
Another effective sampling method (Algorithm 1 in Song et al. ()) is to iteratively re-add noise after denoising.
Similar to ancestral sampling, we also define a sequence of time steps $t_{1}<\dots<t_{N}=T$.
However, different from ancestral sampling where we gradually remove noise, we directly map ${\mathbf{x}}_{T}$ to ${\mathbf{x}}_{0}$ by $\bm{f}_{\bm{\theta}}({\mathbf{x}}_{T},T,0)$.
We then add a fresh Gaussian noise to ${\mathbf{x}}_{0}$, mapping it from time 0 0 to time $t_{n-1}$, i.e.,
${\mathbf{x}}_{t_{n-1}}={\mathbf{x}}_{0}+t_{n-1}\bm{\sigma}$, where $\bm{\sigma}\sim\mathcal{N}(0,{\bm{I}})$.
This process repeats in this zigzag manner until all the designated time steps are covered.
The two-step zigzag sampler effectively reduces FID in CMs (Song et al., ) and is theoretically supported (Lyu et al., ).
However, the fresh noise can alter the content of the image after each iteration, which is undesirable, especially considering that our tasks will both include generation and inversion.
One may immediately think that setting the injected noise to be the same as the initial random noise can fix this issue.
However, we reveal that this significantly damages the quality of the generated images.

Fortunately, our proposed BCM provides a direct solution, leveraging its capability to traverse both forward and backward along the PF ODE.
Specifically, rather than manually reintroducing a large amount of fresh noise, we initially apply a small amount and let the network amplify it.
In a nutshell, for iteration $n$ ($n=N,N-1,\cdots,2$), we have

$$ $\displaystyle{\mathbf{x}}_{0}\leftarrow\bm{f}_{\bm{\theta}}({\mathbf{x}}_{t_{n }},t_{n},0),\quad{\mathbf{x}}_{\varepsilon_{n-1}}\leftarrow{\mathbf{x}}_{0}+ \varepsilon_{n-1}\bm{\sigma},\quad{\mathbf{x}}_{t_{n-1}}\leftarrow\bm{f}_{\bm{ \theta}}({\mathbf{x}}_{\varepsilon_{n-1}},\varepsilon_{n-1},t_{n-1}).$ (11) $$

where $\varepsilon_{n-1}$ is the scale of the small noises we add in $n$-th iteration, and $\bm{\sigma}\sim\mathcal{N}(0,{\bm{I}})$ is a fresh Gaussian noise.
We detail this scheme in Algorithm 3.
To verify its effectiveness, in Figure 2, we illustrate some examples to compare the generated images by 1) manually adding fresh noise, 2) manually adding fixed noise, and 3) our proposed sampling process, i.e., adding a small noise and amplifying it with the network.
We can clearly see our method maintains the generated content.

Figure: (a)
Refer to caption: x5.png

Combination of Both.
Long jumps along the PF ODE in zigzag sampling can lead to accumulative errors, especially at high noise levels, which potentially hampers further improvements in sample quality.
Therefore, we propose a combination of ancestral sampling and zigzag sampling.
Specifically, we first perform ancestral sampling to rapidly reduce the large initial noise to a more manageable noise scale and then apply zigzag sampling within this small noise level.
We describe this combined process in Algorithm 4.
We empirically found this combination results in superior sample quality compared to employing either ancestral sampling or zigzag sampling in isolation.
We provide ablation to evidence the effectiveness of this combination in Section B.2.3.

### 3.4 Inversion

BCM inverts an image following the same principle of sampling.
Specifically, we also set an increasing sequence of noise scales $\varepsilon=t_{1}<t_{2}<\cdots<t_{N}\leq T$.
Note that, in contrast to the generation process, it is not always necessary for $t_{N}$ to equal $T$. Instead, we can adjust it as a hyperparameter based on the specific tasks for which we employ inversion.
Then, given an image ${\mathbf{x}}_{0}$,
we first inject a small Gaussian noise by ${\mathbf{x}}_{t_{1}}={\mathbf{x}}_{0}+\varepsilon\bm{\sigma}$, and then sequentially add noise with the network, i.e.,
${\mathbf{x}}_{t_{n+1}}=f({\mathbf{x}}_{t_{n}},t_{n},t_{n+1}),\quad n=1,2,
\cdots,N-1.$
The adoption of small initial noise is due to the observation that the endpoint of the time horizon is less effectively covered and learned during training, as discussed in Section B.2.4.
Empirically, we find this minor noise does not change the image’s content and leads to lower reconstruction errors when $\varepsilon\approx 0.07$.
One may also include denoising steps interleaved with noise magnifying steps, like zigzag sampling, but we find it helps little in improving inversion quality.
We summarize the inversion procedure in Algorithm 5.

## 4 Experiments and Results

In this section, we first present the results of the two basic functions of BCM, i.e., generation and inversion.
Then in Section 4.3, we show some potential applications enabled by these two functions.

### 4.1 Image Generation

Figure: Table 1: Sample quality on CIFAR-10 (left) and ImageNet-64 (right). We train BCM from scratch on CIFAR-10 and fine-tune it using our reproduced iCT model on ImageNet-64. ^∗Results estimated from Figure 13 in Kim et al. (). ^†For our BCM and BCM-deep, we use ancestral sampling when NFE=2, zigzag sampling when NFE=3, and the combination of both when NFE=4. Our results indicate that ancestral and zigzag sampling can individually improve FID, and their combination can achieve even better performance. ^∗∗Results by our reproduction.
Refer to caption: extracted/5889717/Figures/generation/onestep-deep-main.png

We first evaluate the image generation performance of BCM.
On CIFAR-10, we train our BCM from scratch, while on ImageNet-64, we first train an iCT model, and fine-tune BCM from it, as we discussed in Section 3.2.
When reproducing iCT on ImageNet-64, we encountered instability during training, and found it difficult to reproduce the reported FID by Song & Dhariwal ().
To mitigate the performance gap, we removed the attention layer at the 32x32 resolution and added normalization to the QKV attention, following (Karras et al., ).
We empirically find these simple modifications bring positive influences on performance.
We include the hyperparameters for training and sampling in Sections C.1 and C.2.
We report image quality and NFE for CIFAR-10 and ImageNet-64 in Table 1, and we visualize some generated samples
Figures 3, 4 and H.
As we can see,

- •
both ancestral sampling and zigzag sampling can improve the sample quality, while the combination of both further yields the best performance;
- •
our proposed BCM achieves comparable or even better results within at least one order of magnitude fewer NFEs compared with diffusion models;
BCM presents better results with even fewer NFEs then the methods boosting sampling speed through fast samplers;
it is also comparable to modern distillation approaches like EM distillation by Xie et al. ();
- •
comparing with CMs, iCT (Song & Dhariwal, ) surpasses BCM for 1-step sampling.
This is not surprising, given that BCMs tackle a significantly more complex task compared to CMs.
However, as the number of function evaluations (NFEs) increases, following the sampling strategies detailed in Section 3.3, BCM starts to outperform.
This indicates that *our approach allows the model to learn bidirectional consistency without hurting the generation performance*;
- •
our model’s performance still falls short of CTM’s (Kim et al., ).
However, CTM relies heavily on adversarial loss and uses LPIPS (Zhang et al., ) as the distance measure, which can have feature leakage (Song & Dhariwal, ; Kynkäänniemi et al., ).
ECT (Geng et al., ) also outperforms BCM on ImageNet-64 with 2 NFEs.
However, we note that their approach—fine-tuning from a pretrained diffusion and using the advanced EDM2 architecture (Karras et al., )—is orthogonal to ours.
Our method could be combined with theirs for potential improvements in both performance and training speed.

Another interesting phenomenon we observed on ImageNet-64 is that our fine-tuned BCM actually outperforms its initialization (i.e., our pretrained iCT) on generation.
In contrast, we empirically find fine-tuning iCT with the same training budget did not yield any improvement.
We suspect this is due to the “soft” trajectory constraint in the bidirectional training objective, which may act as a form of regularization, aiding optimization.
We leave further investigation of this for future work.

### 4.2 Inversion and Reconstruction

We evaluate BCM’s capability for inversion on CIFAR-10 and ImageNet-64, and report the per-dimension mean squared error (scaled to [0,1]) in Figure 7.
We also present ODE-based baselines, including DDIM (Song et al., ) and EDM (Karras et al., ) for comparison.
The hyperparameters for inversion are tuned on a small subset of the training set and are discussed in Section C.2.
We can see BCM achieves a lower reconstruction error than ODE-based diffusion models, with significantly fewer NFEs, on both data sets.

We visualize the noise generated by BCM at Figure 16 in Appendix G, from which we can see that BCM Gaussianizes the input image as desired.
We also provide examples of the reconstruction in Figure 17 in Appendix G.
We observe that using only 1 NFE for inversion sometimes introduces slight mosaic artifacts, possibly because both endpoints of the time horizon are less effectively covered and learned during training.
Fortunately, the artifact is well suppressed when using more than 1 NFEs.
Also, we find that the inversion process can occasionally alter the image context.
However, it is important to note that this is not unique to BCM but also occurs in ODE-based diffusion models, such as EDM (see Figures 17(f) and 17(g) for examples), even though they employ more NFEs.

### 4.3 Applications with Bidirectional Consistency

Figure: (a) CIFAR-10.
Refer to caption: x9.png

Figure: Figure 8: JPEG (QF20) restoration with I^2SB, Palette and BCM on ImageNet-64.
Refer to caption: x11.png

Since BCM inherits the consistency of CMs, it
can handle the applications introduced by Song et al. () in the same way,
including colorization, super-resolution, stroke-guided image generation, denoising, and interpolation between two *generated* images.
However, BCM’s unique bidirectional consistency can enhance or enable more applications.
In this section, we showcase that BCM can interpolate two *real* images and achieve superior inpainting quality with fewer NFEs.
We also demonstrate its downstream applications for blind restoration of compressed images.

Interpolation.
While CMs can perform interpolations between two *generated* images (Song et al., ), our BCM can interpolate between two given *real* images, which is a more meaningful application.
Specifically, we can first invert the given images into noises, smoothly interpolate between them, and then map the noises back to images.
We illustrate some examples in Figure 7.
Here, we note a caveat in the implementation of BCM’s interpolation:
recall that when inverting an image, we first inject a small initial noise, as described in Section 3.4.
In the context of interpolation, we find it crucial to inject *different* initial noises for each of the two given images to avoid sub-optimal results, as illustrated in Figure 19.
We defer further discussions to Section C.3.1.

Inpainting.
BCM’s bidirectional consistency can also help with inpainting:
having an image with some pixels missing,
we first add a small noise to the missing pixels and then invert the image by multi-step inversion.
Different from our standard inversion algorithm in Algorithm 5, at each iteration, we manually replace the region of missing pixels with a new Gaussian noise, whose scale corresponds to the current time step.
In this way, we gradually fill in the missing region to in-distribution noise.
In the end, we map the entire noisy image back to time 0 0, finishing the inpainting.
We summarize this process in Algorithm 6, and include more details on the hyperparameters in Section C.3.2.
We illustrate our BCM inpainting results on CIFAR-10 in Figure 7, with the CM’s inpainting results using the algorithms proposed by Song et al. () (Algorithm 4 in (Song et al., )) for comparison.
Our method produces better inpainting results within much fewer NFEs.

Blind restoration of
compressed image. BCM’s bidirectional consistency can offer broader applications.
For example, we can invert an image with OOD artifacts to the noise space and re-generate the image from the noise.
During inversion, the model maps OOD regions to in-distribution noise, effectively eliminating these artifacts.
One potential application leveraging this property is blind restoration of compressed image, where OOD artifacts originate from compression.
We prove this concept by applying BCM to restore images compressed by JPEG.
We present the results in Figure 8, and also include two established restoration approaches, I^2SB (Liu et al., ) and Palette (Saharia et al., ), for comparison.
We include hyperparameters for our approach and these baselines in Section C.3.3.
Surprisingly, BCM achieves better performance compared to both I^2SB and Palette when NFE is limited.
Note that I^2SB and Palette require paired data under known degradation for training, while BCM is fully zero-shot and blind.
This result strongly evidences the potential of BCM in downstream inversion tasks.

## 5 Related Works

Accelerating Diffusion’s Generation.
In addition to CM and CTM we discussed in Section 2, many attempts have also been made to accelerate the generation of DMs.
Existing methods include faster ODE solvers (Song et al., ; Zhang & Chen, ; Lu et al., ; Zhou et al., ) and distillation (Salimans & Ho, ; Luhman & Luhman, ; Zheng et al., ; Luo et al., ; Xie et al., ).
However, these ODE solvers generally require 5-10 steps for satisfactory samples, and the distillation approach may cap the performance at the level of the pretrained diffusion model, as highlighted by Song & Dhariwal ().
On the other hand, while distillation approaches have achieved impressive generation quality, an intrinsic property of CMs is their ability to enable fast traversal along ODEs.
This allows for the design of broader applications; for example, Zhang et al. () leverage this property to develop an efficient importance sampler.
Our proposed BCM further enriches this by enabling both forward and backward traversal.

Inversion.
Inversion of an input image is a key step in downstream tasks like image editing (Mokady et al., ; Wallace et al., ; Hertz et al., ; Huberman-Spiegelglas et al., ).
Current methods typically include encoders (Richardson et al., ; Tov et al., ), optimization (Mokady et al., ; Abdal et al., ), and model fine-tuning or modulation (Roich et al., ; Alaluf et al., ).
For diffusion-based models, the most common inversion method is based on solving PF ODE backward, as first proposed by Song et al. ().
However, since the ODE solver relies on local linearization, this ODE-based inversion typically provides an insufficient reconstruction, especially when using fewer diffusion steps.
One way to address this issue is to perform gradient optimization using the ODE-based inversion as an initialization, as introduced by Mokady et al. ().
Our proposed BCM can provide better inversion with fewer NFEs and is compatible with Mokady et al. ()’s approach for a better reconstruction quality.

## 6 Conclusions and Limitations

In this work, we introduce the Bidirectional Consistency Model (BCM), enhancing upon existing consistency models (Song et al., ; Song & Dhariwal, ; Kim et al., ) by establishing a stronger consistency.
This consistency ensures that points along the same trajectory of the probability flow (PF) ODE map to each other, thereby unifying generation and inversion tasks within one framework.
By exploiting its bidirectional consistency, we devise new sampling schemes and showcase applications in a variety of downstream tasks.
Unlike other distillation approaches, a notable property of CMs is their ability to enable fast traversal along the entire PF ODE.
Our proposed BCM further enriches this, and we believe that it will open a new avenue for further exploration.

One limitation of our method is that while employing more steps in generation or inversion can initially enhance results, the performance improvements tend to plateau quickly.
Increasing the Number of Function Evaluations (NFEs) beyond a certain point does not yield further performance gains.
This is similar to CMs, where there is no performance gain with more than 2 NFEs.
A potential solution involves employing the parameterization and tricks proposed by Kim et al. ().
Additionally, our method delivers imperfect inversion, which sometimes alters the image content.
However, we should note that this also happens in ODE-based DMs.
Future work can involve developing more accurate inversion techniques, like the approach by Wallace et al. ().

## Broader Impact and Ethics Statements

BCM can accelerate the generation and inversion of diffusion models, which may pose a risk of generating or editing images with harmful or inappropriate content, such as deepfake images or offensive material.
Strong content filtering or even regulatory rules are required to prevent the creation of unethical or harmful content.

## Acknowledgment

We are grateful to Zongyu Guo for the helpful discussion throughout this project, especially for pointing out the significance of the inverse process, suggesting some potential applications, and proofreading the preprint version of this work.
We also thank Wenlin Chen and Sergio Calvo-Ordoñez for reviewing and providing feedback on the manuscript.
JH acknowledges support from Prof. José Miguel Hernández-Lobato’s Turing AI Fellowship under grant EP/V023756/1.

## Appendix A Algorithms

Figure: Algorithm 2 BCM’s ancestral sampling

Figure: Algorithm 5 BCM’s inversion

Figure: Algorithm 6 BCM’s inpainting

Figure: Algorithm 7 BCM’s inpainting with refinement

## Appendix B Additional Experiments

### B.1 Additional Applications with Bidirectional Consistency

Here, we provide more demonstrations of BCM’s application.

#### B.1.1 Inpainting on Higher-Resolution Images

Figure: (a) FFHQ $64\times 64$
Refer to caption: extracted/5889717/Figures/inpaint/inpaint_ffhq_small.png

In the main text, we demonstrate that BCM can yield better inpainting results with fewer NFEs on CIFAR-10 images compared with CM’s inpainting algorithm proposed by Song et al. ().
On high-resolution images, however, we find that simply following the same algorithm as CIFAR-10 (Algorithm 6) is suboptimal, possibly because the correlation between pixels is smaller than that in low-resolution.
Fortunately, we find Algorithm 6 can quickly fill the missing region with imperfect but sensible results.
Therefore, we can add a few more steps to refine the inpainting results by iteratively adding some small noise to the inpainting regions and denoising by BCM.
We summarize this algorithm in Algorithm 7.
This can be viewed as a combination of BCM’s inpainting by Algorithm 6 with CM’s inpainting algorithm proposed by Song et al. ().
We observe that this combination can reduce the NFEs without hurting the inpainting performance.

We evaluate Algorithms 6 and 7 on FFHQ $64\times 64$ and LSUN Bedroom $256\times 256$ in Figure 9 and also compare the results of CM’s inpainting. The hyperparameters are included in Section C.3.2.
As we can see: 1) simply adopting BCM’s inpainting as described in Algorithm 6 can fill in relatively sensible content but is imperfect in the overall color and details; 2) CM’s inpainting algorithm yields satisfactory performance with significantly more NFEs; and 3) applying BCM’s inpainting with refinement as described in Algorithm 7 can achieve a comparable performance with fewer NFEs.

### B.2 Additional Ablation Studies and Conclusions

Figure: Figure 10: Tracking the loss with different objection functions. We include the loss curve of iCT (Song & Dhariwal, ) for reference. We can see that the model with BCT loss defined in Equation 10 converges well. Conversely, the model applying Equation 8 instead of Equation 9 for the soft constraint has a much higher loss at the end of the optimization. While the one without CT loss totally diverges.
Refer to caption: x12.png

#### B.2.1 Comparison between Loss defined with Equation 8 and Equation 9

Figure: (a) original images
Refer to caption: extracted/5889717/Figures/ablation/original_eq13.png

In Section 3.2, we discussed that we optimize Equation 9 instead of Equation 8.
Here we provide experimental evidence for our design choice.

We track the loss by models trained with both choices in Figure 10, where we can see that the model trained with Equation 8 features a much higher loss in the end.
This echoes its failure in the inversion process:
as shown in Figure 11, the model trained with Equation 8 fails to provide an accurate inversion.
This is because Equation 8 contains two trajectories, starting from ${\mathbf{x}}_{u}$ and ${\mathbf{x}}_{t}$.
While both of them are along the SDE trajectories starting from the same ${\mathbf{x}}_{0}$, they do not necessarily reside on the same PF ODE trajectory;
in fact, the probability that they are on the same PF ODE trajectory is 0 0.
On the contrary, Equation 9 bypasses this issue since it only involves trajectories starting from the same ${\mathbf{x}}_{t}$.

#### B.2.2 Ablation of CT loss

Recall our final loss function has two terms, the soft trajectory constraint term and the CT loss term.
We note that the soft constraint defined in Equation 9 can, in principle, cover the entire trajectory, so it should also be able to learn the mapping from any time step $t$ to 0 0, which is the aim of CT loss.
However, we find it crucial to include CT loss in our objective.
We provide the loss curve trained without CT loss term in Figure 10, where we can see the training fails to converge.
For further verification, we also visualize the images generated by the model trained with full BCM loss and the model trained without CT loss term after 200k iterations in Figure 12.
We can clearly see that the model without CT loss cannot deliver meaningful outcomes.

Figure: (a)
Refer to caption: extracted/5889717/Figures/ablation/bct_loss.png

#### B.2.3 Albation of different sampling schemes

In this section, we compare image generation quality using different sampling schemes we proposed in Section 3.3.
Specifically, we compare FID v.s. NFE on CIFAR-10 using ancestral sampling, zigzag sampling and their combination in Figure 13.
As we can see, both zigzag sampling and ancestral sampling offer performance gain over single-step generation.
However, as NFE increases, zigzag sampling and ancestral sampling cannot provide further improvement in generation quality.
This phenomenon has also been discussed by Kim et al. ().
However, the combination of both can yield better FID than either of them in isolation.

Figure: Figure 13: Comparing zigzag sampling, ancestral sampling, and their combination on CIFAR-10. Both zigzag sampling and ancestral sampling offer performance gain over single-step generation, where their combination is better than either in isolation.
Refer to caption: x13.png

#### B.2.4 Coverage of Trajectory during Training

Figure: (a)
Refer to caption: x14.png

In Figure 14, we visualize the probability of selecting a $(t_{n},t_{n^{\prime}})$ pair during the BCT process, where $(t_{n},t_{n^{\prime}})$ is defined in Equation 10.
This offers insights into how well the entire trajectory is covered and trained.
We can see that most of the probability mass is concentrated in small-time regions.
This reveals some of our observations in the experiments:

- •
first, in the generation process, we find the combination of a zigzag and ancestral sampling yields optimal performance because the ancestral sampler can rapidly jump over the large-time regions, which we cover less in training;
- •
second, it also explains why adding a small initial noise in inversion helps:
simply adding a small $\varepsilon\approx 0.085$ noise increases the probability of being selected during BCT by more than a thousandfold;
- •
third, it offers insights into the necessity of incorporating CT loss into our final objective, as defined in Equation 10:
while theoretically, the soft constraint is expected to cover the entire trajectory, including boundary conditions, it is highly inefficient in practice.
Therefore, explicitly including CT loss to learn the mapping from any noise scale $t$ to 0 0 is crucial.

This also points out some future directions to improve BCM.
For example, we can design a better sampling strategy during training to ensure a better coverage of the entire trajectory.
Or using different sampling strategy for CT loss term and the soft trajectory constraint term.

## Appendix C Experiment Details

In this section, we provide the experiment details omitted from the main paper.

### C.1 Training Settings

CIFAR-10.
For all the experiments on CIFAR-10, following the optimal settings in Song & Dhariwal (), we use a batch size of 1,024 with the student EMA decay rate of $0.99993$, scale parameter in Fourier embedding layers of 0.02, and dropout rate of 0.3 for 400,000 iterations with RAdam optimizer Liu et al. () using learning rate $0.0001$.
We use the NCSN++ network architecture proposed by Song et al. (), with the modification described in Section 3.1. In our network parameterization, we set $\sigma_{\text{data}}=0.5$ following Karras et al. () and Song et al. ().
Regarding other training settings, including the scheduler function for $N(\cdot)$, the sampling probability for $t_{n}$ (aka the noise schedule $p(n)$ in (Song et al., ; Song & Dhariwal, )), the distance measure $d(\cdot,\cdot)$, we follow exactly Song & Dhariwal (), and restate below for completeness’s sake:

$$ $\displaystyle d({\mathbf{x}},{\mathbf{y}})=\sqrt{||x-y||^{2}+c^{2}}-c,\quad c= 0.00054\sqrt{d},\quad d\text{ is data dimensionality},$ (12) $\displaystyle p(n)\propto\text{erf}\left(\frac{\log(t_{n+1}-P_{\text{mean}})}{ \sqrt{2}P_{\text{std}}}\right)-\text{erf}\left(\frac{\log(t_{n}-P_{\text{mean} })}{\sqrt{2}P_{\text{std}}}\right),\quad P_{\text{mean}}=-1.1,\quad P_{\text{ std}}=2.0,$ (13) $\displaystyle N(k)=\min(s_{0}2^{\lfloor\nicefrac{{k}}{{K^{\prime}}}\rfloor},s_ {1})+1,\quad s_{0}=10,\quad s_{1}=1280,\quad K^{\prime}=\left\lfloor{\frac{K}{ \log_{2}[s_{1}/s_{0}]+1}}\right\rfloor,$ (14) $K$ is the total training iterations $\displaystyle t_{n}=\left(t_{\text{min}}^{\nicefrac{{1}}{{\rho}}}+\frac{n-1}{N (k)-1}\left(t_{\text{max}}^{\nicefrac{{1}}{{\rho}}}-t_{\text{min}}^{\nicefrac{ {1}}{{\rho}}}\right)\right)^{\rho},\quad t_{\text{min}}=0.002,\quad t_{\text{ max}}=80,\quad\rho=7.$ (15) $$

FFHQ $64\times 64$ and LSUN Bedroom $256\times 256$. Except the differences stated below, we train BCMs on FFHQ $64\times 64$ and LSUN Bedroom $256\times 256$ using exactly the same settings:

- •
For FFHQ $64\times 64$, we use a batch size of 512 but still train the model for 400,000 iterations.
- •
For LSUN Bedroom $256\times 256$, we (i) use a dropout rate of 0.2, (ii) adopt the network architecture configuration on this dataset in Song et al. (), (iii) use a batch size of 128, and (iv) adopt a different discretization curriculum from Equation 14.
Specifically, in Equation 14, the training procedure is divided into 8 stages.
Each stage uses a different $N$ and has $K/8$ iterations.
In our experiments, we still divide the entire training procedure into 8 stages, and still set $N(k)=\min(s_{0}2^{j-1},s_{1})+1$ at stage $j$, for $j=1,2,\cdots,8$.
However, we do not divide these stages evenly.
Instead, we assign 75,000, 60,000, 45,000, 30,000 iterations to the 1st/2nd stage, 3rd/4th stage, 5th/6th stage, 7th/8th stage, respectively
(^3^33We assign more iterations to earlier stages with small $N$, as we empirically find that it yields better performance *given limited computing budget*.).
The total number of iterations is 420,000.

Moreover, since our experiments include many applications that are necessarily conducted on test sets (e.g., inpainting), we split FFHQ into a training set of 69,000 images and a test set containing the last 1,000 images. For LSUN Bedroom, we use its official validation set as test set.
We implement our model and training algorithm based on the codes released by Song et al. () at [https://github.com/openai/consistency_models_cifar10](https://github.com/openai/consistency_models_cifar10) (Apache-2.0 License).

We specially emphasize that the settings we use to train FFHQ $64\times 64$ and LSUN Bedroom $256\times 256$ are not fine-tuned to optimal (in fact, most of them have not been tuned at all).
For instance, Song et al. () trained LSUN Bedroom $256\times 256$ with a significantly larger batch size of 2,048 for 1,000,000 iterations, meaning that their model has seen more than $38\times$ samples than ours during training.
Therefore, we note that our settings for these two datasets should NOT be used as guidance for training the models to optimal performance, and our results are only for demonstration purposes and should NOT be directly compared with other baselines unless under aligned settings.

ImageNet-64.
Instead of training from scratch, we apply bidirectional consistency fine-tuning on ImageNet-64.
Specifically, we first train an iCT on ImageNet-64, and then initialize and fine-tune BCM from it.
Since there is no official implementation or available checkpoints for iCT, we reproduce it by ourselves.
Therefore, we first state our training settings for iCT:

Reproducing iCT on ImageNet-64.
Our training setting follows Song & Dhariwal () closely.
Most of the training parameters for ImageNet-64 are consistent with those used for training BCM on CIFAR-10, with the exceptions noted below:

- •
We use a batch size of 4096 and train the model for 800,000 iterations.
- •
The EMA decay rate is set to 0.99997.
- •
We adopt the ADM architecture and remove AdaGN following Song & Dhariwal ().
- •
Instead of Fourier Embeddings, we employ the default positional embedding.
- •
The dropout rate is set to 0.2 and only applied to convolutional layers whose feature map resolution is smaller or equal to $16\times 16$.
- •
We use mix-precision training.

Additionally, when reproducing iCT on ImageNet-64, following EDM2 (Karras et al., ), we slightly modify the network architecture as follows:

- •
The self-attention layer at resolution $32\times 32$ is removed.
- •
The $Q,K$ and $V$ vectors are normalized in the self-attention layers.

We empirically find these simple modifications bring positive influences on performance.

Fine-tuning BCM on ImageNet-64.
We initialize our BCM with the pretrained iCT weights and fine-tune it using Equation 10.
Recall that BCM takes in three arguments:
1) the sample ${\mathbf{x}}_{t}$ at time $t$, 2) the current time step $t$, and 3) the target time step $u$, and outputs the sample at time $u$, i.e., ${\mathbf{x}}_{u}$.
In contrast, the pretrained iCT model only takes the first two inputs and can be viewed as a special case of BCM when the target time step $u$ is set to 0 0.
Therefore, we aim to initialize BCM such that it preserves the performance of the pretrained CM when $u=0$.
To achieve this, we carefully initialize the BCM as follows:

- •
In iCT, the time embedding of $t$ is passed through a 2-layer MLP with SiLU activation, after which it is fed into the residual blocks.
At the initialization of BCM, we duplicate the 2-layer MLP (with the same pretrained weights) for $u$.
However, during fine-tuning, the MLPs for $t$ and $u$ are not tied.
- •
We do not want to expand the input dimension of the pretrained residual blocks, so first concatenate the embedding of $t$ and $u$ together, and then reduce its dimension by half with a simple linear layer (without bias). No additional activation function is applied for this simple linear layer.
- •
In order to maintain the generation quality of the pretrained iCT, we initialize this linear projection as $[{\mathbf{I}},\bm{0}]$, in which ${\mathbf{I}}$ represents an identity matrix and $\bm{0}$ represents a zero matrix.
As a result, the influence brought by the newly introduced time embedding of $u$ is zeroed at the beginning and learned gradually during fine-tuning.

With the modified architecture initialized with the pretrained CM weights, we tune the models using our proposed BCT loss till convergence, using the same learning rate, ema rate, etc., as in iCT.
The only difference is that we start the fine-tuning from $N=320$ and increase it to $N=480$ and subsequently $N=640$. Specifically,

- •
we tune BCM for 210k, 16k and 8k iterations respectively at $N=320,480$ and 640.
- •
we tune BCM-deep for 360k, 100k and 10k iterations respectively at $N=320,480$ and 640.

The number of iterations for different values of $N$ is determined empirically:
we increase $N$ when we observe no performance improvements by extending the training duration at the same $N$.

### C.2 Sampling and Inversion Configurations

Here we provide hyperparameters for all the experiments on CIFAR-10 and ImageNet-$64\times 64$ to reproduce the results in the main paper.

Sampling.
On *CIFAR-10*:
for BCM (not deep), we use ancestral sampling with $t_{1}=1.2$ for NFE$=2$, zigzag sampling with $\varepsilon_{1}=0.2,t_{1}=0.8$ for NFE$=3$ and the combination of ancestral sampling and zigzag sampling with $t_{1}=1.2,\varepsilon_{1}=0.1,\tau_{1}=0.3$ for NFE$=4$. For BCM-deep, we use ancestral sampling with $t_{1}=0.7$ for NFE$=2$, zigzag sampling with $\varepsilon_{1}=0.4,t_{1}=0.8$ for NFE$=3$ and the combination with $t_{1}=0.6,\varepsilon_{1}=0.14,\tau_{1}=0.3$ for NFE$=4$.
On *ImageNet-$64\times 64$*:
for both conditional BCM and BCM-deep, we use ancestral sampling with $t_{1}=2.4$ for NFE$=2$, zigzag sampling with $\varepsilon_{1}=0.1,t_{1}=1.2$ for NFE$=3$ and the combination of ancestral sampling and zigzag sampling with $t_{1}=3.0,\varepsilon_{1}=0.12,\tau_{1}=0.4$ for NFE$=4$.

Inversion.
On *CIFAR-10*:
we set $\varepsilon=t_{1}=0.07$, $t_{2}=6.0$ and $t_{3}=T=80.0$ for NFE$=2$ and $t_{2}=1.5,t_{3}=4.0,t_{4}=10.0,t_{5}=T=80.0$ for NFE$=4$.
These hyperparameters are tuned on 2,000 training samples, and we find them generalize well to all test images.
On *ImageNet-$64\times 64$*:
for both BCM and BCM-deep, we set $\varepsilon=t_{1}=0.07$, $t_{2}=15.0$ and $t_{3}=T=80.0$ for NFE$=2$.
In the experiments we always use 1-step generation to map the inverted noise to reconstructed images (though using more than one step may improve the results) and evaluate the per-dimension MSE between the original images and their reconstructed counterparts.

We highlight that the hyperparameters are relatively robust for 2-step sampling/inversion, and the trend that the combination of ancestral and zigzag sampling is superior is also general.
However, to achieve optimal performance with more steps, the tuning of each specific time step may require great effort.
We should note that similar effort is also required in CMs, where Song et al. () use ternary search to optimize the time steps.

Inversion Baselines. For DDIM, we use the reported MSE by Song et al. (); for EDM, we load the checkpoint provided in the official implementation at [https://github.com/NVlabs/edm?tab=readme-ov-file](https://github.com/NVlabs/edm?tab=readme-ov-file) (CC BY-NC-SA 4.0 License), and re-implement a deterministic ODE solver following Algorithm 1 in (Karras et al., ).

### C.3 Applications with Bidirectional Consistency

Here we provide more details about the applications we demonstrated in Section 4.3 and Section B.1.

#### C.3.1 Interpolation

We first invert the two given images ${\mathbf{x}}_{1}$ and ${\mathbf{x}}_{2}$ to noise at $T=80.0$ using Algorithm 5.
To avoid subscript overloading, in this section, we denote
their noise as ${\mathbf{z}}_{1}$ and ${\mathbf{z}}_{2}$, respectively.
Specifically, we find that adopting a 3-step inversion with $\varepsilon=t_{1}=0.07,t_{2}=1.5,t_{3}=6.0$, and $t_{4}=T=80.0$ for CIFAR-10, and a 2-step inversion with $\varepsilon=t_{1}=0.07,t_{2}=1.8$, and $t_{3}=T=80.0$ for FFHQ and LSUN bedroom are sufficient for good reconstruction results.
Note, that we use two different small initial noise, i.e., $\bm{\sigma}_{1},\bm{\sigma}_{2}\sim\mathcal{N}(0,{\bm{I}}),\bm{\sigma}_{1}\neq
\bm{\sigma}_{2}$ when inverting ${\mathbf{x}}_{1}$ and ${\mathbf{x}}_{2}$ respectively.

Since BCM learns to amplify the two initial Gaussian i.i.d. noises, it is reasonable to hypothesize that the amplified noises (i.e., the embeddings) ${\mathbf{z}}_{1}$ and ${\mathbf{z}}_{2}$ reside on the same hyperspherical surface as if ${\mathbf{z}}_{1}$ and ${\mathbf{z}}_{2}$ are directly sampled from $\mathcal{N}(0,T^{2}{\bm{I}})$.
Therefore, following Song et al. (), we use spherical linear interpolation as

$$ $\displaystyle{\mathbf{z}}=\frac{\sin[(1-\alpha)\psi]}{\sin(\psi)}{\mathbf{z}}_ {1}+\frac{\sin[\alpha\psi]}{\sin(\psi)}{\mathbf{z}}_{2},$ (16) $$

in which $\alpha\in[0,1]$ and $\psi=\arccos\left(\frac{{\mathbf{z}}_{1}^{T}{\mathbf{z}}_{2}}{\|{\mathbf{z}}_{
1}\|_{2}\|{\mathbf{z}}_{2}\|_{2}}\right)$.

As we discussed in the main text, it is crucial to set $\bm{\sigma}_{1}\neq\bm{\sigma}_{2}$ for inversion.
A possible reason is that if using $\bm{\sigma}_{1}=\bm{\sigma}_{2}$ for inversion, the inverted noises ${\mathbf{z}}_{1}$ and ${\mathbf{z}}_{2}$ may reside on an unknown submanifold instead of the hyperspherical surface of Gaussian, and hence Equation 16 cannot yield ideal interpolation results.
In Appendix F, we present visualization and discussions on the geometric properties of the noise space.

#### C.3.2 Inpainting

We describe BCM’s inpainting process in Algorithm 6.
In our experiments on CIFAR-10, we set the initial noise scale $s=0.5$.
Additionally, instead of inverting the image back to $T=80.0$, we empirically find inverting the image to $T=2.0$ already suffice for satisfactory inpainting outcomes, and hence, we use a 3-step inversion (i.e., $N=4$), where $t_{1}=\varepsilon=0.07$, $t_{2}=0.4$, $t_{3}=1.0$ and $t_{4}=T=2.0$, and 1-step generation from $T$ to 0 0.

As for the results by CMs, we implement CM’s inpainting algorithm (Algorithm 4 in Song et al. ()). The total number of time steps (NFEs) is set to 18 according to the official inpainting script at [https://github.com/openai/consistency_models_cifar10/blob/main/editing_multistep_sampling.ipynb](https://github.com/openai/consistency_models_cifar10/blob/main/editing_multistep_sampling.ipynb).
To ensure a fair comparison, we opt for the improved model, iCT (Song & Dhariwal, ), over the original CMs, since iCT delivers superior generation performance.
Nonetheless, due to the absence of officially released codes and checkpoints by the authors, we reproduce and train our own iCT model.
Our reproduced iCT yields an FID score of single-step generation closely matching that reported by Song & Dhariwal () — our reproduced FID is $2.87$ compared to the reported $2.83$ — affirming the reliability of our results.

For higher-resolution images, we summarize BCM’s inpainting with refinement in Algorithm 7.
For a fair comparison among BCM’s inpainting algorithm, CM’s inpainting algorithm and BCM’s inpainting with refinement, we run these three algorithms on the same BCM trained on FFHQ $64\times 64$/LSUN Bedroom $256\times 256$.
Then, we perform the inpainting using the following hyper-parameters:

- •
FFHQ $64\times 64$:
BCM inpainting: we adopt initial noise scale $s=1.0$, 1-step inversion with $t_{1}=\varepsilon=0.5$ and $t_{2}=T=20.0$, and 1-step generation from $T$ to 0 0 in BCM’s inpainting.
CM inpainting:
we adopt a 24-step noise schedule following Song et al. ().
BCM inpainting w. refine:
we first perform BCM inpainting, followed by refinement using only the last $M=12$ time steps used in CM inpainting.
- •
LSUN Bedroom $256\times 256$:
BCM inpainting:
we use initial noise scale $s=0.5$, 2-step inversion with $t_{1}=\varepsilon=0.5,t_{2}=2.5$ and $t_{3}=T=200.0$, and 2-step ancestral generation with $t_{1}=4.4$ in BCM’s inpainting.
CM inpainting:
we adopt a 40-step noise schedule following Song et al. ().
we adopt the same 40-step noise schedule in Song et al. ().
BCM inpainting w. refine:
we first perform BCM inpainting, followed by refinement using only the last $M=26$ time steps used in CM inpainting.

#### C.3.3 JPEG Restoration

We invert the input image to reach the noise scale of $T=3.2$ using $1/2/3$ steps and reconstruct it by single-step generation, making a total NFE of $2/3/4$ as in Figure 8. For 1-step inversion, we use $\varepsilon=t_{1}=0.5,t_{2}=T=3.2$. For 2-step inversion, we use $\varepsilon=t_{1}=0.5,t_{2}=1.2,t_{3}=T=3.2$. For 3-step inversion, we use $\varepsilon=t_{1}=0.5,t_{2}=1.2,t_{3}=2.0,t_{4}=T=3.2$.

For the I^2SB (Liu et al., ) baseline, since there’s no pretrained checkpoints available for ImageNet-$64\times 64$, we reproduce it using the official codebase at [https://github.com/NVlabs/I2SB](https://github.com/NVlabs/I2SB) (NVIDIA Source Code License).
There is also no official implemention or checkpoints available for Palette (Saharia et al., ), so we also reproduce it by modifying from the I^2SB codebase.
We initialize both methods with a pretrained ADM model with weight available at [https://github.com/openai/guided-diffusion](https://github.com/openai/guided-diffusion) (MIT License).

However, as there is no unconditional ADM weight for ImageNet-$64\times 64$ and our BCM is also trained with class conditions, we *include labels in both the baseline methods and our method* for convenience and comparison fairness.
This can lead to an overestimation of classifier accuracy or Inception Score (IS).
Therefore, to fairly compare our method with the baselines, we choose to report MSE between the restored images and the ground truth, along with FID.
This choice is motivated by the known rate-perception-distortion trade-off (Blau & Michaeli, ; ). In essence, for the same compression rate, there is a trade-off between MSE and realism, the latter of which is reflected by FID in our experiments.
In our experiments, we found both our BCM and I^2SB models achieved a per pixel MSE of around 0.004.
As for the FID, as shown in Figure 8, when NFE is limited, BCM achieves a better FID compared to I^2SB.
This indicates that, *when NFE is limited, our BCM obtains a better restoration than I^2SB in a Pareto sense.*

## Appendix D Derivation of the Network Parameterization

In this section, we provide more details about our network parameterization design.
To start with, recall that in Song et al. (), they parameterize the consistency model using skip connections as

$$ $\displaystyle\bm{f}_{\bm{\theta}}({\mathbf{x}}_{t},t)=c_{\text{skip}}(t){ \mathbf{x}}_{t}+c_{\text{out}}(t)F_{\bm{\theta}}(c_{\text{in}}(t){\mathbf{x}}_ {t},t),$ (17) $$

in which

$$ $\displaystyle c_{\text{in}}(t)=\frac{1}{\sqrt{\sigma_{\text{data}}^{2}+t^{2}}} ,\quad c_{\text{out}}(t)=\frac{\sigma_{\text{data}}(t-\varepsilon)}{\sqrt{ \sigma_{\text{data}}^{2}+t^{2}}},\quad c_{\text{skip}}(t)=\frac{\sigma_{\text{ data}}^{2}}{\sigma_{\text{data}}^{2}+(t-\varepsilon)^{2}},$ (18) $$

that ensures

$$ $\displaystyle c_{\text{out}}(\varepsilon)=0,\quad c_{\text{skip}}(\varepsilon)=1$ (19) $$

to hold at some very small noise scale $\varepsilon\approx 0$ so $\bm{f}_{\bm{\theta}}({\mathbf{x}}_{0},\varepsilon)={\mathbf{x}}_{0}$(^4^44While the parameterization written in the original paper of Song et al. () did not explicitly include $c_{\text{in}}(t)$, we find it is actually included in its official implementation at [https://github.com/openai/consistency_models_cifar10/blob/main/jcm/models/utils.py#L189](https://github.com/openai/consistency_models_cifar10/blob/main/jcm/models/utils.py#L189) in the form of Equations 17 and 18.).
Since we expect the output is a noise image of target noise scale $u$, we expand the parameterization of $c_{\text{skip}},c_{\text{out}}$ and $c_{\text{in}}$ to make them related to $u$, as

$$ $\displaystyle\bm{f}_{\bm{\theta}}({\mathbf{x}}_{t},t,u)=c_{\text{skip}}(t,u){ \mathbf{x}}_{t}+c_{\text{out}}(t,u)F_{\bm{\theta}}(c_{\text{in}}(t,u){\mathbf{ x}}_{t},t,u).$ (20) $$

Our derivation of network parameterization shares the same group of principles in EDM Karras et al. ().
Specifically, we first require the input to the network $F_{\bm{\theta}}$ to have unit variance.
Following Eq. (114) $\sim$ (117) in EDM paper (Karras et al., ), we have

$$ $\displaystyle c_{\text{in}}(t,u)=\frac{1}{\sqrt{\sigma_{\text{data}}^{2}+t^{2} }}.$ (21) $$

Then, as we discussed in the main text, we expect the model to achieve consistency in Equation 7 along the entire trajectory.
According to Lemma 1 in Song et al. (), we have

$$ $\displaystyle\nabla\log p_{t}({\mathbf{x}}_{t})$ $\displaystyle=\frac{1}{t^{2}}\left(\mathbb{E}[{\mathbf{x}}|{\mathbf{x}}_{t}]-{ \mathbf{x}}_{t}\right)$ (22) $\displaystyle\stackrel{{\scriptstyle(i)}}{{\approx}}\frac{1}{t^{2}}\left({ \mathbf{x}}-{\mathbf{x}}_{t}\right)$ (23) $\displaystyle=-\frac{\bm{\sigma}}{t},$ (24) $$

in which we follow Song et al. () to estimate the expectation with ${\mathbf{x}}$ in $(i)$.
When $u$ and $t$ are close, we can use the Euler solver to estimate ${\mathbf{x}}_{u}$, i.e,

$$ $\displaystyle{\mathbf{x}}_{u}$ $\displaystyle\approx{\mathbf{x}}_{t}-t(u-t)\nabla\log p_{t}({\mathbf{x}}_{t})$ (25) $\displaystyle={\mathbf{x}}_{t}+(u-t)\bm{\sigma}$ (26) $\displaystyle={\mathbf{x}}+u\bm{\sigma}.$ (27) $$

Therefore, when $u$ and $t$ are close, and base on Song et al. ()’s approximation in $(i)$, we can rewrite the consistency defined in Equation 7 as

$$ $\displaystyle d\left(\bm{f}_{\bm{\theta}}({\mathbf{x}}+t\bm{\sigma},t,u),{ \mathbf{x}}+u\bm{\sigma}\right)$ (28) $\displaystyle=$ $\displaystyle\left|\bm{f}_{\bm{\theta}}({\mathbf{x}}+t\bm{\sigma},t,u)-({ \mathbf{x}}+u\bm{\sigma})\right|$ (29) $\displaystyle=$ $\displaystyle\left|c_{\text{skip}}(t,u)({\mathbf{x}}+t\bm{\sigma})+c_{\text{ out}}(t,u)F_{\bm{\theta}}(c_{\text{in}}(t,u)({\mathbf{x}}+t\bm{\sigma}),t,u)-( {\mathbf{x}}+u\bm{\sigma})\right|$ (30) $\displaystyle=$ $\displaystyle\left|c_{\text{out}}(t,u)F_{\bm{\theta}}(c_{\text{in}}(t,u)({ \mathbf{x}}+t\bm{\sigma}),t,u)-({\mathbf{x}}+u\bm{\sigma}-c_{\text{skip}}(t,u) ({\mathbf{x}}+t\bm{\sigma}))\right|$ (31) $\displaystyle=$ $\displaystyle\,|c_{\text{out}}(t,u)|\cdot\bigg{|}F_{\bm{\theta}}(c_{\text{in}} (t,u)({\mathbf{x}}+t\bm{\sigma}),t,u)$ $\displaystyle\quad\quad\quad\quad\quad-\left.\frac{1}{c_{\text{out}}(t,u)} \left((1-c_{\text{skip}}(t,u)){\mathbf{x}}+(u-c_{\text{skip}}(t,u)t)\bm{\sigma }\right)\right|.$ (32) $$

For simplicity, we set $d(\cdot,\cdot)$ to $L_{1}$ norm in Equation 29.
Note that, in practice, for $D$-dimensional data, we follow Song & Dhariwal () to use Pseudo-Huber loss $d({\mathbf{a}},{\mathbf{b}})=\sqrt{||{\mathbf{a}}-{\mathbf{b}}||^{2}+0.00054^{
2}D}-0.00054\sqrt{D}$, which can be well approximated by $L_{1}$ norm.

We should note that Equation 28 is based on the assumption that $u$ and $t$ are reasonably close.
*This derivation is only for the pursuit of reasonable parameterization and should not directly serve as an objective function.*
Instead, one should use the soft constraint we proposed in Equation 9 as the objective function.

The approximate effective training target of network $F_{\bm{\theta}}$ is therefore

$$ $\displaystyle\frac{1}{c_{\text{out}}(t,u)}\left((1-c_{\text{skip}}(t,u)){ \mathbf{x}}+(u-c_{\text{skip}}(t,u)t)\bm{\sigma}\right).$ (33) $$

Following Karras et al. (),
we require the effective training target to have unit variance, i.e.,

$$ $\displaystyle\mathrm{Var}\left[\frac{1}{c_{\text{out}}(t,u)}\left((1-c_{\text{ skip}}(t,u)){\mathbf{x}}+(u-c_{\text{skip}}(t,u)t)\bm{\sigma}\right)\right]=1,$ (34) $$

so we have

$$ $\displaystyle c_{\text{out}}^{2}(t,u)$ $\displaystyle=\mathrm{Var}\left[(1-c_{\text{skip}}(t,u)){\mathbf{x}}+(u-c_{ \text{skip}}(t,u)t)\bm{\sigma}\right]$ (35) $\displaystyle=(1-c_{\text{skip}}(t,u))^{2}\sigma_{\text{data}}^{2}+(u-c_{\text {skip}}(t,u)t)^{2}$ (36) $\displaystyle=(\sigma_{\text{data}}^{2}+t^{2})c_{\text{skip}}^{2}(t,u)-2( \sigma_{\text{data}}^{2}+tu)c_{\text{skip}}(t,u)+(\sigma_{\text{data}}^{2}+u^{ 2}),$ (37) $$

which is a hyperbolic function of $c_{\text{skip}}(t,u)$. Following Karras et al. (), we select $c_{\text{skip}}(t,u)$ to minimize $|c_{\text{out}}(t,u)|$ so that the errors of $F_{\bm{\theta}}$ are amplified as little as possible, as

$$ $\displaystyle c_{\text{skip}}(t,u)=\mathop{\arg\min}_{c_{\text{skip}}(t,u)}|c_ {\text{out}}(t,u)|=\mathop{\arg\min}_{c_{\text{skip}}(t,u)}c_{\text{out}}^{2}( t,u).$ (38) $$

So we have

$$ $\displaystyle(\sigma_{\text{data}}^{2}+t^{2})c_{\text{skip}}(t,u)$ $\displaystyle=\sigma_{\text{data}}^{2}+tu$ (39) $\displaystyle c_{\text{skip}}(t,u)$ $\displaystyle=\frac{\sigma_{\text{data}}^{2}+tu}{\sigma_{\text{data}}^{2}+t^{2 }}.$ (40) $$

Substituting Equation 40 into Equation 36, we have

$$ $\displaystyle c_{\text{out}}^{2}(t,u)$ $\displaystyle=\frac{\sigma_{\text{data}}^{2}t^{2}(t-u)^{2}}{\left(\sigma_{ \text{data}}^{2}+t^{2}\right)^{2}}+\left(\frac{\sigma_{\text{data}}^{2}t+t^{2} u}{\sigma_{\text{data}}^{2}+t^{2}}-u\right)^{2}$ (41) $\displaystyle=\frac{\sigma_{\text{data}}^{2}t^{2}(t-u)^{2}+\sigma_{\text{data} }^{4}(t-u)^{2}}{\left(\sigma_{\text{data}}^{2}+t^{2}\right)^{2}}$ (42) $\displaystyle=\frac{\sigma_{\text{data}}^{2}(t-u)^{2}}{\sigma_{\text{data}}^{2 }+t^{2}},$ (43) $$

and finally

$$ $\displaystyle c_{\text{out}}(t,u)=\frac{\sigma_{\text{data}}(t-u)}{\sqrt{ \sigma_{\text{data}}^{2}+t^{2}}}.$ (44) $$

One can immediately verify that when $u=t$, $c_{\text{skip}}(t,u)=1$ and $c_{\text{out}}(t,u)=0$ so that the boundary condition

$$ $\displaystyle\bm{f}_{\bm{\theta}}({\mathbf{x}}_{t},t,t)={\mathbf{x}}_{t}$ (45) $$

holds.

On the side of CMs, setting $u=\varepsilon$ will arrive at exactly the same form of $c_{\text{in}}(t,u)$ and $c_{\text{out}}(t,u)$ in Equation 18. While $c_{\text{skip}}(t,u)$ does not degenerate exactly to the form in Equation 18 when taking $u=\varepsilon$ and $t>\varepsilon$, this inconsistency is negligible when $\varepsilon\approx 0$, as

$$ $\displaystyle\left|c_{\text{skip}}^{\text{BCM}}(t,\varepsilon)-c_{\text{skip}} ^{\text{CM}}(t)\right|$ $\displaystyle=\left|\frac{\sigma_{\text{data}}^{2}+t\varepsilon}{\sigma_{\text {data}}^{2}+t^{2}}-\frac{\sigma_{\text{data}}^{2}}{\sigma_{\text{data}}^{2}+(t -\varepsilon)^{2}}\right|$ (46) $\displaystyle=\frac{\left|\left(\sigma_{\text{data}}^{2}+(t-\varepsilon)^{2} \right)\left(\sigma_{\text{data}}^{2}+t\varepsilon\right)-\sigma_{\text{data}} ^{2}\left(\sigma_{\text{data}}^{2}+t^{2}\right)\right|}{\left(\sigma_{\text{ data}}^{2}+t^{2}\right)\left(\sigma_{\text{data}}^{2}+(t-\varepsilon)^{2} \right)}$ (47) $\displaystyle=\frac{\left|\varepsilon^{2}\sigma_{\text{data}}^{2}-\varepsilon t \sigma_{\text{data}}^{2}+\varepsilon t(t-\varepsilon)^{2}\right|}{\left(\sigma _{\text{data}}^{2}+t^{2}\right)\left(\sigma_{\text{data}}^{2}+(t-\varepsilon)^ {2}\right)}$ (48) $\displaystyle=\frac{\varepsilon(t-\varepsilon)\left|(t-\varepsilon)t-\sigma_{ \text{data}}^{2}\right|}{\left(\sigma_{\text{data}}^{2}+t^{2}\right)\left( \sigma_{\text{data}}^{2}+(t-\varepsilon)^{2}\right)}$ (49) $\displaystyle<\frac{\varepsilon(t-\varepsilon)\max\left\{(t-\varepsilon)t, \sigma_{\text{data}}^{2}\right\}}{\left(\sigma_{\text{data}}^{2}+t^{2}\right) \left(\sigma_{\text{data}}^{2}+(t-\varepsilon)^{2}\right)}$ (50) $\displaystyle\leq\frac{\varepsilon(t-\varepsilon)\max\left\{t^{2},\sigma_{ \text{data}}^{2}\right\}}{\left(\sigma_{\text{data}}^{2}+t^{2}\right)\left( \sigma_{\text{data}}^{2}+(t-\varepsilon)^{2}\right)}$ (51) $\displaystyle<\frac{\varepsilon(t-\varepsilon)}{\sigma_{\text{data}}^{2}+(t- \varepsilon)^{2}}$ (52) $\displaystyle=\frac{\varepsilon}{\frac{\sigma_{\text{data}}^{2}}{t-\varepsilon }+(t-\varepsilon)}$ (53) $\displaystyle\leq\frac{\varepsilon}{2\sigma_{\text{data}}}.$ (54) $$

Therefore, we conclude that our parameterization is compatible with CM’s parameterization, so with the same CT target of Equation 6, any CT techniques (Song et al., ; Song & Dhariwal, ) should directly apply to our model and it should inherit all properties from CMs just by setting $u=\varepsilon$, which is a clear advantage compared with models that adopt completely different parameterizations (e.g., CTM Kim et al. ()).

## Appendix E Comparison of CT, CTM and BCT

We compare the training objective of CT, CTM and our proposed BCT in Table 2, where we can see how our method naturally extends CT and differs from CTM.

**Table 2: Comparison of CT, CTM training, and BCT training methodology. The figures illustrate the main objective of each method, where $\bar{\bm{\theta}}$ stands for stop gradient operation. Note that for BCM, there are two possible scenarios corresponding to the denoising and diffusion direction, respectively.**
| Model | Illustration of Training Objective | Detailed Form of Loss |  |
| --- | --- | --- | --- |
| CT |  | $\mathcal{L}_{CT}=\mathbb{E}_{t_{n},{\mathbf{x}}}[\lambda(t_{n})d],$<br>${\mathbf{x}}$ is the training sample,<br>$\lambda(\cdot)$ is the reweighting function,<br>$t_{n},d$ are illustrated in the left plot. |  |
| CTM |  | $\begin{aligned} \mathcal{L}_{CTM}=&\mathbb{E}_{t_{n},t_{n^{\prime}},t_{n^{<br>\prime\prime}},{\mathbf{x}}}[d]\\<br>&+\lambda_{GAN}\mathcal{L}_{GAN}+\lambda_{DSM}\mathcal{L}_{DSM}\end{aligned}$<br>$\mathcal{L}_{DSM}$ is the adversarial loss,<br>$\mathcal{L}_{DSM}$ is Denoising Score Matching loss (Song et al., ; Vincent, ),<br>$\lambda_{GAN},\lambda_{DSM}$ are the reweighting functions,<br>$t_{n},t_{n^{\prime}},t_{n^{\prime\prime}},d$ are illustrated in the left plot. |  |
| BCT |  | $\mathcal{L}_{BCT}=\mathbb{E}_{t_{n},t_{n^{\prime}},{\mathbf{x}}}[\lambda(t_{n}<br>)d_{1}+\lambda^{\prime}(t_{n},t_{n^{\prime}})d_{2}],$<br>$\lambda(\cdot),\lambda^{\prime}(\cdot,\cdot)$ are the reweighting functions,<br>$t_{n},t_{n^{\prime}},d_{1},d_{2}$ are illustrated in the left plot. |  |

## Appendix F Understanding the Learned Noise (“Embedding”) Space

Figure: (a) images inverted with the same initial noise are shown in the same color
Refer to caption: x20.png

This section provides some insights into the learned “embedding” space.
Recall that during inversion (Algorithm 5), we first inject a small Gaussian noise to the image.
Here we investigate the influence of this noise and the original image content on the noise generated by inversion.

We randomly select 500 CIFAR-10 images, and randomly split them into 10 groups.
We then invert the images to their corresponding noise by Algorithm 5.
We inject the *same* initial noise during inversion for images in the same group.
Figure 15 visualize the t-SNE results Van der Maaten & Hinton () of the inversion outcomes.
In Figure 15(a), images injected with the same initial noise are shown in the same color; while in Figure 15(b), we color the points according to their class label (i.e., airplane, bird, cat, …).

Interestingly, we can see that images inverted with the same initial noise are clustered together.
We, therefore, conjecture that each initial noise corresponds to a submanifold in the final “embedding” space.
The union of all these submanifolds constitutes the final “embedding” space, which is the typical set of $\mathcal{N}(0,T^{2}{\bm{I}})$, closed to a hypersphere.
This explains why applying the same initial noise is suboptimal in interpolation, as discussed in Section C.3.1.

## Appendix G Additional Visualizations

Figure: (a)
Refer to caption: extracted/5889717/Figures/inversion/noise_space.png

Figure: (a) Ground Truth
Refer to caption: extracted/5889717/Figures/inversion/original.png

Figure: Figure 18: Interpolation between two real CIFAR-10 images (injecting *different* initial noise in inversion).
Refer to caption: extracted/5889717/Figures/interpolate/intepolate_1.png

Figure: Figure 19: Interpolation between two real CIFAR-10 images (injecting the *same* initial noise in inversion).
Refer to caption: extracted/5889717/Figures/interpolate/intepolate_2.png

Figure: Figure 20: Interpolation between two real FFHQ $64\times 64$ images (injecting *different* initial noise in inversion).
Refer to caption: extracted/5889717/Figures/interpolate/interpolate_ffhq.png

Figure: Figure 21: Interpolation between two real LSUN Bedroom $256\times 256$ images (injecting *different* initial noise in inversion).
Refer to caption: extracted/5889717/Figures/interpolate/interpolate_lsun.png

Figure: (a) Masked images.
Refer to caption: extracted/5889717/Figures/inpaint/ori_inpaint.png

Figure: (a) Masked images.
Refer to caption: extracted/5889717/Figures/inpaint/ori_inpaint.png

Figure: Figure 24: Inpainting on FFHQ $64\times 64$.
Refer to caption: extracted/5889717/Figures/inpaint/inpaint_FFHQ.png

Figure: Figure 25: Inpainting on LSUN Bedroom $256\times 256$.
Refer to caption: extracted/5889717/Figures/inpaint/inpaint_LSUN.png

## Appendix H More Generation Samples

Figure: (a) 1-step generation (FID=2.64)
Refer to caption: extracted/5889717/Figures/generation/onestep-deep.png

Figure: (a) 1-step generation (FID=3.14)
Refer to caption: extracted/5889717/Figures/generation/appendix_imagenet1.png

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: - Typos: Notably, there is a typo in the abstract (“wh ich”).

- Time Sampling and Discretization: It would be beneficial to comment on whether uniform sampling is an optimal choice. Drawing from practices in EDM, it could be advantageous to sample the pair (u,t)(u, t)(u,t) primarily from the mid-trajectory region, potentially with a center-concentrated distribution, to capture more informative states along the trajectory.

- Inpainting Experiments: Instead of initializing with a masked image filled with Gaussian noise within the masked regions, an alternative approach might be to retain zero-masking, invert to identify the noise, interpolate in the noise space, and then generate. Performing these manipulations directly in the latent noise space might offer more interpretability and flexibility for different tasks. This aligns with the interpretative benefits of the noise space and is supported by ideas found in [1].

[1] Chang P, Tang J, Gross M, Azevedo VC. How I Warped Your Noise: a Temporally-Correlated Noise Prior for Diffusion Models. InThe Twelfth International Conference on Learning Representations 2024.
soundness: 3
strengths: - Addressing bidirectionality in consistency models is a timely and impactful problem, contributing to the field's advancement in invertibility and versatility.

- The approach is conceptually simple, making it accessible and logical.

- Experimental results convincingly demonstrate that BCM achieves notable gains compared to CM and CTM, underscoring its potential.
summary: This submission focuses on consistency models, addressing the challenge of bidirectionality by ensuring these models are invertible, allowing a path from data back to noise. To achieve this, the paper proposes Bidirectional Consistency Models (BCM), which leverage a single neural network for both generation and inversion. This design enables one-step generation and inversion. For low NFE settings, the experiments demonstrate that BCM outperforms both I2SB and Palette in a fully zero-shot manner.
weaknesses: The methods section could benefit from clearer writing; given the simplicity of the core idea, a more structured and precise explanation would improve readability.

---

confidence: 3
contribution: 4
initial_rating: 5
presentation: 3
questions: 1. In line 252, why is the reweighting parameter $\lambda'(t_n, t_n')$ defined in this way, and how does it help maintain the loss scale?

2. In both the Zigzag sampling and the inversion process, the authors introduce a small noise term, explaining that the endpoint of the time horizon is less effectively covered. This explanation, however, is somewhat confusing. If the endpoint at $t = 0$ is indeed less effectively covered, why is it necessary to return to $t = 0$ during sampling? Additionally, do the authors compare the effects of different sampling endpoints, such as $t = 0$ versus $t = 0.07$ (mentioned in line 334)?
soundness: 2
strengths: 1. The method proposed in this paper shows strong potential. Nearly all diffusion model-based image editing techniques require inversion, and previous methods have largely depended on DDIM and DDIM Inversion, which typically require hundreds of function evaluations (NFE). To my knowledge, BCM may be the first method to reduce NFE to fewer than ten. This advancement suggests that BCM could have a significant impact on applications requiring diffusion model inversion.

2. The paper’s writing is clear, with a logical organization that is easy to follow. The detailed descriptions provided make the method straightforward to reproduce.
summary: The paper introduces Bidirectional Consistency Models (BCM), which extend the capabilities of consistency models by enabling an inversion process. Building on the previously proposed consistency training loss, the authors add an additional loss term to ensure the bidirectional property between any two-time steps. Beyond training, the paper also proposes several novel sampling methods specific to BCM. Experimental results demonstrate that BCM performs comparably to previous methods in image generation while unlocking new functionalities, such as solving inverse problems. These enhanced capabilities arise from BCM’s inversion property.
weaknesses: 1. The training loss proposed in Equation 10 and the sampling method outlined in Section 3.3 are primarily derived from an intuitive perspective. Compared to previous works on consistency models and consistency trajectory models, this paper lacks some theoretical analysis regarding convergence.
2. While the paper includes sufficient experiments to validate its methods, it does not provide enough evidence to emphasize the bidirectional property, especially in terms of application. In Section 4.3, for the interpolation results, I think the difference between interpolating generated images and real images doesn't fully demonstrate the strength of the bidirectional property. Similarly, inpainting is also achievable with the consistency model, it does not showcase the unique advantage of bidirectionality. Also, both inpainting and blind restoration of compressed images could be approached as inverse problems, solving without inversion. Thus, none of the experiments in Section 4.3 clearly illustrate the distinct benefits of bidirectionality. Additionally, Section 4.1 focuses on image generation, which is not directly related to inversion. As a result, only Section 4.2 directly supports BCM’s efficiency in inversion.

I think the paper could contain some applications that cannot achieved without inversion, such as image editing. Otherwise, why do others choose BCM, as iCM and CTM have better generation performance?

---

confidence: 5
contribution: 2
initial_rating: 5
presentation: 2
questions: 1. The experiments in this paper primarily focus on CIFAR-10 and ImageNet-64, which are relatively small-sized datasets. Testing the proposed method on larger, more complex datasets (e.g., ImageNet-256) could better validate its scalability and practical performance in real-world applications. 

2. While the paper claims that BCM significantly reduces the number of function evaluations (NFE), it would be beneficial to provide a direct comparison of inference time against other models, particularly consistency models and diffusion models with fast samplers.

3.  It is interesting to note that in Table 1, the FID score for BCM worsens when the NFE increases from 2 to 3. Could you provide more details?

4. For the inpainting task, the paper only applies a box mask on CIFAR-10, leaving out tests on other datasets with this mask.

5. Given BCM’s demonstrated performance on inpainting and JPEG artifact removal, it would be valuable to investigate its effectiveness on other image restoration tasks, such as super-resolution. 

6. Figure 8 provides quantitative FID scores for JPEG restoration but lacks visual examples of the restored images.
soundness: 2
strengths: 1. BCM implements one-step generation and inversion operations, greatly reducing the amount of computation required.
2. The model is suitable for a variety of tasks such as interpolation, restoration, and image compression recovery, and has a wide range of application potential.
summary: The paper proposes a model called Bidirectional Consistency Model (BCM), which aims to simultaneously optimize image generation and inverse mapping (i.e., noise reduction) tasks. The design of BCM enables the model to perform bidirectional generation and inversion operations on the probability flow ODE (PF ODE) trajectory by introducing bidirectional consistency. The traditional diffusion model (DM) and consistency model (CM) have the problems of slow generation and unstable reverse processes. BCM introduces a single neural network to complete the generation and reverse tasks, thus significantly improving the computational efficiency and performance.
weaknesses: The experiment section should be improved. Please refer the detailed comment below.

---

confidence: 3
contribution: 2
initial_rating: 6
presentation: 4
questions: - What is the idea of the exponential Moving average (EMA) in line 140 ? Is the optimization algorithm to minimize (3) only converging in average ?
- In Algorithm (1), the BCT seems to be a SGD algorithm to minimize $\mathcal{L}_{BCT}$, to your point of view, what will be the gain of using other stochastic algorithms of order 1 such as RMSProp or ADAM to minimize this loss ?
- What is the gain of using the Pseudo-Huber loss instead of the $L_1$ norm if they are close as stated in line 1283 ? Could you provide any theoretical advantages of Pseudo-Huber loss in this specific context ?
- Can you explain what is the stop gradient operation ? (line 135 and line 219)
- On Figure 10, the loss of BCT seems to decrease with piece wise constant, how to you interprete that behavior ?
- In the zigzag strategy, a random noise is added then amplified with the network. Why applyfing random noise with the network is different from adding random noise with the right scale imediately ? Could you provide quantitative empirical evidence that demonstrate any advantage of amplifying the noise instead of adding larger noise (Figure 2 gaves a qualitative comparison) ? Could you explain theoretically the different between these two strategies ?
- In table 1, most results are taken from other papers, how do you ensure that these results are comparable ? Especially, these methods are not using the same network. Can you include the number of parameter in the table ?
- What are the theoretical guarantees that the BCM sampling strategies, with all empirical approximations (neural network), effectively sample the desire distribution ?
soundness: 3
strengths: This paper is well written and progressive. BCM shows outstanding experimental performance. The exact method including hyperparameters choice is explained in details in Appendix and the choice of reparametrization is well motivated. The comparison with recent state of the art method is well explained.
summary: This paper present a new model based on diffusion named bidirectional consistency models (BCM). It learns the forward and the backward PF ODE of diffusion wth the same model. It allows to develop new sampling strategy including zigzag sampling where noise are removed and amplified with the BCM. The authors show experimentally that BCM outperforms other methods with the same NFE for image generation or image restoration.
weaknesses: Major weakness:
- It is not clear why there is a need to learn the forward PF ODE. In fact, adding noise to an image is easy. Therefore, it is not clear what is the gain of BCM compare to CM.
- No theoretical analysis are provided. It is not clear why the sampling strategy of BCM will effectively sample the right distribution.

Minor weakness:
- I suggest in table 1 to separate more clearly each method. Because some methods are runned with various NFE, I suggest to separate with a horizental line each method.

---

confidence: 4
contribution: 3
initial_rating: 5
presentation: 3
questions: If inversion is so effective with this method, can it be incorporated into editing algorithms for fine-tuning an image? e.g. LEDITS++? This should be discussed.

I will be happy to raise the rating of the paper if I get satisfactory answers to all the above issues.
soundness: 3
strengths: 1. This is a well written paper and pleasant to read. 
2. The authors worked hard on the various experiments presented.
3. The proposed method seems to work very well, with a slight cost in the quality of image synthesis.
summary: Following Consistency Models (CM - Song 2023) and Consistency Trajectory Models (CTM - Kim 2024), this paper offers their generalization - a learned network that enables going in both directions of the trajectory of the ODE solution path. More specifically, while CM enables a jump from any t to t=0, and CTM from any t to any u<t, the proposed BCM is designed to remove the u<t constraint, thereby enabling moving up and down the trajectory. The bidirectionality is obtained by desiginbing a specific loss that generalizes that of CTM, and training the network accordingly, either by starting from scratch or by leveraging an existing pretrained CM machine.

The main benefits (as claimed) in this new capability are editing and recovery abilities, such as image interpolation, inpainting, and JPEG restoration. The cost is a slight loss in synthesis performance, as the network is now more challenged.
weaknesses: 1. The novelty in composing the generalization offered is mild at best (the change in the loss), as the method is very similar to CM and CTM.
2. As the key benefits of the proposed approach are the ability to invert the generation process and leverage it, I find several flaws/weaknesses in the proposed experiments and results: 
a. The inversion error is not negligible, and without much control over its magnitude. Could something be done to reduce this error using more NFE's?
b. In JPEG restoration, it is not enough to show FID versus NFE as Figure 8 does. Are the obtained images consistent with the given compressd image in PSNR terms? i.e. if we recompress the image using the same Quality factor, will we get close enough to the starting image? This performance measure (CPSNR) should be reported. Also, example image results should be included. 
c. On the same JPEG topic - there are other techniques that are far better than PALETTE that should have been used in the comparisons (e.g. https://openreview.net/forum?id=O3WJOt79289, DPS: https://openreview.net/forum?id=OnD9zGAGT0k, PiGDM: https://openreview.net/forum?id=9_gsMA8MRKQ).
d. The inpainting results are far from satisfactory, as evident esspecialy in Figure 9. If 14 NFE are necessary, a comparison to methods beyond CM should be provided, in the spirit of the comment above - DPS, PiGDM and DDRM (https://ddrm-ml.github.io/) should be compared. Indeed, rich work in the past 3 years offers inpaiting techniques via diffusion and these should be mentioned and compared with. See [1-12] below. 
e. To summarize, if the key benefit of thsi work is the reverse diretion that CM and CTM do not provide, this work is failing to convience of the benefits in this new capability. See my later comment on editing - perhaps this should have been the main application outlet to this work. 

[1] Zehao Dou and Yang Song. Diffusion posterior sampling for linear inverse problem solving: A filtering perspective. In The Twelfth International Conference on Learning Representations, 2024.
[2] Berthy T. Feng, Jamie Smith, Michael Rubinstein, Huiwen Chang, Katherine L. Bouman, and William T. Freeman. Score-based diffusion models as principled priors for inverse imaging. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 10520–10531, October 2023.
[3] Gabriel Cardoso, Yazid Janati el idrissi, Sylvain Le Corff, and Eric Moulines. Monte carlo guided denoising diffusion models for bayesian linear inverse problems. In The Twelfth International Conference on Learning Representations, 2024.
[4] Hyungjin Chung, Jeongsol Kim, Michael Thompson Mccann, Marc Louis Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. In The Eleventh International Conference on Learning Representations, 2023.
[5] Ciprian Corneanu, Raghudeep Gadde, and Aleix M. Martinez. Latentpaint: Image inpainting in latent space with diffusion models. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 4334–4343, January 2024.
[6] Anji Liu, Mathias Niepert, and Guy Van den Broeck. Image inpainting via tractable steering of diffusion models. In The Twelfth International Conference on Learning Representations, 2024.
[7] Andreas Lugmayr, Martin Danelljan, Andr´es Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11451–11461, 2022.
[8] Litu Rout, Negin Raoof, Giannis Daras, Constantine Caramanis, Alex Dimakis, and Sanjay Shakkottai. Solving linear inverse problems provably via posterior sampling with latent diffusion models. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
[9] Bowen Song, Soo Min Kwon, Zecheng Zhang, Xinyu Hu, Qing Qu, and Liyue Shen. Solving inverse problems with latent diffusion models via hard data consistency. In The Twelfth International Conference on Learning Representations, 2024.
[10] Yinhuai Wang, Jiwen Yu, and Jian Zhang. Zero-shot image restoration using denoising diffusion null-space model. The Eleventh International Conference on Learning Representations, 2023.
[11] Guanhua Zhang, Jiabao Ji, Yang Zhang, Mo Yu, Tommi S. Jaakkola, and Shiyu Chang. Towards coherent image inpainting using denoising diffusion implicit models. CoRR, abs/2304.03322, 2023.
[12] Morteza Mardani, Jiaming Song, Jan Kautz, and Arash Vahdat. A variational perspective on solving inverse problems with diffusion models. In The Twelfth International Conference on Learning Representations, 2024.