Iterative $\alpha$-(de)Blending: Learning a Deterministic Mapping Between Arbitrary Densities
Authors: Eric Heitz
Laurent Belcour
Thomas Chambon
Venue: Submitted to ICLR 2023 (2023)
Keywords: Deterministic Denoising Diffusion

Abstract: We present a learning method that produces a mapping between arbitrary densities, such that random samples of a density can be mapped to random samples of another. In practice, our method is similar to deterministic diffusion processes where samples of the target density are blended with Gaussian noise. The originality of our approach is that, in contrast to several recent works, we do not rely on Langevin dynamics or score-matching concepts. We propose a simpler take on the topic, which is based solely on Bayes' theorem. By studying blended samples and their posteriors, we show that iteratively blending and deblending samples produces random paths between arbitrary densities. We prove that, for finite-variance densities, these paths converge towards a deterministic mapping that can be learnt with a neural network trained to deblend samples. Our method can thus be seen as a generalization of deterministic denoising diffusion where, instead of learning to denoise Gaussian noise, we learn to deblend arbitrary data.

Under review as a conference paper at ICLR 2023
ITERATIVE d-(DE) LEARNING A DETER- MINISTIC MAPPING BETWEEN ARBITRARY DENSITIES
Anonymous authors Paper under double-blind review
ABSTRACT
We present a learning method that produces a mapping between arbitrary densi- ties, such that random samples of density can be mapped to random samples of another. In practice, our method is similar to deterministic diffusion processes where samples of the target density are blended with Gaussian noise. The origi- nality of our approach is that, in contrast to several recent works, we do not rely on Langevin dynamics or score-matching concepts. We propose simpler take on the topic, which is based solely basic sampling concepts. By studying blended samples and their posteriors, we show that iteratively blending and deblending samples produces random paths between arbitrary densities. We prove that, for finitevvriiane densities, these paths converge towards deterministic mapping that can be learnt with neural network trained deblend samples. Our method can thus be seen as a generalization of deterministic denoising diffusion where, instead of learning to denoise Gaussian noise, learn to deblend arbitrary data.
We provide a short video overview of the paper in our supplementary material.
increasing a
Gaussian Swiss Roll Gaussian
S-shape
S-shape
Cat
Texture
Face
alend Qi deblend
aleblend
ag-deblend
az-blend
Tay
Pas
Fez ~ Poz
Fag ~ Pag
Z
Figure 1: Iterative a-blending and deblending. We train a neural network to deblend blended
inputs. By deblending and reblending iteratively we obtain mapping between arbitrary densities.

---

Under review as a conference paper at ICLR 2023
INTRODUCTION
Diffusion models have recently become one of the most popular generative modeling tools (Ramesh)
ct al. 2022). They have outperformed state-of-the-art GANs (Karras et al. 2020 2021 and been
applied to many applications such as image generation (Rombach et al. 202 Dhariwal & Nichol,
2021] image processing (Saharia et al.. 2021 Kawar et al.] 2022; Whang et al. 2022). text-to-
image (Saharia et al. 2022b), video (Ho et al. 2022) or audio (Kong et al. 2020p.
First, there were stochastic diffusion models... These diffusion models have in common that they can be formulated as a Stochastic Differential Equations (SDEs) (Song et al. 2021b) such as Langevin dynamics. Langevin's 's equation models a random walk that obeys a balance between two operations related to Gaussian noise: increasing noise by adding more noise and decreasing noise by climbing the gradient of the log density. Increasing noise performs large steps but puts the samples away from the true density. Decreasing noise projects the samples back the true density. Care- fully tracking and controlling this balance allows to perform efficient random walk and provides a sampling procedure for the true density. This is the core of denoising diffusion approaches. Noise Conditional Score Networks (NCSNs) (Song & Ermon] 2019; 2020) use Langevin's 's equation di- rectly by leveraging the fact that the score (the gradient of the log density in Langevin's equation) can be learnt via a denoiser when the samples are corrupted with Gaussian noise (Vincent 1201T). Denoising Diffusion Probabilistic Models (DDPMs) (Ho et al., 2020; Nichol & Dhariwal] 2021) use a Markov chain formalism with a Gaussian prior that provides an SDE similar to Langevin dynamics where the score is also implicitly learnt with a denoiser.
...then came deterministic diffusion models. Langevin's 's SDEs variants describe an equilibrium between noise injection and noise removal. Nullifying the noise injection in these SDEs yields Or- dinary Differential Equations (ODEs), also called Probability Flow ODEs [Song et al. (2021b), that simply describe the deterministic trajectory of a noisy sample projected back in the true density, For instance. Denoising Diffusion Implicit Models (DDIMs) (Song al., 2021a) are the ODE variants of DDPMs. These ODEs provide a smooth deterministic mapping between the Gaussian noise den- sity and the true density. Deterministic diffusion models have been motivated recently because an ODE requires much fewer solver iterations than its SDE counterpart. Furthermore, a deterministic mapping presents multiple practical advantages because samples are uniquely determined by their prior Gaussian noise, can be interpolated via the Gaussian noise, etc.
Is there a simpler approach to deterministic diffusion? The point of the above story is that, in the recent line of work on diffusion models, stochastic diffusion models came first and deterministic diffusion models came after, framed as special cases of the stochastic ones. They hence inherited the underlying mindset and mathematical framework. As a result, advanced concepts such as Langevin dynamics, score matching, how they relate to Gaussian noise, etc. appear to be necessary back- ground to grasp recent deterministic diffusion models. We argue that this is a significant detour to something that can be framed in much simpler and more general way. We propose fresh take on deterministic diffusion with another mindset, using only basic sampling concepts.
We derive a deterministic liffusion-like model based on the sampling interpretation of blending and deblending. We call it Iterative Blendingg (IADB) in reference to the Computer Graphics a a-blending technique that composes images with transparency parameter [Porter& & Duff (1984). Our model defines a mapping between arbitrary densities (of finite-variance).
We show that when the initial density is Gaussian, the mappings defined by IADB are exactly the same as the ones defined by DDIM (Song al.] (221aa) On the theoretical side, our model can thus be seen generalization of DDIM to arbitrary sampling densities rather than just Gaussian. Furthermore, our alternative derivation leads to a more numerically stable sampling formulation. Our experiments show that IADB consistently outperforms DDIM in terms of final FID on several datasets and is more stable with small number of steps in the sampling stage.
We explore the generalization to arbitrary non- non-Gaussian densities provided by our model. We report that, although this generalization seems promising on the theoretical side, the application possibilities were disappointing in our experiments in image generation. We found that sampling with non-Gaussian densities can significantly lower the quality of the generated samples and that the mappings are not always interesting for image processing applications.
2

---

Under review as a conference paper at ICLR 2023
2 A DETERMINISTIC MAPPING BETWEEN ARBITRARY DENSITIES
We consider two densities Po: Pi IR R+ represented respectively by the red triangle and the green square in Figure 2
Our objective is to define deterministic mapping such that i.i.d. samples To ~ Po passed through the mapping produce i.i.d. samples #1 ~ P1-
2.1 BLENDING AND DEBLENDING AS SAMPLING OPERATIONS
(a) a-blending. We call the density of the blended samples Za = (1 - a) zu + a 01 obtained by blending random samples 21) ~ po x P with a blending parameter a E [0, 1]. This illustrated in Figure (a).
(b) a-deblending, We call a-deblending the inverse sampling operation, i.e. generating ran- dom To and 21 from the initial densities that could have been a-blended into point x, as shown in Figure (b)). More formally, it means generating random posterior samples (Io, 21) (a..) (Po [ Note that we never use these posteriors samples in practice, use them only for the derivation of our method.
Proposition I Rd is a fixed point, the posteriors samples (Io: 21) ()) (po P1)) are distributed the posterior densities. However, if I Pa is random sample, the posteriors samples are distributed in the initial densities: (10, 21 (s..paaa) (po p1).
Proof. It follows directly from the law of total probability. We provide more details in Appendix
(c) aleddinl Let's consider two blending parameters on 02 [0, 1]. Using the previous proposition, we can chain on -ebeeennng and az-blending to map a random sample Tay ~ Po, to random sample Foz Pag- Indeed, by sampling posteriors for random sample Fos Pas we obtain random samples (æo, æ1) (Po x p1) from the initial densities, and blending them with parameter a2 provides a random sample Doe ~ Paz This illustrated in Figure (c).
(a) a-blending (IO,
a-deblending (Jop I1 (ia
Za
~PoXpp Xpi
Poo
rupo Xpr
To
30
3
(c) adddebblending
(d) iterative e--dde
Toy
(No, x1)
(May PPP an)
Fog ~ag
F0
Fo
01
xPP
Popo
rup1
Do Po
Figure Blending and deblending as sampling operations.
3

---

Under review as a conference paper at ICLR 2023
2.2 ITERATIVE BDEEEBLLNN (IADB)
We introduce Iterative a-(de)Blending (IADB), an iterative algorithm that can be implemented stochastically or deterministically. Our main result is that both variants converge towards the same limit, which yields a deterministic mapping between the densities Po and P1 shown in Figure (d).
Algorithm t iterative o-(de)blending (stochastic). Let's consider a number of iterations and evenly distributed blending parameters
T)), This algorithm creates a sequence (Io, ~ Po,. = {0, .., T}) that starts with a random sample æig ~ Po and ends with a random sample Zar I1 P1 by applying a-(dejblending iteratively. In each iteration, Ia, Pa: is ae-deblended by sampling random posteriors, which are sampled and ay41-blended again to obtain sample z Pas+1. End-to-end, this algorithm provides a stochastic mapping between samples 2o ~ and samples 21 ~ P1-
1/TTt
Algorithm 2 iterative a-(de)blending (deterministic). This algorithm is the same as Algo- rithm [
except that, in each iteration, the random posteriors samples are replaced by their expec- tations. The algorithm is thus not stochastic but deterministic.
Theorem I: convergence of iterative a-(de)blending. Po and pı Riemann-integrable den- sities of finite variance, the sequences computed by Algorithm aand Algorithm 2 converge towards the same limit the number of steps T increases, i.e. for any [0, we have
lim computed by Algorithm To. T) = .lim computed by Algorithm 2 ro. T). T-00
(1)
T-000
Proof. We detail this proof in Appendix B Intuitively, in each iteration, Algorithm makes
small step Ara = (#1 - xo) aa along the segment given by random posterior samples. As the number of iterations increases, many small random steps average out, and the infinitesimal steps
described by an ODE that involves the expected posteriors like in Algorithm
dx (81[(aa.o) To)(ma)0)) da.
Algorithm 1 Iterative a-dde)blending (stoch.)
Algorithm 2 Iterative a-(de)blending (deter.)
Require: To. T for t = 0,.,, T' - do
Require: Zo, T
for = 0, O.,, T - do
(po, Pi (() Toct1 = (1 - a1+1) 3o + ax+1 $1
sample (Io, x1)
(To, 51 E(po.r) (iaue (1) ine) [(r,
Tast1
(1
0++1 To + 04+1 T1
end for
end for
T steps
= 1000 steps
Figure 3: Both algorithms step iteratively by moving the samples along segments defined by their posterior densities. The difference is that Algorithm [ uses segments between random posterior samples, which creates stochastic paths, while Algorithm 2 uses the segment between the average of the posterior samples, which creates deterministic paths. As the number of steps 7 increases, the randomness of the stochastic paths averages out and they converge towards the deterministic paths.
4

---

Under review as a conference paper at ICLR 2023
3
LEARNING ITERATIVE a- B-DDEBLLENDING
In this section, we explain how to use iterative jblndingg in a machine learning context, where we train neural network Do predict the average posterior samples used in Algorithm 2
3.1 VARIANT FORMULATIONS ITERATIVE (DF))BLENDIN
A direct transposition of Algorithm 2 means learning the averages of both posterior samples To and T1. However, one is implicitly given by the other one such that is not necessary to learn both and variants of Alg. 2 are possible. The fact that multiple, theoretically equivalent, variants are possible pointed out by Salimans& & Ho (2022). However, they are not equivalent in practice. In Table we summarize four variants derived in Appendix C and compare their practical properties. Variant It is highly unstable because instead moving the current sample Fay the sample Tai plainly recomputed from the outputs of the neural network, such that its residual learning errors accumulate in each step. The larger the number of steps T. the more this variant diverges. Variants (b) and (c) consist of learning either only To T1. The sampling suffers from numerical stability near respectively a₁ = O and o = because of the respective divisions by 0s and 1 Otg- We recommend using variant (d) that consists of learning the average difference vector J To. It is a direct transposition of the ODE defined in Equation 2 This variant updates the current samples in each iteration without any division. We found it to be the most stable variant for both training and sampling.
(a) is the vanilla transposition of Algorithm 2
(a) learn Ty and Æ1 TTo F1 )
(b) learn only To To = Do (Tay (it)
(c) learn only σ1
(d) learn ;1 Zo
Do (Mar, Ott)
31 Dg (Tay Ott)
31 In Do (Fa, Ct)
Tast1
Ta.t1 3D
Toc+1
Tatt] Fia +
(1 a141) 1 &o + 0
ac+1 (No. To)
(1-ar=1) (Nor T1)
(a++1 (1) (0r - 500)
(1-a:)
unstable
unstable when a₁, - 0
unstable when az -
stable
Table 1: Variant formulations of iterative a-(de)blending (equivalent in theory, not in practice).
3.2 TRAINING AND SAMPLING
Following variant (d) of Table I we train the neural network Dg to predict the average difference vector between the posterior samples. Our learning objective defined by
| |
min
E Or
(Ia, Zo. or) o13 (cro.a)
E
(3)
Note that minimizing the 12 to the average of a distribution equivalent to minimizing the 12 to all the samples of the distribution. We obtain the equivalent objective
[IDo (Fa, a) (I1l(ea.a) - ()))))))) [|]]
min 0
E (a))) ( (s)
(4)
Finally, as explained in Section 2.1 sampling Ta ~ Pa and (Io] a) T1)(2. a)) in this order is equivalent to sampling Zo Po and Di pı and blending them to obtain Ca Pa- We obtain our final learning objective
min
E 0,
DD ((1 ai) x0 az) (x1 -xo)1
(5)
which we use to optimize 9 in Algorithm 3 Finally, in Algorithm we iteratively map samples Fo Pu to samples I1 Pı in the same way in Algorithm where we use the neural network
Do to obtain the average posterior difference.
Algorithm Training
Algorithm 4 Sampling
Require: 20 Po. #1 ~ P1, [0,1] F. = [1 a) vo + 0x1
Require: To. T
for 0,.., T do
1= Do (Ia. a) [x1 21
Jo = Fo. (a -a,) Dy (Ior ai)
backprop from 1 and update 0
end for
5

---

Under review as a conference paper at ICLR 2023
4 EXPERIMENTS WITH ANALYTIC DENSITIES
Mapping 1D densities. In Figure 4 we experiment with analytic 1D densities where the expec- tation 31 To can be computed analytically rather than being learnt by a neural network Dg. The experiment confirms that the analytic version matches the reference and that the neural network trained with the approximates the same mapping. We also tested training the neural network with the 11, which makes the neural network approximate the median of 21 rather than its average. The resulting mapping does not match the reference. This confirms that learning the average l2 training is a key component of our model, as explained in Section 3.2
reference
analytic average nn (l2 training)
analytic median an (h1 training)
increasing
IADB with analytic expressions neural networks (nn) for De
Figure 4: We map bi-Normal distribution with modes = -0.5 and 0.5 with 01/2 = 0.1 (in red) Normal distribution of unit variance (in blue). The reference shows the intermediate blended densities Pa obtained by analytically convolving both densities. The other densities are the histograms of the samples Fa computed Algorithm 4 using either analytic expressions or neural networks (nn) for Dg. The neural network is MLP with hidden layers of 64 filters.
Mapping 2D densities. Figure 5 shows that the intermediate blended densities Pa computed by our mapping match the reference blended densities. Figure shows how our algorithm maps the samples of to samples of p1.
reference
IADB
Figure 5: We display blended samples Va ~ Pa (red) and samples za computed by Algorithm
using a MLP with 5 hidden layers of 64 filters for D9 (green).
S
S
AA and p1
IADB
3-Normal to 2-Normal
Normal to S-curve
Roll to S-curve
Figure 6: We show samples the reference densities Po and p1 and the mapping computed by Algorithm usinn MLP with 5 hidden layers of 64 filters for Dg. The final samples computed by the algorithm (green) match the reference samples Iı Pı (red).

---

Under review as a conference paper at ICLR 2023
5 RELATION TO DDIM
In this section, we explain that IADB can be framed as a more stable generalization of DDIM (Song al. 202 that enjoys the possibility of using Gaussan densities for po-
Proposition 2. Po is a Gaussian density, IADB and DDIM define the same deterministic mapping.
Proof. In Appendix D we show that with simple change of parameterization, the update rule of DDIM is exactly the variant (b) in our Table T
Experimental comparison of IADB and DDIM. Figure 7 we experiment under the same conditions (architecture, training time, Ist-order solver, uniform schedule, Gaussian po) and measure the FID score (Heusel et al. 2017) for varying number of sampling steps on 3 image datasets: LSUN Bedrooms (64x64), CelebA (64 x 64) and AFHQ Cats(128x128) for 120 hours of training. We use a U-Net architecture (from Diffusers library We observe a consistently better performance of IADB compared to DDIM. This might be due to our numerically stable formulation explained in Section 3.1 The formulation generally used in DDIM corresponds to the variant (b) presented in Table they train a denoiser predict the Gaussian noise present the noisy image samples, i.e. their model learns to predict žg. However, we explain that this variant makes the sampling less stable because of the division near As a matter of fact, in their implementation, the sampler starts some > O precisely to avoid dividing by O. Our variant (d) does not suffer from this problem. Another possibility is that the learning objective defined by variant (d) provides a better optimization landscape than variant (b). For instance, the effort for learning To likely imbalanced in a while the effort for learning I To is likely more balanced over a.
Discussion. The Langevin/score tnnnecccnccoreemaaccinng approach puts the emphasis on the fact that the gradient of the log density in Langevin's 's equation can be learnt via Ty when Po is Gaussian Vincent (2011).
This mindset naturally leads to variant (b) in Table In contrast, the derivation of IADB emphasizes are possible.
that Foe, To and # are aligned and thus that all the four variants of Table are
IADB
DDIM
CelebA
Bedrooms Cats
=
T ==4 4
T = 8
T = 16
T = 32
T = 128
DDIM
200.89
39.32
22.98
18.12
13.66
9.26
Cats
IADB
86.68
39.40
17.50
11.15
8.06
6.69
DDIM
177.86
46.58
19.50
11.91
9.23
6.93
CelebA
IADB
108.13
51.79
22.68
12.15
7.52
5.56
DDIM
307.76
104.55
38.58
24.26
20.34
16.82
Bedrooms
IADB
238.45
57.60
18.55
14.12
14.57
15.93
Figure 7: Comparing IADB and DDIM. We use the same Gaussian noise to sample images with IADB and DDIM. We obtain very close images because the underlying theoretical mappings are the same. LADB achieves better FID scores w.r.t. the number of steps T than DDIM most of the time.
7

---

Under review as a conference paper at ICLR 2023
6 (DISAPPOINTING) EXPERIMENTS WITH ARBITRARY IMAGE DENSITIES
(Disappointing)) sampling quality. In the experiment of Figure 8 we use IADB to compute map- pings between different image datasets, i.e. in contrast the experiment of Figure 7 we use real images rather than Gaussian noise to sample other images. Note that, in contrast to the Gaussian density, the implicit density represented by an image dataset might not be Riemann integrable, for instance if the images are distributed on a diver-dimensional manifold. To make sure that our theo- rem applies, we regularize po by applying little amount of noise the images. theory, with this regularization, IADB is proven to produce correct sampling of p1 regardless of Po. However, in practice, we observe that the qualitative performance of the mapping is significantly lower than with Gaussian noise. This might be because for denoising Gaussian noise or because denoising noise might be fundamentally simpler than de- blending arbitrary images. In any case, our takeaway is that an experimental set up that works well with Gaussian noise does not necessarily transpose successfully to other densities.
use architecture that has been designed specifically
Pebble
CelebA
Leave
CelebA
Cat
Dog
Map
Sat
Figure& 8: Sampling with non-Gaussian densities. We use the same experimental set up as in Figure 7 except that we replace the Gaussian noise by an image database.
(Disappointing) mappings. For some applications, we wish that IADB would learn a "meaning- ful" mapping between po and p1- Unfortunately, this is not always the case. Indeed, the theory predicts that the mapping will produce valid sampling of p1 using Po but not that the mapping will be what human user expects. For instance, Figure shows result where IADB trained to learn mapping between corrupted images and clean images and we wish that the mapping would behave like image restoration process. Unfortunately, although IADB effectively learns to sam- ple clean images using corrupted ones, the mapping does not produce a valid restoration because the clean images do not always resemble the corrupted ones.
Corrupt
CelebA
Corrupt
CelebA
Figure 9: Image restoration with LADB. In this experiment, we use IADB to map corrupted images to clean images. (left) The corruption is downscaling+noise. (right) The corruption is decoloriza- tion+noise. Ideally, the mapping learnt by IADB would restore the corrupted image. Unfortunately, the mapping creates a clean image but it does not match the corrupted one anymore.
8

---

Under review as a conference paper at ICLR 2023
7 RELATED WORK
Stochastic Differential Equations (SDEs). The random sequence computed by the stochastic version of IADB presented in Algorithm I is Markov chain. This algorithm might thus be remi- niscent of stochastic diffusion models Song et al. (2021b) based on SDEs. However, it is not related an SDE, Indeed. SDEs model stochastic behaviors the infinitesimal scale while our mapping is stochastic for discrete steps and becomes a deterministic ODE in the infinitesimal limit.
Non-Gaussian denoising diffusion. Some previous works are dedicated to replacing Gaus- sian noise by other noise distributions such the generalised normal (exponential power) dis- tribution (Deasy et al. 2021) or the Gamma distribution (Nachmani et al. [021]. Our more general derivation works with any finite-variane density rather than specific noise alternatives. Peluchetti (Peluchetti] 2022) proposes a more general SDE framework. Our ODE can be derived from his SDE by nullitying the stochastic component and following the aggregation method.
Alternative deterministic diffusion. Cold diffusion (Bansal et al. 2022) shows that diffusion- like methods can reverse a variety of empirical image degradation processes. Our method similar to their animorphosis example where human faces are progressively blended and deblended with animal faces. Our model provides a proven way to sample from the right density for this application.
Image-to-image translation. In Section 6 have seen that IADB's mapping does not provide a faithful image-to-image translation. Previous works show that conditioning the diffusion process seem be necessary get faithful translations. For instance, adding energy guide during the ODE integration (Zhao et al.] 2022). by progressively injecting features (Meng et al. 2021 l), or by sampling conditional densities Saharia et al.] (2021 2022a). In Figure I0, we experiment with the latter with Gaussian noise for zo, clean images for Ti, and corrupted version of x1 for the condition
e passed as an additional argument to the neural network: Dn a) To a 21, C, a) ãΙ To.
generations
generations
condition
condition
Figure 10: Conditional image restoration with IADB, From a corrupt image, either downscaling (left) or decolorization (right), we create various restorations using Gaussian noises To-
CONCLUSION
The objective of this work was to find simple and intuitive way to approach deterministic diffusion. We derived Iterative Blending (IADB), a deterministic diffusion model based on a sampling interpretation of blending and deblending. Our model is similar to DDIM [ong et al. (2021a) but its derivation is significantly simpler and reveals that the model is valid for arbitrary (fiite-vaariane) densities rather than only Gaussian densities. Furthermore, our derivation leads to a variant learning formulation that happens to be more numerically stable than the one of DDIM.
An important takeaway of our experiments is that our results in image generation are significantly worse when using Gaussinn densities. The theory allows to use non-Gaussian densities for sam- pling and our experiments on 2D non-Gaussiaa densities were successful in this regard. However, this did not transpose well to image generation in practice. This might be because used neu- ral networks that have been designed specifically for the denoising task or because denoising noise might be fundamentally simpler than deblending arbitrary images.
Finally, note that we experimented with our model in its vanilla setting with a uniform blending schedule and first-order ODE solver. might benefit from the improvements brought to denoising diffusion such as better blending schedules and higher-order ODE solvers (Karras et al. 2022).

---

Under review as a conference paper at ICLR 2023
REFERENCES
Arpit Bansal, Fitan Borgnia, Hong-Min Chu, Jie S. Li, Hamid Kazemi, Furong Huang, Micah Gold- blum, Jonas Geiping, and Tom Goldstein. Cold diffusion: Inverting arbitrary image transforms without noise, 2022.
Jacob Deasy, Nikola Simidjievski, and Pietro Liò. Heavy-tailed denoising score matching. ArXiv, abs/21 12.09888, 2021.
Prafulla Dhariwal and Alexander Quinn Nichol. Diffusion models beat gans image synthesis. volume 34, 2021.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 32, 2020.
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J. Fleet. Video diffusion models, 2022.
Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Ana- lyzing and improving the image quality of stylegan. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8107-8116, 2020.
Tero Karras, Miik Aittala, Samuli Laine, Erik Härkönen, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Alias-free generative adversarial networks. In Proc. NeurlPS, 2021.
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models, 2022.
Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restoration models. arXiv preprint 17Xi::2201.11993, 2022.
Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versatile diffusion model for audio synthesis, 2020.
Chenlin Meng, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon. Sdedit: Im- age synthesis and editing with stochastic differential equations. arXiv preprint arXiv:2108.01073, 2021.
Eliya Nachmani, Robin San Roman, and Lior Wolf. Denoising diffusion gamma models, 2021.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic mod- els. In Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp.
Stefano Peluchetti. Non-denoising forward- time diffusions, 2022.
URL https://
openreview www.eet/frrrte /otm://oomm/ddo//EEuuuii
Thomas Porter and Tom Duff. Compositing digital images. SIGGRAPH Comput. Graph., 18(3): 253-259, 1984.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with clip latents, 2022.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models, 2021.
Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David Fleet, and Mohammad Norouzi. Image super-resolution via iterative refinement. arXiv preprint 200.:::0004076660 2021.
Chitwan Saharia, William Chan, Huiwen Chang, Chris Lee, Jonathan Ho, Tim Salimans. David Fleet, and Mohammad Norouzi. Palette: maage-to-image diffusion models. In ACM SIGGRAPH 2022 Conference P Proceedings, 2022a.
10

---

Under review as a conference paper at ICLR 2023
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kam- yar Seyed Ghasemipour, Burcu Karagol Ayan, S. Sara Mahdavi, Rapha Gontijo Lopes, Tim Sal- imans, Jonathan Ho, David J Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffu- sion models with deep language understanding, 2022b.
Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusioon models. In International Conference on Learning Representations, 2022. URL [tPpssppp///.p..eeeiieee Sttt/EorunnId-TTIIXXIE
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. Interna- tional Conference on Learning Representations, 202 la.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in Neural Information Processing Systems, 32, 2019.
Yang Song and Stefano Ermon. Improved techniques for training score-based generative models. ArXiv, abs/2006.0901 1, 2020.
Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. International Conference Learning Representations, 2021b.
Pascal Vincent. A connection between score matching and denoising autoencoders. Neural Compu- tation, 1661:166,11677, 2011.
Jay Whang, Mauricio Delbracio, Hossein Talebi, Chitwan Saharia, Alexandros G Dimakis, and Pey- man Milanfar. Deblurring via stochastic refinement. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16293-16380 2022.
Min Zhao, Fan Bao, Chongxuan Li, and Jun Zhu. Egsde: Unpaired image-to-image translation via energy-guided stochastic differential equations. arXiv preprint arXiv:2207.0663: 2022.
11

---

Under review as a conference paper at ICLR 2023
A LAW OF TOTAL PROBABILITY
The law of total probability states that the prior density Po x P1 can be written as a weighted linear combination of its posterior densities:
(o( pi (11)) = / (Po(wo) pi ((.))))) Pa( dra:
(6)
where Pa (za) is the weight of each posterior density (po(2o) Pı (21)) (x) As a result, sampling
the prior density
P1 can be achieved by choosing random posterior density proportionally its weight, i.e. sampling Hi Pa. and choosing a random sample in the posterior density conditioned by parameter Fa. L.e. sampling (ro, 21 (mto "P:.))
This is illustrated in Figure 11
posterior samples for fixed locations Za (rr, (s...)) (Po x Pi (z..a)
posterior samples for random locations Ta Pa
(vo, $1 (z.cp...))
Po X p
Each posterior density represents only subset of the prior density, However, thanks the law of total probability, we know that the average of random posterior densities is exactly the prior density. As result, sampling random blended samples Ta Pa and random posterior samples (zg, 21) (x..aa ( a) (Po x p1) (e...) is equivalent sampling random (10,21) Po X P1 directly in the prior density.
Figure 1I:
12

---

Under review as a conference paper at ICLR 2023
B PROOF OF THE ITERATIVE a-(DE) CONVERGENCE THEOREM
B.1 PRELIMINARIES
We first recall some properties that are required in the derivation of the limit of Algorithm
The posterior distributions have finite variance. The theorem requires that po and are of finite variance, such that their respective posterior densities Pol(ar,)) and Pi[(z.)) are also fimite variance. This is because the idea of the proof is that averaging many small random steps (provided by the posterior distributions) converges towards their expectations and true only their variance is finite. We use this between Equation (23) and Equation 24
The expectations of the posterior distributions are continuous. If and Pi are classic, bremann-integrable densities, then they are continuous almost everywhere. Since the blended distributions are essentially convolutions of and Pt, it follows that the posterior densities (o)()) and Pı](2,a are also continuous almost everywhere and the expectation of their samples (2,(), ~ Poj(z,a) and Fi((a,a) (a.(o. are continuous everywhere (the expectation cancels out the null set where they not continuous). In summary, for any Rd and [0, 1] we have:
lim (o)(
[T]()))))))
lim
E [1(((())
[11((,,))]
x -
lim E [m.e))
= E [rok))]]
lim
E [[](())))] = [I][[=,)]]
(8)
of +0
We use this between Equation
24 and Equation
25
B.2 OBJECTIVE OF THE PROOF
To prove that Algorithm and Algorithm 2 converge towards the same limit as the number of steps T increases, need to show that the trajectories of the samples are the same. This is the case if, in the limit, the derivatives dæa. are the same with both algorithms. The discrete update at step is: da
1 T
Aos 04+11
04
ATa:
Faat1
Tor
(10)
and we want to prove that for any the same with both algorithms:
1] and point xa Rd the continuous limit exists and is
dra lim
Ara
(11)
da Aa
Aa
B.3 LIMIT OF ALGORITHM 2
In step of Algorithm
we use the average the posterior samples that are such that:
Va. = - axt) To To ai) + Ot Fil(sa,
(12)
Eass = (1 - a(+1) 1 x as) + ax+1 o 01[(200)00)
(13)
where Equation is property of the average posteriors of Va. and Equation (3) is by We thus have the discrete difference:
definition in Algorithm 2
AT., Jatt Va. =
Aot (F1l(sas,ae) - (o))
(14)
We obtain the discrete ratio
ATa 01](2a,0) aa)
(15)
Aa
which is independent of Aa. The limit hence exists and defined by
dza lim
Azo Ata 311(x..a) Fo((nn.))-
(16)
da
A0 Aa
An
13

---

Under review as a conference paper at ICLR 2023
B.4 LIMIT OF ALGORITHM
In step 1 of Algorithm such that:
we sample random posterior samples Tul(za .a) and In (ila that are
Za. (1 ar) 2o((Har a.)) Ot 2i((Fo)
(17)
Tas+1 (1 a++1) 20((2a, ) + 0++1 2 Fil(ea,)a)))
(18)
where Equation 17 is property of the posteriors of Una and Equation (18) is true by definition in Algorithm
We thus have the discrete difference:
Az Jo.t1
Iaa Das (T1l(sa))a)) Tu((Fo,
(19)
We obtain the discrete difference for any parameter E [0, 1] and any location Ta E Rd
Aza Aa (Zi](z...)) (a)
(20)
Furthermore, increasing the number of steps is equivalent to decomposing each step Aa into N smaller steps Aa/N. We rewrite the discrete difference
Aa
N-1
Az
Σ N n=0
Fillsaonna//o.atnaaaaa 20
(21)
With this modification, if the derivative exists, defined by the limit:
dæa
ATo
lim
lim
(22)
da
Ao
Aa
N-1
= -im
lim
Σ
Til(za www.a//wwnn.aaaaae - (F. www.n.
(23)
40 N-
N
n=0
Thanks to the finite-variance condition of po and pi, the normalized average sum converges towards the average of the posterior samples over a' E (a, a Ao] as N increases.
dæ
lim 40-40
E ["']
E
(24)
da
a'Ela,atAa
Finally, because the expectations of the posterior densities are continuous, obtain that the expectations over [a, a + Aor] converge towards the expectation in o, such that
dra
E
(1)
E [o)().)))) = Fi](a..n)) - -n (x..:))
(25)
da
This is the same result as in Equation (16) with Algorithm 2
14

---

Under review as a conference paper at ICLR 2023
C
VARIANT FORMULATIONS
We derive the variant formulations introduced in Section 3.1
Blended samples. A blended sample is by definition the blending of its posterior samples
Fa. (1 02)) zo 0 I1-
(26)
Since blending is linear, blended sample is also the blending of the average of its posterior samples:
To. = (1 - 02)) To + of T1-
(27)
We can thus rewrite its average posteriors samples To and म in the following way:
To.
C2 T1
To = 1 Oit
(28)
1 0t. (1 (4x) To 31 For
(29)
a4
Variant (a): In the vanilla version of the algorithm, a blended sample of parameter 0441 is ob-
tained by blending To and FT:
Far+1
(1 a441) Fo + 0441 ø1-
(30)
Variant (b): By expanding To from Equation 30 using Equation 28 we obtain:
Fatt1 = (1 a1+1) To + a++1 I1,
(31)
To.
(1 - az) To
= (1 a++1) 1) 3g + a041
(32)
at
064 (1 aa) - -04111
To + ax++1
Toy
(33)
a4
at
at+1
at+1 Daet
Tu +
(34)
at
at
Zo + 0t++1
(xo,
- ūo).
(35)
at
Variant (c): By expanding I. from Equation 30 using Equation 29 we obtain:
Fatt = (1 ac+1) To + aa++1.0 T1:
(36)
Fa.
24 51
= (1 at+1)
+ #1411 31
(37)
1 1 a2
1 1-a1/ 0/
(1 (1- 1-00+1) at
1- 01+1 Bast
a1+1
Σ1 +
(38)
1 - a2
1 az
1 - 01+1
1- -044 Ioee
1
31 +
(39)
1- 0l
1- 04.
1 a++l
2 +
(Fos 21).
(40)
1 aq
Variant (d): By rewriting a1+1 = 0++1 + a, a in the definition of Tazt1 we obtain:
Foci1 (1 ax+1) Zo + a+++ 3T
(41)
(1 a1+1 + 0: an) 2o + (at+1 + - a1) Z1.
(42)
(1 -a,) uo + a J + (at+1 at) (0) To)
(43)
Fe.. + +aa+11 - a.) (क) -0)
(44)
15

---

Under review as a conference paper at ICLR 2023
D RELATION TO DDIM
In this section, we follow the notation [Song et al. (021a): Zo is a sample of a target density and e is a random Gaussian sample. The denoiser DDIM is defined such that, for an input It va + v1 - aze, it learns
( (xt)
(45)
We define
It
91
(46)
Voi + v Ott
= Bero + (1 - (c) c;
(47)
va
with 3 = Vax + vI az
Yt is an alpha-blended sample such as the one we defined in Section
follows that we have:
It
3/1
(48)
Va
Be
We now turn to Equation (13) of 'Snng et al. (2021a)
T++1
Oz
(0)
(It)
(49)
0t-1
0x
a+++1
az
By injecting into this expression of the scaled coordinate at line
48
we obtain:
(v
y++1
+
004
Oz
E
(50)
3+11
B.
a++1
a2
and
B1+1 Bit1 (1
1 1-a+1
a1+1
1 at
Ve Vt+1
(V
(51)
3r
a++1
at
B441
1
1 a++1
Oz
V1
Beb141
C,
(52)
3r
B3
06-1
04
since
(1
11-104+ a++1
I - at
= 3r (1 - -B4+1) - (1 -3:) Be+1
(53)
= 3g - Br+1
(54)
can simply line 12 to:
B
31 B141
Vc+1 = y1
+
(55)
3r
Bu41
3
ē + Vt
Br E,
(56)
3e
B++1
= E
(yr
E).
(57)
3,
This last form is exactly variant-(b) of IADB (see Table Tb. To validate this claim, we trained a DDIM denoiser and applied the rescaling formula the output samples. We show in Figure T2 that when we rescale the output of DDIM, the generated trajectory maps with IADB.
16

---

Under review as a conference paper at ICLR 2023
S
a) IADB
b) DDIM
DDIM rescaled (Equation 48
We trained a MLP with 5 hidden layers of 64 filters to learn Dg for IADB (a) and the same architecture to learn e for DDIM (b) and (c). For (c), we convert points generated by DDIM using the scaling equation. The trajectories of the samples for IADB (a) and DDIM rescaled (c) match.
Figure 12:
17


Reviews:
[{"review_id": "rdCHvUWiieW", "reviewer": "Reviewer_j8qM", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The goal of this paper is to develop a deterministic denoising diffusion between arbitrary densities without relying on complex concepts like Langevin dynamics or score-matching. Thanks to Bayes' theorem, it shows that iteratively blending and deblending samples produce random paths between arbitrary densities and proves this converges to a deterministic mapping that can be learned to deblend samples. Here, blending means generating samples from a convex combination of the start and end densities, while deblending means generating samples from the posterior distributions of the start and end densities given the blended distribution and the blending parameter.\n\nThe equivalence to the DDIM in a special situation has been established and a more numerically stable sampling process is proposed. The main evaluations on image generation with Gaussian noise as the target density indeed demonstrate a favorable sample quality according to FID. On the other hand, disappointing generation results are also illustrated between different image datasets.", "summary_of_the_review": "In summary, the paper provides a simpler but more flexible derivation of deterministic denoising diffusion models that works with arbitrary start and end densities and has the DDIM as a special case when the target density is Gaussian. The flexibility of the formulation also warrants different learning variants where one of them is more numerically stable than others and compares favorably with DDIM in terms of image generation quality measured by FID. Despite the disappointing results with arbitrary image densities, I think this work is a nice addition to the community. ", "strength_and_weaknesses": "Strengths:\n- The proposed iterative $\\alpha$-(de)blending (IADB) can be considered as a more general denoising diffusion process that can (in theory) work with arbitrary densities, which has the DDIM as one of special cases when the target density is Gaussian. The main idea is simple, and the nice thing about IADB is that the derivation does not rely on complex concepts which makes it more flexible. I really like the underlying motivation of the work to find a simpler approach to deterministic diffusion using only basic sampling concepts.\n- With the flexibility of the formulation, the authors are able to derive different learning variants where one of them is more numerically stable than others. The results on LSUN Bedrooms, CelebA and AFHQ Cats are favorable comparing to DDIM in terms of FID.\n\nWeakness:\n- The proposition on Page 3 and the proof in Appendix A could be improved. This proposition is a critical part in understanding the proposed method and the formula used for Bayes' theorem is not the famous one ($P(A |B) = P(B | A) P(A) / P(B)$), thus it deserves more discussion in the main text.\n- The experimental comparison between the proposed IADB and DDIM. It is nice that the authors try to control all the conditions so that the performance difference can be solely attributed to the different learning processes. But the resulting FID scores for the DDIM are considerably worse than those reported in the original paper, this may raise questions about the validity of the comparison.\n- The disappointing results with arbitrary image densities. As one of the main contributions of this work, neither the image generation nor restoration gives satisfactory results among arbitrary image densities. The authors argue that \"experimental set up that works well with Gaussian noise does not necessarily transpose successfully to other densities\". It would be much better if the authors could provide an architectural design choice that works. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "qILc3MkUi9m", "reviewer": "Reviewer_XE6Y", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes an algorithm to determine a mapping between arbitrary densities. The algorithm is particularly conceptually simple compared to e.g. Langevin dynamics. The authors illustrate the algorithm on a variety of examples. \n", "summary_of_the_review": "To sum up, the paper proposes a new method to map between densities. The method works well in Gaussian-to-arbritrary density setting. The paper is clearly written. Overall, my recommendation is accept.\n\nUPDATE: in light of the other reviews, and in particular that the discussed paper can be seen as a a special case of the previous work [1], I would like to lower my score. This and the weaknesses I highlighted previously outweigh the strong points of the paper.\n\n[1] Peluchetti, Stefano. \"Non-Denoising Forward-Time Diffusions.\" (2021).", "strength_and_weaknesses": "**Strengths**.\nThe paper clearly proposes a novel method to map densities. The theoretical background is sound and rigorous. The authors also support their claims by the empirical evidence. As the new method provides a significant improvement against the baseline, this is a major contribution to the field.\n\n**Weaknesses**\nEven though theoretically the approach could work with any densities, it turns out only Gaussian source density work well in practice. The authors give some alleged reasons for this behaviour but this could have been investigated a little more. Perhaps this is a direction for future work.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "bvHN6jsL4mq", "reviewer": "Reviewer_rt47", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a formulation of discrete diffusion models that do not rely on the concept of score matching or stochastic differential equations. Instead, authors leverage an interpolation function for samples from two distributions, and show that an iterative interpolation ($\\alpha$-blending) and inverse interpolation ($\\alpha$-deblending) process can transport samples from one distribution to the other. Authors further provide training objectives for this process and prove that it is a generalization of conventional deterministic diffusion models, such as DDIM. Empirically, authors obtain better sample quality than DDIMs with fewer sampling iterations. Lastly, authors report experimental results of using non-Gaussian noise for sample generation.", "summary_of_the_review": "The idea is simple, intuitive, yet very effective. Experiments provide insights into when the method works better, and when not. However, this work lacks discussion of some previous papers that explored closely related ideas.", "strength_and_weaknesses": "## Strength\n\n1. The formulation is simple yet effective. Writing is clear and easy to understand.\n2. The proposed method generalizes existing deterministic diffusion models (DDIM in particular), offering new insights into their inner workings.\n3. Negative results on image translation and non-Gaussian diffusion are thought-provoking.\n\n## Weaknesses\n\n1. The proposed idea is nice but not entirely new. The formulation can be reduced to a special case of the method in [1]. Specifically, one can set the SDE to $d x_t = (x_1 - x_0) dt + 0 dw$ (which actually becomes an ODE) and leverage Theorem 1 in [1] to derive the aggregated  ODE $d x_t = E[x_1 - x_0 | x_t] dt$, where the conditional expectation can be estimated by training a neural network $D_\\theta(x_t, t)$ with mean squared error minimization. This recovers the exact formulation in Algorithm 4, and yields the same exact training objective in equation (5) in this paper. Unfortunately, the work of [1] is not cited nor discussed in this paper.\n\n2. The connection between $\\alpha$-(de)blending and DDIM is implied in the work [2] (see Appendix D). Specifically, you can understand $x$, $\\epsilon$, $z_\\phi$ in [2] as $x_0$, $x_1$ and $x_\\alpha$ in this work. The (d) formulation in Table 1 amounts to $v$-prediction in [2]. In light of this connection, it is important to discuss and credit the work of [2].\n\n## References\n[1] Peluchetti, Stefano. \"Non-Denoising Forward-Time Diffusions.\" (2021).\n\n[2] Salimans, Tim, and Jonathan Ho. \"Progressive distillation for fast sampling of diffusion models.\" arXiv preprint arXiv:2202.00512 (2022).", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "SPOs-Y8azy", "reviewer": "Reviewer_W2Pp", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a new (bijective) transformation method between arbitrary pairs of distributions. The idea is to iteratively blend and de-blend the samples along the trajectories. Effectively, the direction in each iteration is the difference between the mean of the posterior samples in the two distributions. The method can be viewed as a generalization of DDIM, a deterministic sampling method for diffusion models. If the starting distribution is standard normal, they are equivalent up to the network parameterization. Experimentally, their method successfully transforms from one distribution to the other.", "summary_of_the_review": "Given the lack of basic baselines and the unclear practical value, the reviewer leans toward rejection for current version.", "strength_and_weaknesses": "### Strong points\n\n- The paper proposed a generalized version of DDIM that allows for bridging arbitrary distribution. The idea itself is interesting and provides an alternative perspective to the well-known deterministic sampling process of diffusion models.\n\n- The proposed training and sampling algorithms are straightforward to implement. \n\n- Experimentally, their method outperforms DDIM on several image datasets when starting from Gaussian, and they showcase the generalization to arbitrary pairs of distributions.\n\n### Weak points\n\n- The paper lacks some important baselines that can also bridge two arbitrary distributions. For example, people can use the bijection map in [1] to first transform distribution A to a distribution on a large hemisphere by the forward-process, and then transform the distribution on hemisphere back to distribution B by a learned backward-process. The simple \"bridging\" process can also define a bijection between an arbitrary pair of distributions.\n\n- The experiments in this paper do not back up the paper’s claim. In particular, the main claim of this paper is that the proposed method can generalize to a generative process between two arbitrary distributions (Section 6). However, also as the authors pointed out, in practice, the performances on these tasks are disappointing, and the positive results only come when fixing the prior sampling distribution to be a Gaussian one. The reviewer has one possible hypothesis: it's difficult for neural networks to learn the difference between two heterogeneous images, such as Leave and CelebA. Instead, the original approach of diffusion models (variant (b) in Table 1) would be better. Is it possible to verify the hypothesis and give the rationale behind the failure?\n\n- The dataset of pure image generation in Section 5 is easy -- they are all single-mode datasets. Could you provide experiments on more challenging datasets, such as CIFAR-10? In addition, it seems that the only difference between the proposed method and DDIM is the network target (or parameterization of the score function). Could you elaborate more on this part of the paper?\n\n- The second experiment in Section 6 is effectively an inverse problem. The author could resort to some manifold-constraint methods such as [2] to see if they help.\n\n### Writing suggestions / Minors\n\n- All theorems and propositions should be numbered.\n- The reviewer is confused about the listed baselines of DDIM. When sampling 128 steps, DDIM is shown to give 18.70 on CelebA $64^2$, while in the original paper, it is 6.53 with 100 steps. I am wondering why there is such a big discrepancy? If the reason is that the models have not yet been trained to convergence, I suggest the authors report the comparison at convergence.\n\n- The reviewer suggests changing the density notation from $f$ to $p$, as $p$ is more commonly used in the community.\n\n- In Section 1: (1) SDE (such as reverse diffusion) is not a variant of Langevin dynamics but a complementary method. (2) \" We derive a deterministic diffusion model ...\" The authors' model is more general than the \"deterministic diffusion model\".\n\n[1] Poisson Flow Generative Models, NeurIPS 2022, https://arxiv.org/abs/2209.11178\n\n[2] Improving Diffusion Models for Inverse Problems using Manifold Constraints, NeurIPS 2022, https://arxiv.org/abs/2206.00941\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject