Warm Diffusion: Recipe for Blur-Noise Mixture Diffusion Models
Authors: Hao-Chien Hsueh
Wen-Hsiao Peng
Ching-Chun Huang
Venue: ICLR 2025 Poster (2025)
Keywords: Diffusion probabilistic models
Image generation
Primary area: generative models

Abstract: Diffusion probabilistic models have achieved remarkable success in generative tasks across diverse data types. While recent studies have explored alternative degradation processes beyond Gaussian noise, this paper bridges two key diffusion paradigms: hot diffusion, which relies entirely on noise, and cold diffusion, which uses only blurring without noise. We argue that hot diffusion fails to exploit the strong correlation between high-frequency image detail and low-frequency structures, leading to random behaviors in the early steps of generation. Conversely, while cold diffusion leverages image correlations for prediction, it neglects the role of noise (randomness) in shaping the data manifold, resulting in out-of-manifold issues and partially explaining its performance drop. To integrate both strengths, we propose Warm Diffusion, a unified Blur-Noise Mixture Diffusion Model (BNMD), to control blurring and noise jointly. Our divide-and-conquer strategy exploits the spectral dependency in images, simplifying score model estimation by disentangling the denoising and deblurring processes. We further analyze the Blur-to-Noise Ratio (BNR) using spectral analysis to investigate the trade-off between model learning dynamics and changes in the data manifold. Extensive experiments across benchmarks validate the effectiveness of our approach for image generation.

Published as a conference paper at ICLR 2025
WARM DIFFUSION: RECIPE FOR BLUR-NOISE MIX- TURE DIFFUSION MODELS
Wen-Hsiao Peng Ching-Chun Huang National Yang Ming Chiao Tung University, Taiwan
Hao-Chien Hsuch
ABSTRACT
Diffusion probabilistic models have achieved remarkable success in generative tasks across diverse data types. While recent studies have explored alternative degradation processes beyond Gaussian noise, this paper bridges two key dif- fusion paradigms: hot diffusion, which relies entirely on noise, and cold diffu- sion, which uses only blurring without noise. We argue that hot diffusion fails to exploit the strong correlation between high-frequency image detail and low- frequency structures, leading to random behaviors in the early steps of generation. Conversely, while cold diffusion leverages image correlations for prediction, it neglects the role of noise (randomness) in shaping the data manifold, resulting in out-of-manifold issues and partially explaining its performance drop. To inte- grate both strengths, we propose Warm Diffusion, unified Blur-Noise Mixture Diffusion Model (BNMD), to control blurring and noise jointly. Our divide-and- conquer strategy exploits the spectral dependency in images, simplifying score model estimation by disentangling the denoising and deblurring processes. We further analyze the Blur-to-Noise Ratio (BNR) using spectral analysis to inves- tigate the trade-off between model learning dynamics and changes in the data manifold. Extensive experiments across benchmarks validate the effectiveness of our approach for image generation.
INTRODUCTION
Diffusion probabilistic models (Sohi-Dickstein et al. 2015 Ho al. 2020 Nichol & Dhariwal 2021 Song et al. (2022) have gained significant attention for their ability to learn data distributions through denoising. leading to impressive generation quality. These generative models typically employ stochastic process that gradually transforms complex data distributions into simpler forms by adding small amount of Gaussian noise in each forward iteration, eventually arriving at simple Gaussian distribution. The reverse process involves using neural network to model the score (Hyvärinen Dayan 2005) of a noise-levell-ependeen marginal distribution, iteratively adapting the denoised samples recover the input data distribution. However, the process of learning this score estimator coomain-agnosiee focusing solely on recovering the underlying signal by removing Gaussian noise without considering the inherent properties of the modeled data. While this universal approach is effective for various data modalities, we argue that leaves room for improvement in modeling images. Specifically, it overlooks the strong correlation between high-frequency image detail and low-frequency structures- relationship we term spectral dependency. This correlation suggests that an efficient image generation process should progress from common low-frequency components to diverse high-frequency detail.
Recently, a large number of studies (Bansal
2022 Daras et al.) 2022 Rissanen et 2023
Hoogeboom & Salimans] [2024] [uo et al.] 2023 Delbracio & Milantar 2024 Liu et al. 2023
Yue et al. 2023) Liu et al.] | (024) explored various alternatives to the conventional drise-driven forward/reverse process of diffusion models, with the aim of accelerating the reverse process and solving specific inverse or image translation problems. Some of these techniques adopt cold diffu- sion process, which replaces the stochastic Gaussian degradation process with deterministic image transformations, e.g. blurring. These advancements underscore the evolving nature of diffusion probabilistic models in formulating the forward/reverse process. Although these methods work well in solving specific restoration tasks, most of them struggle with generating diverse and high-quality samples, compared to typical noise-driven hot diffusion models.

---

Published as a conference paper at ICLR 2025
to Biue-zzic ristare diffoiie pracos (Soc-4.) 4.1)
(b enborgan 1eee. 4.) Depicalmnttie
Cagpar
Dusa
Pepstec
(e) Dua masiflli dui (Sec. 4.3)
Nokee level (
Gl Lawe BNR
Figure 1: Illustration of Warm Diffusion, the proposed two-pronged diffusion process. Graph- ical models of the proposed blur-noise mixture diffusion processes, offering flexibility in selecting blur and noise levels, thereby enabling a smooth transition between (1) Hot Diffusion and (4) Cold Diffusion. (b) The proposed divide-and-conquer strategy employs a joint model for denoising and deblurring. It recovers the blurry signal obscured by noise and restores missing high- high-frequency de- tail while explicitly accounting for the spectral dependency of images. (c) Data manifolds for the two diffusion processes at varying blur-to-noise ratios (BNR). The red and blue lines represent means of Gaussian distributions derived from samples sharing low-frequency content but differing in high-frequency detail. As BNR increases, the data manifolds shift and merge at earlier stages, as blurring filters out high-fiu frequecyy detail, leaving only the shared low-frequency signal. This shift may lead to out-of-mant of--mmffoloold issues, discussed in detail Sec.
As shown in Fig. we revisit the design of diffusion probabilistic models, expanding the pure de- noising process to joint denoising and deblurring approach. We discuss the limitations of existing hot and cold diffusion methods, particularly in terms of network learning and data manifold mod- eling. By taking into account the spectral dependency of images and the diversity introduced by Gaussian noise, we propose a method that balances blurring and noise levels. This enhances the learning process of the decoding neural network, maintains the diversity of the data manifold, and ultimately improves the quality of generated images. Our approach, targeting image generation, has the following contributions:
We propose a warm diffusion process that combines blurring and noise the forward process. The scheme allows flexible control over blur and noise levels, enabling joint de- blurring and denoising to enhance image generation quality.
We introduce the new concept of Blur-to-Noise Ratio (BNR) control and show that in- creasing the BNR (leaning toward cold diffusion) simplifies model learning by leveraging spectral dependency effectively. However, this also increases the risk of samples deviating from the data manifold during the reverse process.
We select the BNR by analyzing the spectra of images and Gaussian white noise. The dif- ference in their power spectral densities guides to find suitable BNR that balances two key factors: preserving the integrity of the data manifold and simplifying neural network learning.
Extensive experiments across various datasets show that our approach outperforms state- of-the-art diffusion methods in terms of image generation quality.
RELATED WORKS
2.1 DIFFUSION PROBABILISTIC MODELS FOR IMAGE GENERATION
Diffusion models eo al. 2020) consist of two key processes: a forward process that progressively transforms the data distribution into Gaussian distribution by adding noise and reverse process
2

---

Published as a conference paper at ICLR 2025
that learns denoise and recover the original data distribution. Various noise scheduling strate- gies have been explored across different studies. IDDPM (Nichol & Dhariwal, 2021) introduced a cosine noise schedule that gradually destroys the signal slower rate. Score-SDE (Song al. 2021) provided a unified framework that represents models like SMIDD (Song & Ermon 2019) and DDPM (Ho al. 2020) within continuous state space, using different diseretizations and distinct Stochastic Differential Equations (SDEs). EDM (Karras et al., 2022) further elucidates the design space of diffusion models. Extensive discussions within this work provide detailed analyses train- ing and sampling strategies. including preconditioning the network's 's input and output and adjusting the loss function. These enhancements, which consider neural network properties by maintaining unit variance for the model's input and output and mitigating large gradient variations, have led to significantly improved results. However, these improvements primarily address challenges posed by Gaussian noise without leveraging key image properties, such as spectral dependency, and they overlook the potential of alternative corruption processes.
2.2 DIFFUSION-LIKE MODELS WITH VARIATIONS OF THE CORRUPTION PROCESS
Recently, researchers have explored modifications to the corruption process within diffusion mod- els. Cold Diffusion (Bansal et al., 2022) replaced traditional noise-based transition functions with transformations such blur, masking, and pixelation. This innovation created diffusion-like gen- erative model by inverting arbitrary image transforms; however, results indicate that cold diffusion struggles maintain high-quality outcomes. Concurrently, IHDM (Rissanen et al., 2023) intro- duced a progressive blurring process, where the model learns to iteratively restore blurred signals, essentially acting as the "inverse" of heat dissipation. Blurring Diffusion (Hoogeboom & Salimans 2024) established connection between IHDM and Gaussian diffusion (Ho et 2020), demonstrat- ing that IHDM can be interpreted as form of Gaussian diffusion in the frequency domain, albeit with different schedules across frequency bands. By integrating the blur and noise schedules from both IHDM and iDDPM (Nichol & Dhariwal 202T}, Blurring Diffusion achieved better generation quality compared to IHDM, though at the cost of more training iterations. However, these afore- mentioned approaches primarily focus on the mean transition function and lack a global perspective that jointly considers the role of Gaussian noise within the diffusion process, which contributes to drop in performance.
PRELIMINARY: DENOISING DIFFUSION IMPLICIT MODELS
Considering a class of inference distributions, indexed by vector a:
T
go (117 = go (0T
II
go (1,-1 xo),
(I)
t=2
where given 2o (#r(20) = (VVor (1 ocr) I), there exists a family of posterior distributions for all t >>> 1:
go (11-1/01, xo) = NN(Vva1100 + V V1- a1 - of 21 - aa of)).
(2)
VI- a4
such that go (x, (xo) = NN(a/ro (1 1 a) for all f, which matches the marginals as DDPM (Ho)
let al. 2020).
The trainable generative process is defined as Markovian where each po (= i (z,) aims to utilized
the knowledge of qo (2x,-11 1 mo). In sense, given a noisy observation 2s, first predict
denoised observation D (x,), and then obtain _T-1 with ga (x,-1 ( (xx)). The learning
objective can be therefore parameterized as:
L(D²) X(t) (() -x.l.
(3)
4 BLUR-NOISE MIXTURE DIFFUSION MODELS (BNMD)
To enhance image generation, we introduce a blur-noise mixture diffusion model (BNMD). BNMD extends the state space of the diffusion process to two dimensions, controlled by the corruption
3

---

Published as a conference paper at ICLR 2025
Forward process
Hlsmin
NostE
HQ space
UNN (1s 10]
Reverse process (a) Model prodiction
Forward process
Sumple ale
L.Net
Spii
Deblar
Denstio
step (Eqq.
LQ
Sample
Demotioc
M.N1 M.
Data
Blucting
Neise covering space
Intermediate sample
Deblurring
Denoising
Figure 2: Workflow of the proposed diffusion process. The forward process progressively applies blurring and noise, controlled by the Blur-to-Noise Ratio (BNR), to degrade the sample from high quality to low quality. During this phase, training pairs are collected to train the prediction model (e.g., U-Net) for use in the reverse process. For sample generation, the reverse process works as follows: (a) The prediction model simultaneously performs denoising and deblurring. (b) With the prediction results, the reverse step transitions the sample from step denoiser gradually guides the sample toward a blurry prediction, while the deblurring prediction helps return the sample higher-quality state.
1. Specifically, the
factors and B. which represent the blur and noise levels, respectively. In the forward process, the schedules of and /3 determine a blend of blurring and noising operations. Consequently, the reverse process iteratively recovers high-quality image by deblurring and denoising sampled image drawn from prior distribution (i.e., standard normal distribution). As shown in Fig. 2 mixing deblurring and denoising throughout the iterative image generation process distinguishes our approach from most existing diffusion models, which typically generate images through denoising alone. Moreover, our forward and generative processes are defined similarly to DDIM (Song al.) 2022), but with focus incorporating blur-noise operations into both stages. For brevity, primarily address the key terms from DDIM (Song al. 2022) that require adaptation for our model.
4.1 BLUR-NOISE FORWARD DIFFUSION PROCESSES
Our blur-noise forward process has the marginal distributions for {1...., as:
(F xx)) N(VMa, V Tu. BII),
where VT and V denote the forward and inverse Discrete Cosine Transform (DCT), respectively. May, a diagonal matrix, specifies the Gaussian blurring mask in the DCT domain, which varies according the Gaussian blur level Ot. The parameter 3r controls the level of Gaussian noise. The corruption sequences aj,..., oer and 31,..... ,,. are defined as monotonically increasing se- quences. As with DDIM (Song al. 2022 Eq. (4) requires the inference transition distributions (i.e.. those used in the reverse process) (.) for all f 1 (see Appendix A.1 to be:
VM V xo
do (War 1,8, 11 Wa Wo) NVVMa, VT / of Ca
I).
3g
(5)

---

Published as a conference paper at ICLR 2025
4.2 GENERATIVE PROCESS IN A SIIIDE-ADDCOONCEE MANNER
Our generative process is a Markovian process specified by learnalle transition distributions Pa(Bo, 1.f4 1 (o, for & > 1. The training of these distributions involves maximizing vari- ational lower bound that requires minimizing among others the sum of the KI--divegeecce terms (F 20o (P Ta..9.) for t >
Notably, parameterizing pa 1 (o....)) is to leverage the knowledge about do (2: L & Fu) and learn mean function predict the mean in Eq. (5) based on the blurry-nois observation &. At the inference time, we do not have access to the input Do: as such, one straightforward approach parameterizing 11 (.., is to learn a network @o Fo Fo( that makes prediction of zo from Ba and have po(wo, 1.8, ( = yr (Fo, 1.9, 1 .0).
Instead of predicting Zo from Far.8, directly with single network, this work introduces novel divide-and-conquet approach to parameterizing po po (o.,),). This is motivated by the observation that the mean in Eq. (5) can be expressed alternatively as:
VMo, V xo Dae Dt.
Fo V(M MA, VV -0 (3.
V
P of)
Be direction pointing blurry atp
add missing (requenyyyyccy detail
where we have additionally added and subtracted the same term Boc..: V Ma, VT mo. This alternative expression suggests that the task of parameterizing po (aaa- 11--1----- [o by learning a mean function to predict that of Eq. (5) can be decomposed into two sub-tasks: deblurring and denoising. The former aims to reconstruct the high-frequency detail of 2o via the iterative generation process, while the latter to recover a blurry version (i.e., V VM,, V an)) of zo from its noisy observation (i.e., 2, by focusing on the reconstruction of the low-frequency components of zo- This interpretation allows us learn two specialized networks for addressing these separate sub-tasks, leading to the more efficient and accurate generation of output images.
Specifically, we learn a network Dg that takes the blurry-noisy observation Fo,..% VMa, VT xo + Sic, c ~ N( I) as input to predict the noise-free yet blurry representation VMa, VT To of Zp. When learned successfully, Do is able to denoise Ta,,8.- For deblurring, a separate network Rg, which takes the same & as input, is learned to update VM. V xo as Zp. That is, Rg aims to recover the missing high-frequency detail in VM, V Tg. in order to re- construct øo. symbols, Ro is meant to predict Tres, xo - -Ma, V xo V(I Ma, VT zn- With our proposed parameterization, and given that Ma, is diagonal matrix, the mean function of pa(#o.- 1. (o, is:
Ba + V(Ma- Mo.)(I 10.M((....-- Ra O4 Bc)
V off Do Da(2ar at, 3r) Po..6 + (B,
Considering the equation Fres, V(I - Ma, V mo, the second term V(M (Ma.- Ma ) V
in Eq. ( can be rewritten as V(M..,, M.. (I (I-M..) Mo.)
- Fres, to match the form of its
counterpart in Eq. (7). To approximate Eq. (6) using Eq. 7). we then train Dp and Ro by minimizing
the following mean square error (MSE) losses
L(Do) = L(Ra) = [Ro at: 3.) - x Dres, l2.
ar 30) - (VMa, VT 2x) and
(8)
(9)
Here, given a noisy and blurry observation Fn,,8. = VM., VF + 3,E at time 4, along with the corresponding corrupiion factors, an and 3,, Do (Denoiser) is trained to recover the pure blurry image by removing the noise component, while Rg (Deblurrer) extracts the high-frequency detail Trese- Finally, at time the prediction of the clean image is given by d.g- Fo Do + Ro-
4.3 THE IMPACT OF BLUR--O-NOISE RATIO ON MODEL BEHAVTOR AND DATA MANIFOLD
We have demonstrated how to combine two degradation factors blurring and noising into a uni- fied diffusion process and introduced training objectives that simultaneously address deblurring and
5

---

Published as a conference paper at ICLR 2025
(50 INN--
(b) WN8-005
(3) BNN-
Revideal signal
Renictal signal
Resictal signal
04.
xprt
12-1
10-1
13
Powerr spstral daneity
Poweer spetral spetral donetty
12.
spstral
Uiide lying signal Re.0
Underbying signal Ausaa
Underlying signal R4
11"1
13 Froquency
14"1
13 Froquency
13"
27 Froquency
Observed sianal
Observed sianal FLSAS
Observed sisnal
Debhluring objeztive
Deroicing objoctive
Blurry sigral
Gaseeise trise
Ulbervee cigral
Figure 3: Impact of Varying BNRs on Model Behavior. We illustrate the observed signal, denoising target, and deblurring target, along with their respective signal spectrum analyses, across different BNR values. From left to right, the noise level remains constant while the BNR value increases. As the BNR rises, the denoising task (red arrow) becomes progressively easier, shifting more responsi- bility to the deblurring task (blue arrow) and effectively utilizing the spectral dependency of images. In contrast, when BNR the model requires a stronger denoiser to directly generate the image, without leveraging the spectral dependency assistance from the deblurrer.
denoising for image generation. However, the relationship between the blur and noise levels remains unclear. To explore this connection, we define a factor called the Blur-to-Noise Ratio (BNR):
Blur Level
a 3
BNR = Noise Level
(10)
which represents the ratio of the blur level to the noise level. As illustrated in Fig. increasing the BNR value from 0 to oo transitions the diffusion path from hot to cold diffusion. To further inves- tigate the model behavior with varying BNR values, examine the learning objectives introduced in Sec. 4.2] which consist of two distinct branches targeting denoising and deblurring, respectively. Fig. shows the signal spectra of images and the corresponding training objectives for different BNR values. With a constant noise level, an increase in BNR would raise the blur level, thereby simplifying the denoising task. This shift occurs because the denoiser no longer needs to restore high-frequency detail, transferring that responsibility to the deblurring task. Additionally, by effec- tively utilizing spectral dependency, the deblurrer can efficiently learn a mapping function from the low-frequency observation to its high-frequency counterpart.
In addition to affecting model behavior, varying BNR values also lead to shifts in data manifolds during forward iterations. To explore this in greater detail, we compare the data manifolds under low and high BNR scenarios with the same blur level in Fig. 4 At each forward or reverse step, low BNR cases exhibit larger noise-covering space due to higher randomness compared to high BNR cases. Consequently, a high BNR results in reduced data diversity, making generated samples more likely to fall out of the data manifold, particularly during the early forward steps. When encountering out- of-manifold cases in sample generation, the diffusion network must handle degraded samples that were not seen during training, leading to less predictable outcomes and lower generation quality.
The analyses above reveal an inherent trade-off between model learning dynamics and data manifold shifts, where the choice of BNR value plays crucial role in generation quality. As the diffusion process transitions from hot to cold, the model increasingly depends on leveraging the spectral dependency of images for learning. However. this shift also introduces the risk of divergence from the data manifold during generation, potentially resulting in degraded performance.
4.4 SELECTING BNR

---

Published as a conference paper at ICLR 2025
Low BNR
High BNR
Low
Blur level
High
Blurring
Forward sample
Noise covering space
Initial sumple
Deblurring step
Reverse sample
Deblurring prediction
Final sample
Figure 4: Illustration of the connection between BNR and the data manifold. When comparing two different BNRs the same blur level, a higher BNR corresponds in narrower noise-covering space, as shown the right. In the deblurring (reverse) step, sample guided toward the deblurring target, representing the mean image of all possible paired outputs. It is important to note that during the forward process, single low-quality (LQ) sample is typically paired with multiple high-quality (HQ) samples for training. Due to this ill-posed nature of the deblurring task, samples with higher BNR values more likely deviate from the data manifold during the transition. Once samples fall out of the manifold, the neural network struggles to produce accurate predictions, leading decline in generation quality.
smaller noise scale, resulting
We've discussed how BNR influences model behavior and the data manifold. A higher BNR allows the model to better exploit spectral dependency, simplifying learning by shifting the focus to de- blurring. However, it also heightens the risk of samples deviating from the data manifold. Striking the right balance is crucial, as one must weigh the benefits more straightforward neural network training against the risk of such deviations. This raises the question of whether better BNR value exists that preserves the data manifold's ' integrity while enhancing model training.
An interesting observation from previous studies (van der Schaaf & van Hateren 1996; Rissanen cct al. 2023) is that the power spectral density of natural images follows an approximate power law, 1/ where all frequency bands. This discrepancy results in a much lower signal-to-noise ratio (SNR) in high- frequency bands compared to low-frequency ones. When the SNR in these bands is sufficiently low, the observed signal becomes dominated by noise, helping to maintain the integrity of the data manifold while attenuating the image signal in these bands.
2. In contrast, Gaussian white noise exhibits flat frequency response across
As shown in Fig. b), our empirical findings indicate that selecting BNR=0.5 causes the image signal begin attenuating when noise intensity exceeds the signal in these frequency bands, keep- ing the nousy signal comparable to the noisy signal in hot diffusion. Beyond this threshold, Fig. () illustrates that for higher BNR value, the observed signal diverges from that in hot diffu- sion, as the image signal attenuates significantly before noise dominates those frequency bands.
5 EXPERIMENTS
5.1 IMAGE GENERATION
Datasets. We validate our proposed diffusion process on three widely used benchmarks: CIFAR-10 Krizhevsky 2009) 32 x 32, FFHQ Karras al. 2019) 64 x 64, and LSUN-church (Yu al. 2016) 128 x 128. These datasets were chosen to demonstrate the effectiveness of our method across various scenarios. The CIFAR- 10 dataset contains 32 x 32 color images across 10 classes, allowing to

---

Published as a conference paper at ICLR 2025
Table 1: Quantitative results and comparison for 32 x 32 and 64 x 64 image generation tasks on CIFAR-10 10 (Krizhevsky, 2009) and FFHQ (Karras et al. 2019) datasets correspondingly. Lower FID and higher IS scores indicate bet- ter sample quality. NFE denotes the "Number of Function Evaluations". The best results are highlighted in bold; the second-best results are underlined.
Table Quantitative results and comparisons for 128 x 128 image generation tasks on the unconditional LSUN-chureh (Yu et al. 2016) dataset. For fair comparison, we evaluate sam- ple quality using the same number of samples as in previous studies.
Methods
NFE
FID 1
Number of samples 10k
Denoising Diflacion Hourchoom & Salimans 7024
1000
4.68
Bluring Dilfission Hnngaseen & NL Nammari b004
1000
5.88
Mothodk
NFF
FID J
IS
Ouns
511
3:47
Uncondiiooall CIFAR-10 10
Number samples 50k
Cold Diffrcion (Rler) et 2002
50
80.08
IHDM (Rissanen et all Ours
Blundnn Uhirecce Honrgooo & Salimnas 2014
IIDM
Rissanen el Kura
1000
200
18.96
3.17
9.51
511
2.56
EDM Kapas
35
1.97
9.78
HDM
Table 3: Ablation study on the impact of dif- ferent BNR values for CIFAR-10, with a fixed number of sampling steps (NFE=35).
PFG
XU
Ourk
Class conditional CIPAR 10
RDM Karss et 2002
35
1.79
EDM 03 N00
kud
35
1.30
BNR
FID J
P
Au
35
1.74
0 (EDM
et al. 2022
1.97
9.78
Ours
35
1.68
10.19
0.1
1.97
9.96
RFHQ
0.3
1.90
10.02
Karge &
0.5
1.85
10.02
PRIM
2.43
0.65
1.91
10.00
Ozrs
79
2:29
3.41
1
2.01
9.96
2
2.57
9.89
11.97
test both unconditional and class-conditional image generation. For the FFHQ and LSUN-church datasets, we evaluate the model in unconditional settings. The FFHQ 64 x 64 dataset comprises 70,000 images of human faces, which have higher degree of shared structure compared to general scene datasets. The LSUN-church 128 x 128 dataset features images of church scenes, enabling us to validate our method in higher-resolution scenarios.
Implementation Details. For training, we adopt the improved DDPM+ (Song et al., )021 network architectures, training strategies, and hyperparameters from the state-of-the-art dif- fusion model, EDM (Krrras et al. 2022). Modifications are made to enable the network to accept two conditioning signals-tth the blur and noise levels-and we double the output channels to produce predictors for deblurring and denoising, respectively. In our current design, the two networks share most components, so altering the BNR keeps the model capacity nearly constant. ensuring fair comparison. For sampling, we adapt Heun's 2nd solver, following EDM (Karras et al. 2022). More details are available in Appendix B
Performance Comparison. To evaluate image generation quality, we use two commonly adopted benchmarks: Fréchet Inception Distance (FID) (Heusel et al. (0018) and Inception Score (IS) (Sal- imans et al.] 2016). The FID score measures the distance between the generated and reference datasets; a lower FID score indicates greater similarity between the two, reflecting better recovery of the data distribution by the generative model. The Inception Score is computed by passing gen- erated images through a pre-trained classifier. The optimal IS is achieved when the entropy of the label distribution for the generated images is minimized and the predictions are evenly spread across classes, indicating sharp and diverse generated images.
Following established procedures, we sample 50,000 images over three rounds and report the mini- mum scores to mitigate random variation effects. As shown in Tab. [ we assess sample quality using FID and IS alongside the number of function evaluations (NFE) during sampling-2 metric closely related to the sampling speed of diffusion- methods. Our approach significantly enhances the performance of the baseline model, EDM, across both CIFAR- 10 and FFHQ datasets, regardless of their differing characteristics. This improvement is evident in both conditional and unconditional settings CIFAR-10, outperforming recent methods designed enhance EDM.
8

---

Published as a conference paper at ICLR 2025
Hot Diftusion
Ours Blurring Diffusion
Diffusion
IHDM
Noise
(an)))
8NR 0.5
BNR - 0.65
BNR =10
tan
2.6
BNR
tanC)
2.2
(an())
2.0
1.8
30
45
toni
tan
ta
tan
NFE
Noise
Figure 5: Illustration of sample quality corre- sponding to different BNR and NFE. Each curve represents a specific BNR value. As shown the chart, higher BNR values require more sam- pling steps to achieve better sample quality.
Figure 6: Comparison of blur and noise sched- ules with previous methods. Most related stud- ies, except the hot diffusion, selected higher BNR values in their schedules, leading out- of-manifold issues. Details in Appendix A.2
Moreover, in addition comparing our approach with techniques aimed at improving hot diffusion, we evaluate it against Cold Diffusion and intermediate methods such as IHDM (Rissanen et al.) 2023) and Blurring Diffusion (Hoogeboom & Salimans 2024). Our method not only achieves su- perior sample quality but also requires sigmiticantly fewer sampling steps for image generation. In Tab. 2 we also show our method outperforms the previous methods on a more complex and higher- resolution dataset, LSUN-church 128 x 128. To make fair comparison, we use the same number of samples for the FID evaluation since the FID score sensitive to number of samples.
5.2 ANALYSIS OF DIFFERENT BNRS AND SCHEDULING
Relation between BNRs and Data Manifolds. We first validate our assumptions by testing various BNR values ranging from to 10. As shown in Tab. 3 using the same number of sampling steps as the baseline method, EDM (Karras et al. (022), the sample quality improves as the BNR value in- creases up to 0.5. Beyond this threshold, performance rapidly declines, eventually falling below the baseline. These results support our hypothesis in Sec. 4.4that a BNR value of 0.5, guided by spectral analysis, can effectively balance the trade-off between mitigating negative impacts on data manifold shifts and enhancing model learning. As discussed in Sec. 4.4 when the BNR value exceeds 0.5, the data manifold tends to shift significantly. Consequently, following the same sampling steps, the samples are
likely fall outside the manifold, resulting in lower generation quality. This suggests that higher BNR values may require additional sampling steps to mitigate out-of-manifold issues and enhance sample quality. In Fig. 5 we further assess the impact of different BNR values and sampling steps on sample quality. The results confirm that higher BNR values indeed demand more sampling steps to reach better sample quality, supporting our analysis.
Revisiting the BNR Scheduling in Prior Studies. We conduct experiments to compare the BNR schedules from prior studies with our proposed method. Specifically, we re-implement the BNR schedule from Blurring Diffusion (Hoogeboom & Salimans] 2024) under the same experimental conditions to eliminate the effects of differing parameterization techniques. The results, presented in Tab. 4 indicate that Blurring Diffusion's is BNR schedule results in poor generation quality when using fewer sampling steps, although quality improves significantly with more steps. This highlights a limitation in Blurring Diffusion's 's BNR scheduling, depicted in Fig. 6 where higher BNR values necessitate additional sampling steps to prevent the reverse process from deviating from the data manifold. Consequently, fewer steps result in poorer sample quality. These findings help explain why previous approaches, such as cool diffusion, have struggled to generate high-quality samples, particularly under limited sampling budgets.

---

Published as a conference paper at ICLR 2025
Table 4: We re-implement Blurring Diffusion using our parameterization and training scheme on
CIFAR-10. Results marked with are those reported by Hoogeboom & Salimans 2024
BNR Schedule
NFE
FID 1
IS f
12.97
8.57
79
4.13
9.27
159
2.91
9,46
Blurring Diffusion
(Hoogeboom & Salimans
2024
239
2.77
9.52
319
2.73
9.54
399
2.71
9.54
999
2.68
9.56
Blurring Diffusion"
HHooeeboom & Salimans 2024
1000
3.17
9.51
Table 5: An ablation study on different parameterization of zy and VMa VTzg in Eq. 6 We
fix constant number of sampling steps (NFE=35) to investigate various parameterizations for the generation task using CIFAR-10.
Training Objectives
Parameterization of
Rø
Do
20
VM., V x
FID +
(a)
Rs V(Mo,) -V De
VMa V Rs
13.19
(b)
VM., V 20
De
9.09
(c)
VMa V TO
Ro
Do
1.98
(d)
Dreeee
VM, V 200
V(I Mo.) -oo) -VT VTTH Re
De
1.85
5.3 ABLATION STUDIES TRAINING OBJECTIVES AND VARIATIONS OF PARAMETERIZATIONS
In Sec. 4.2 we reformulate the reverse function, Eq. (6) to simplify the neural network's training objectives through divide-and cnnnque approach. This method separates the task of predicting the clean signal 2o into two sub-tasks: denoising to a blurry signal and predicting the residual signal for deblurring. this subsection, we explore various parameterization strategies derived from Eq. (6) and evaluate their performances. demonstrating the advantages of our and-uonuuer strategy for model learning.
As shown in Tab. ala), using single branch to directly predict the entire clean signal results in significantly poorer generation quality, as this task proves too challenging. In Tab. 5bb). shifting the target to learn blurry signal yields slight improvement in sample quality since this target is easier to model; however, it still faces issues with inaccuracies in high-frequency components, which may be exacerbated during sampling, occasionally resulting to noisy patterns in the generated samples. In Tab. ()c), employing two branches to predict both clean and blurry signals effectively addresses these challenges, leading to substantially better results, though they remain comparable to those of hot diffusion models as EDM (Karras et al. 2022). Finally, in Tab. (id). the proposed divide-and- conquer strategy further improves performance, benefiting especially from the BNR schedule and the parameterization. More visual examples are provided in Fig.
CONCLUSION
In this paper, we introduce a unified Warm Diffusion framework that effectively bridges the gap between and cold diffusion models while addressing their inherent limitations. Our analysis reveals that hot diffusion models underutilize the spectral dependency of images, whereas cold dif- fusion models risk reverse sampling steps that deviate from the data manifolds. By examining the Blur-to-Noise Ratio (BNR), uncover its significant influence model behavior and data man- ifolds. This insight enables us to propose a strategy for balancing the trade-off between hot and cold diffusion, ultimately enhancing diffusion models for image generation. Experimental results across various benchmarks validate the effectiveness of our approach, demonstrating improvements in sample quality over state-of-the-art diffusion models.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This work was financially supported in part (project number: 112UA10019) by the Co-creation Platform of the Industry Academia Innovation School, NYCU, under the framework of the National Key Fields Industry- University Cooperation and Skilled Personnel Training Act, from the Ministry of Education (MOE) and in- dustry partners in Taiwan. It also supported in part by the National Science and Technology Council, Taiwan, under Grant NSTC- DSTC-112-223) 11-12221-EE-- 1--1-------0---- A44-089-MYY- Grant NSTC- 1 10-2221-EBB 1-----4--066------- Grant NSTC-111: 2634-F-A49-010, Grant NSTC- 12- 2-25-H-A2--- H-A49-001, and in part by the Higher Education Sprout Project of the National Yang Ming Chiao Tung University and the Ministry of Education (MOE), Taiwan. We also would like to express our gratitude for the support from MediaTekk Inc, Hon Hai Research Institute (HHRI), E.SUN Financial Holding Co Ltd, Advantech Co Lid, Industrial Technology Research Institute (ITRI)
REFERENCES
Arpit Bansal, Fitan Borgnia, Hong-Min Chu, Jie S. Li, Hamid Kazemi, Furong Huang, Micah Goldblum, Jonas Geiping. and Tom Goldstein. Cold diffusion: Inverting arbitrary image transforms without noise, 2022. URL atttss://arzte
httg://b//2200/ 09392
Christopher M Bishop and Nasser M Nasrabadi. Pattern recognition and machine learning, volume 4. Springer, 2006.
Giannis Daras, Mauricio Delbracio, Hossein Talebi, Alexandros G. Dimakis, and Peyman Milanfar. Soft diffu- sion: Score matching for general corruptions. 2022. URL btttp:://arxiv.orr//aaaaa
Mauricio Delbracio and Peyman Milanfar. Inversion by direct iteration: An alternative to denoising diffusion for image restoration, 2024. URL /ettpp:://axxiv.orr/bbsss
11435
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Berhhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium, 2018. URL abs 1tt./7776..000 08500
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models, 2020. URL. https:
//axxiv.crg/ abs/ 11239
Emiel Hoogehoom and Tim Salimans. Blurring diffusion models, 2024. URL http:tttt///axii.rrr////
2209.05557
Aapo Hyvärinen and Peter Dayan. Estimation non-normalized statistical models by score matching. Journal of Machine Learning Research, 6(4), 2005.
Tero Karras, Samuli Laine, and Timo Aila. style-based generator architecture for generative adversarial networks. In 2019 JEFE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4396- 4405, 2019, doi: 10.1 00.1099CCPR.2219.00045
Tero Karras, Miia Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models, 2022. httphhtsss//ariiieeep/ http://rrgggaabaabeeeeeeeee abs//206...2./////////// 00364
Alex Krizhevsky. Learning multiple layers of features from tiny images. 2009. URL https://api. semanticscholest wwwg/ooorgooDaa. 182668444
Guan-Horng Liu, Arash Vahdat, De-An Huang, Evangelos A. Theadorou, Weili Nie, and Anima Anandkumar. ss::: Image-to-image schrödinger bridge, 2023. URI. httphhtssss/aaiiiieeee//
wwtp:///////aaa//aaaa/////// http::pppp///aav.ooorr//////// 05872
Jiawei Liu, Qiang Wang, Huijie Fan, Yinong Wang, Yandong Tang, and Liangqiong Qu. Residual denoising
diffusion models, 2024.
13712
Ziwei Luo, Fredrik K. Gustafsson, Zheng Zhao, Jens Sjölund, and Thomas B. Schön. Image restoration with mean-reverting stochastic differential equations, 2023. URL 11699
Alex Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models, 2021. URL https: 09672
//arxiv.crg/aatt
Mang Ning, Mingxiao Li, Jianlin Su, Albert Ali Salah, and Itir Onal Ertugrul. Elucidating the exposure bias in org/ 2abs/2333 133..1332111111111
diffusion models, 2024. /tth:ts::
Severi Rissanen, Markus Heinonen, and Aroo Solin. Generative modelling with inverse heat dissipation, 2023. hwll/tttts///rrii..ooobbbb.ee 13397

---

Published as a conference paper at ICLR 2025
Tim Salimans, Ian Goodfellow. Wojeiech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans, 2016. Ut/pstpps//aralvoot/tbbe///
Jascha Sohl-Dickstein Eric A. Weiss, Niru Maheswaranathan. and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. 2015. URL /al.attt///axvv.o/////aa/a
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models, 2022. URL https:
//arxiv.cro wwt.p//g../aaaeeeeoeoo...o..esps
abs/ abs//010.....eo.o./..po.aaaaaa.a 2010. 02502
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distri- bution. Н. Wallach, Н. Larochelle, A. Beygelzimer, d'Alché-Bue, E. Fox, R. Gar- nett (eds.), Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL https: /ttp::://////eeeee wtw.://wwwddn........e.eee.eee.e /appeppprr /t.esspaper/22199/5 300
Yang Song, Jascha Sohl- Dickssiinniin Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations, 2021. URL httll::t///sattt
a2g/abs/2014 13456
A. der Schaaf and J.H. Hateren. Modelling the power spectra of natural images: Statistics and in- formation. Vision Research, 36(17):2759 2770, 1996. ISSN 0042-6989, doi: https///do..rr//0000006 0042-6989 6980009000) 8. URL / http::/wwwww.cleeeeeeeeeee... wwt.//o///ccceeeee httpa//c....ooooooo////////////. 00426939960000228
Yilun Xu, Ziming Liu, Yonglong Tian, Shangyuan Tong, Max Tegmark, and Tommi Jaakkola. Pfgm++: Un- locking the potential of physics-inspinn generative models, 2023a. URL 2302. 04265
Yilun Xu, Shangyuan Tong, and Tommi Jaakkola. Stable target field for reduced variance score estimation in diffusion models, 2023b. URL Ittpp::/ /atp:v///aaivaaaaa wwwpivv.ogg/ae. org. orggabs//////oooo..///./////.bar abs 2202 00670
Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. Lsun: Construction of a large- scale image dataset using deep learning with humans in the loop. 2016. URL httpa:://paaxp>>>> org/ abs /1506. 03365
Zongsheng Yue, Jianyi Wang, and Chen Change Loy. Resshift: Efficient difflusion model for image super- resolution by residual shifting, 2023. URL htttps://axxix orge htrg/abb//3001 12348
A DERIVATION
A.1 PROOF
In this section, prove that the inference transition distribution defined in Eq. 5 matches the marginal distribution defined Eq. using induction hypothesis.
Base Case:
For = 7', is given that
lo (Zar. -N(VMa, V (2
(II)
so the base case holds.
Induction Hypothesis:
Assume that for the following holds
9 (m (mo) (VVMa, V xo, T.
(12)
now aim to show holds
Inductive Step:
We have
qr (wn, 1. [wo)
ye (xams (20)), (a., 1. 1.00 94 Wo (
(13)
Zoc.0e
and also the inference transition distribution
VMa, V xo j1 (14)
Go (Zaa-1180 (aa .P: xo) NVV Ma.,, V=2 1 o7( - Boe
12

---

Published as a conference paper at ICLR 2025
Bishop & Nasrabadi (1 21r-1
Following
(006) (2.115), have that do (Ro, 1.0, (20) is Gaussian, denoted
where
VM Tyap To
VMa, V m0)
ph- = VMo V vu + B 07
Bt
(15)
VM V
and
5 σi
E.-1 = 0
10
Pi-1I
(16)
31.
Therefore we have
Ga (w (2 Pi- [kn) = (V V wo, 1
(17)
Conclusion:
marginal distribution go (M (mro) N(VMa, V æo, I) holds for according to the derivation above.
A.2 BNR SCHEDULE PRIOR STUDIES
Here, we further analyze the BNR schedule prior studies as depicted in Fig. 6
Blurring Diffusion
(Hoggeooom & Salimans 2024). we begin by examining its blurring and noising schedule indexed by [1,,... TT. As defined in (Hoogeboom & Salimans 2024). the blurring schedule, ou is parameterized as:
ae sin?
n
(18)
2T
Given that a cosine noise schedule is applied scaling the signal throughout the diffusion process the noising schedule, 3r, can be parameterized from the perspective of the signal-to-non noisc ratio (SNR) as:
sin
tan .
B1
(19)
COS 2T
2T
Thus the BNR schedule can he written
BNR.
ae
20 s0n'n
2
27
3
Lan
s0
cos
(20)
2T
27
= 10
T
Regarding other related work, IHDM (Rissanen et al. constant amount noise while progressively increasing blur levels. result, the BNR schedule for IHDM represented vertical line in 6 In contrast, [Bansal et al. (2022) develop Cold Diffusion by constructing the diffusion process entirely without noise, leading to a BNR value of co.
2023) introduces diffusion process applying small,
B ADDITIONAL DETAILS OF OUR IMPLEMENTATION
adopt network architectures, training techniques. and hyperparameterss from the state-of-the-art diffusion model, EDM (Karras el al. 20022. making only minor modifications to preserve constant model capacity. In this section, we first provide detailed information on the architecture and training settings, demonstrating that the observed improvement in generation quality is not due to larger nor hyperparameter fine-tuning. Subscquently, we present the algorithms both training and sampling within our proposed diffusion process to facilitate clearer understanding of methodology.
complex network design
B. NETWORK ARCHITECTURES AND HYPERPARAMETERS
Tab. we list the hyperparameters used experiments. For CIFAR-10 and FFHQ, adopt the same settings as EDM (Karras et al. 2022), without tuning for optimal hyperparameters. For LSUN-church, we follow the network architecture and settings from Blurring Diffusion to ensure fair comparison. Across all datasets, we apply slight modifications the network architecture,
depicted in Fig. 7 These modifications include: (1) incorporating additional conditioning signal for the blurring level, using embedding branch fused with the noise-level embedding, and (2) doubling the output channels the neural network and splitting them to compute separate losses, allowing the model to predict both the residual underlying blurry signal. These two changes result in less than 0.15 % increase in model size, as shown in Tah. 6 indicating that the observed improvement in sample quality stems from the proposed method rather than the architecture.
detail and the
13

---

Published as a conference paper at ICLR 2025
Table 6: Hyperparameters and model sizes used in the experiments.
CIFAR-10
FFHQ 64 x 64
LSUN-church 128 x 128
Hyperparameter
Baseline
Ours
Baseline
Ours
Baseline
Ours
Number of GPUs Duration (Mimg)
8
200
200
200
200
512
200
Minibatch size
512
512
256
256
256
256
Learning rate x x10-*
10
2
LR ramp-up up (Mimg)
10
10
10
10
EMA half- life (Mimg)
0.5
0.5
0.5
0.5
0.89
0,5
Dropout probability
0.13
0.13
0.05
0.05
0.1
0.1
Channel multiplier
128
128
128
128
64
Channels per resolution
2-2-2
2-2-2
1-2-2-2
1-2-2-2
1-2-4-6-8
1-2-4-6-8
Residual blocks per resolution
4
Attention resolutions
(16)
{16)
(16)
(16)
{8, 16, 32)
{8, 16, 32)
Attention heads
4-6-8
4-6-8
Model size
55.734
55.741 M
62.761 M
62.765 M
117.955 M
117.957 M
Sum
Denoising Loss
U-Net
Deblurring Loss
Figure 7: Our model architecture includes an additional embedding branch to incorporate condi- tioning signal for the blurring and noising levels. Furthermore, to enable the network to simultane- ously perform both deblurring and denoising, the output channels are doubled and split handle each task separately.
B.2 PROCEDURE OF TRAINING AND SAMPLING
We present the training procedure in Algorithm 1 which follows the training scheme of the state-of-the-art diffusion model EDM (Karras sampling a noise level 3, we further determine corresponding blur level a based on the predefined BNR value. The image signals are then transformed into blurry and noisy signals according to the sampled blur and noise levels. The decoding neural network is then trained to simultaneously denoise and deblur these signals, conditioned the blur and noise levels. Unlike methods that require separate function evaluations for denoising and deblurring, we utilize single forward pass predict both signals once and split the output into two branches, each handling one task.
2022). with slight modifications. To prepare the training data pairs, after
In Algorithm 2 we outline the sampling procedure of our proposed diffusion process, integrated with Heun's 2nd order sampling method, similar to EDM (Karras et al. 2022). Following predefined sampling schedule, for each reverse step, first the denoiser's prediction guide the sample toward blurry prediction. then apply deblurring prediction move sample toward sharper state, as outlined Eq. (6). To integrate Heun's s 2nd order method, we temporarily update the signal æ; to obtain æ. then repeat the process to obtain a refined update direction. The two predictions from consecutive iterations are averaged correct the update step, yielding the final sample 27,11- This process is applied throughout, except for the last step, ensuring more accurate and stable sampling.
14

---

Sample Gaussian noise Sample a noise level Published as a conference paper at ICLR 2025
Algorithm Training Phase
1: Require: Hyperparameters {Pmeany Palds BN
2: Initialize Neural network Fø
3: repeat
4: Zo ((20)
> Sample from training dataset
5:
E~ (O,I)
6:
(n( ~ (N means Sta)
7:
a BNR
Get corresponding blur level from given noise level Apply blurring operation Add noise
8:
aa,0 = VMaV V
9:
Ba,8 = Da.0 + ,
10:
Tres = x0 Fo.0
Compute the residual signal Split the output of the neural network
De, Rs 0 a, ()
12:
Take gradient step on
13:
VoA(3) Dr oll + Ro xra)ll?)
> Jointly learn denoising and deblurring
until converged
Algorithm Generation phase: Deterministic sampling with Heun's ' 2nd order method
1: Require: Neural network Fg, Sampling schedule {(ao, (a1 31), (ow,Bx)) 8
2: sample N (0,
3: for E {N. N 1
} do
4:
Dø. Rg Fo (4, 36)
Split the model prediction
5:
é 2.
> First-order gradient term
8;
6:
ty V(I Ma.) -'
Transform residual signal &res to get æo I:-1 I, V V(Ma. V to + (B1-1 3,)
7:
Eq. (6), taking Euler's step
8:
if # then
Apply second-order correction except for the last step
9:
D/ Rt 1i
é' 2--1-Da D'
31-1 S/ V(I Ma..)) VT
2;-1 x; V(Mo--1 Ma.) VT tuy
+ (Bi-1 (
13: end if
end for
15:
return 20
C ADDITIONAL EXPERIMENTS
C.I SAMPLING SCHEDULES FOR IMAGE GENERATION
This section explains how the noise level schedule affects the process in diffusion model. Before starting the reverse process, a predefined sequence of blur and noise levels needed each reverse step. Since the blur level in our method is controlled by the BNR and linked to the noise level, we only need to focus on the schedule of the noise level.
Following the formula proposed by Karras al. 2022 the sequence of noise levels can be formulated as:
- i (8 Biine ))² 80 = O,
BesiSN = (Brar +
(21)
where controls the emphasis different phases of noise levels. Setting P 1 corresponds to uniform discretization, while higher p places emphasis lower noise levels, resulting in more sampling steps this phase.
Empirically, Karras et al. (0022) found that = 7 worked well across various tasks, and adopted this value in all experiments. However, we discovered that our method benefits from using higher value, as illustrated in Fig. This improvement arises because model learns simpler target in the early stages of the reverse diffusion process, thereby reducing prediction error. By shifting the focus lower noise levels, more sampling
15

---

Published as a conference paper at ICLR 2025
1.90
Baseline (1.912)
Unconditional
2.38
Baselne (2.375)
2.36
1.80
2.32
Beseeline (1.733)
2.30
1.70
(1.676
10
15
20
20
30
30
60
P
(a) CIFAR-10
(b) FFHQ 64 64
Figure 8: Sampling schedule and sample quality. Compared to the baseline noise schedule proposed by EDM (i.e., p = 7), our method benefits from using higher & value. With fixed number of sampling steps, this adjustment enables our sampling process to focus more the later stages of the reverse process, where high-frequency detail are generated. As a result, the sample quality is improved across both CIFAR-10 10 and FFHQ datasets.
Table 7: Quantitative comparisons of FID and scores on unconditional CIFAR-10 for different BNR values, evaluated using the DDPM (Ho et al. 2020) architecture.
Methods
BNR
FID
IS
DDPM Ho et al. 2020)
3.17
9.46
Ours
0.3
3.11
9.48
Ours
0.5
3.03
9.51
steps allocated generate high frequency detail (i.c., the later stages of the reverse process), ultimately enhancing the quality of the generated samples.
C.2 EXPERIMENTAL RESULTS ON DDPM ARCHITECTURE
the main manuscript, we demonstrate the effectiveness of our proposed approach using the stronger baseline, improved DDPM++/NSC+++ introduced by Karras et al. (2022). ) In this section, we extend our experiments to the network architecture proposed in DDPM (Ho et al. 2020). The quantitative comparison on unconditional CIFAR-10 is presented in Tab. 7 Our method achieves lower FID score and higher IS, indicating improved image distribution modeling and enhanced sample quality. The consistent performance improvement across different network architectures further demonstrates the generalizability our proposed method. Additionally, we conduct experiments with a lower BNR value. as reported in Tab. 7 and the trend of the results aligns with those presented in Tab. 3 These findings highlight the enhanced generation quality achieved hy transitioning from Hot Diffusion to proposed Warm Diffusion. This improvement stems from effectively leveraging the spectral dependency of images while addressing out-of-manifole issues avoiding excessive blurring in the diffusion process.
D ADDITIONAL RESULTS
In this section, we provide additional visual comparisons to present more qualitative results. Fig. showcases
generated samples for various BNR values, as discussed in Tab. 3 Fig. TO illustrates samples produced by
re-implementation of Blurring Diffusion (Hoogeboom & Salimans 2024 outlined in Tab. 4 A visual
comparison of different parameterization techniques, discussed Tab. is shown Fig. Furthermore, uncurated (non-cherry-pickee samples from datasets used our experiments are presented Fig.
16

---

Published as a conference paper at ICLR 2025
FID 197 NFB
FID 197 N5
FID 1.5 NFE
FID 257 NFES5
FID 11.97 NFF
Figure 9: Visual comparison of different BNR values with constant number of sampling steps. For cases where BNR 0.5, using the same number of sampling steps, the generated samples tend to become blurrier as the BNR value increases, leading to poorer FID scores. This highlights the trade-off between blurring and noising, where overly prioritizing the blurring process negatively impact sample quality.
N5
RID4 NF
2:1 NPB
2.51 399
FID 2.48 NFR
Figure 10: Visual results of our re-implemented Blurring Diffusion (Hoogeboom & Sallimans 2024 generated samples with different NFEs. As the number of sampling steps increases, the generated samples show progressively more detailed nigh-frequecyy components. Lower NFEs lead to blurrier samples due to insufficient reverse steps as discussed in Sec. 4.3 due to the data manifold shifts.
Setting
Setting (b)
13.19 35
35
Setting
Setting
FID 1.98 NFE
FID L.S5 NFE
Figure 11: Visual comparison of different training objectives and parameterization techniques dis- cussed in Tab. 5 Setting (a) and (b) either suffer from inaccurate predictions or amplified high- frequency signals, which lead to poorer generation quality. Although Setting (c) generates better results, it does not take advantage of the BNR adjustment to ease the neural network's learning pro- cess. In contrast, Setting (d), which employs a divide-and-cooquer strategy, benefits from the BNR adjustment by better leveraging the spectral dependency of images. This leads to improved learning for the neural network and results in a noticcable improvement in sample quality.
17

---

Published as a conference paper at ICLR 2025
Plane
Car
Bird
Dear
Dog
Frog
Horse
Ship
Truck
(a) Unconditional CIFAR-10, FID 1.85
(b) Class-conditional CIFAR-IO, FID 1.68
(c) FFHQ 64 x 64, FID 2.29
(d) L.SUN-church 128 x 128, FID 2.56
Figure 12: Uncurated samples from CIFAR-IO, FFHQ 64 64, and LSUN-church 128 128.
18


Reviews:
[{"review_id": "Epjw8j018G", "reviewer": "Reviewer_iqF4", "summary": "This paper proposes Warm Diffusion, a unified Blur-Noise Mixture Diffusion Model (BNMD), to control blurring and noise jointly.", "questions": "see above", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The paper is well-written", "weaknesses": "1.An improvement of 1-2 points in FID does not result in any noticeable change in visual effects. In fact, the actual visual quality may not be better than those with lower FID scores.\n\n2.I hope the authors can present results that are sufficiently stunning or impactful. There are currently many papers in this area, and everyone is focused on slightly improving FID and IS, but the visual quality is still much worse than the current FLUX. This leaves me with no motivation to decide whether to accept any of these papers.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RpDAPFJzj7", "reviewer": "Reviewer_tjvC", "summary": "The paper proposes a new diffusion framework named \"Warm Diffusion,\" which introduces a unified Blur-Noise Mixture Diffusion Model (BNMD) to bridge the gap between hot diffusion (using noise) and cold diffusion (using blurring) models. The authors argue that both hot and cold diffusion paradigms have inherent limitations: hot diffusion ignores the correlation between image structures, while cold diffusion neglects the importance of noise in modeling the data manifold. Warm Diffusion leverages a Blur-to-Noise Ratio (BNR) to integrate both noise and blurring in a \"divide-and-conquer\" approach, enhancing model learning and image quality. Extensive experiments show that the method outperforms existing diffusion-based generative models on several benchmarks.", "questions": "Q1. Although the authors repeatedly mention “spectral dependency”, the definition of this term is not clear. I can't clearly understand what exactly the “strong correlation” between high-frequency and low-frequency structures is. In Figure 3, why “shifting more responsibility to the deblurring task” means “effectively utilizing the spectral dependency of images”?\n\nQ2. In regard to blur-to-noise-ratio (BNR) defined by this work, it seems that the noise level and the blur level do not change for all time t. I want to further confirm that mentioning the effects of BNR, does the proposed diffusion model actually have a sequence of (a_1,…,a_T) and (b_1,…,b_T) or just one a and b for all timesteps?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The idea of combining Blur and Noise is impressive as it exploits spectral dependencies of images while preserving the data manifold.\n\n2. The proposed BNMD framework is evaluated across several benchmarks, including CIFAR-10, FFHQ, and LSUN-church datasets, sufficiently verifying the effectiveness of the proposed method.", "weaknesses": "1. I think the motivation of the work is not very clearly presented in the introduction section. The exact necessity of introducing blurring into the diffusion model should be given, which is the most important motivation of this work. Figure 5 shows that a lower BNR brings better FID results, which somehow seems to say that “blur” does not help to improve the quality of the generated image.\n\n2. For the sampling process presented in algorithm 2, the sampling starts from a zero-mean Gaussian distribution. How can this be revealed from Eq. 4 (at time T)?\n\n3. As shown in Table 5, there exist large deviations among the results produced by different parameterizations. More explanation need to be provided.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EslbrlWc0A", "reviewer": "Reviewer_GSiJ", "summary": "The authors analyze the characteristics and limitations of hot diffusion (noise-based) and cold diffusion (blur-based) and propose a unified framework that combines both diffusion processes. This approach effectively mitigates the shortcomings of each model, achieving complementary advantages. The authors also conduct extensive experiments and data analyses to demonstrate the superiority of the proposed method.", "questions": "1. Please provide a more detailed explanation of the images in Figure 1.\n2. Provide experimental results using DDPM as a baseline, showing the performance improvements of the proposed method compared to the original model and alternative approaches.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The authors analyze the characteristics and limitations of both hot and cold diffusion, leading to a unified diffusion architecture that combines both approaches.\n2. The authors introduce the new concept of Blur-to-Noise Ratio (BNR), which enables better analysis of diffusion models.\n3. Extensive quantitative and qualitative analyses, including comparisons with state-of-the-art methods and detailed data analysis, are provided.\n4. Experimental results demonstrate the effectiveness of the proposed method in image generation.", "weaknesses": "1. In Figure 1, the authors do not sufficiently explain the meaning of each module. For example, the significance of different-sized circles in the left chart is unclear, as well as the meaning of “Data manifold (indexed by noise level)” and the images depicted. It is suggested that the caption be revised to simplify understanding.\n2. The authors use the improved DDPM++/NCSN++. However, it would be beneficial to experiment with the proposed approach on other baseline architectures, such as the original DDPM [1]. This would help demonstrate the generalizability of the method beyond specific model improvements.\n\n[1] Ho J, Jain A, Abbeel P. Denoising diffusion probabilistic models. In NeurIPS, 2020.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pJgLHfHn7t", "reviewer": "Reviewer_W1SP", "summary": "This paper proposes warm diffusion to bridge hot diffusion, which relies entirely on noise, and cold diffusion, which uses only blurring without noise. This paper claims that noise prediction has the property of good diversity and data manifold, and the blur prediction has the advantage of better spectral dependency and convergence. Then, the blur-to-noise ratio problem in proposed unified Blur-Noise Mixture Diffusion Model (BNMD) is discussed. The performance on the choosen datasets is promissing.", "questions": "My question mainly focuses on the training target and the corresponding optimization diffuculty. Refer to the weakness for details. \nMy final score depends on the response and I am willing to raise the score with convincing responses.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The concept of blur-noise mixture is interesting. The analyses of cold diffusion and hot diffusion are insightful. It is also suggested to verify the claims about these two diffusion paradigms with supporting evidence or visualization, which is missing in the current paper.\n2. The analyses of the blur-to-noise ratio are clearly discussed, which is an important aspect in the proposed mixture paradigm. \n3. The results on the conducted small-scale datasets are promissing.", "weaknesses": "1. This paper claims to decompose the training object into two parts: deblurring and denoising. However, the training target of the deblurring part is the residual. This is quite confusing, since this \"deblurring\" transition also involves denoising and is not even doing the deblurring job. \n2. This two training target operation enforces the network to simultaneously modeling the distribution of the blur and the residual. Besides, the blur also has various levels. These may make the distribution modeling task harder than just predicting the noise/data, especially when the data scale is large.\n3. The experimental results on large-scale datasets are missing, eg ImageNet.  Since the recent sota diffusion models usually conduct experiments on ImageNet and even larger-scale datasets, the results absence on these datasets make the actual performance and the superiority of this method ambiguous.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)