Contrastive Sampling Chains in Diffusion Models
Authors: Junyu Zhang
Daochang Liu
Shichao Zhang
Chang Xu
Venue: NeurIPS 2023 poster (2023)
Keywords: diffusion models
contrastive loss
discretization error
contrastive sampling chain
TL;DR: We demonstrate that an appropriate combination of the contrastive loss and score matching serves as an upper bound of the KL divergence between the true data distribution and the model distribution.

Abstract: The past few years have witnessed great success in the use of diffusion models (DMs) to generate high-fidelity images with the help of stochastic differential equations (SDEs). However, discretization error is an inevitable limitation when utilizing numerical solvers to solve SDEs. To address this limitation, we provide a theoretical analysis demonstrating that an appropriate combination of the contrastive loss and score matching serves as an upper bound of the KL divergence between the true data distribution and the model distribution. To obtain this bound, we utilize a contrastive loss to construct a contrastive sampling chain to fine-tuning the pre-trained DM. In this manner, our method reduces the discretization error and thus yields a smaller gap between the true data distribution and our model distribution. Moreover, the presented method can be applied to fine-tuning various pre-trained DMs, both with or without fast sampling algorithms, contributing to better sample quality or slightly faster sampling speeds. To validate the efficacy of our method, we conduct comprehensive experiments. For example, on CIFAR10, when applied to a pre-trained EDM, our method improves the FID from 2.04 to 1.88 with 35 neural function evaluations (NFEs), and reduces NFEs from 35 to 25 to achieve the same 2.04 FID.

Contrastive Sampling Chains in Diffusion Models
Junyu Zhang Central South University zhangjunyu@ese edu cn
Daochang Liu The University of Sydney daochang liu@sydney. edu au
Shichao Zhang
Chang Xu"
Central South University zhangsc@csu. edu. en
The University of Sydney
xuudsddeyy.... edu. au
Abstract
The past few years have witnessed great success the use of diffusion models (DMs) to generate high-fidelity images with the help of stochastic differential equations (SDEs). However, discretization error is an inevitable limitation when utilizing numerical solvers solve SDEs. To address this limitation, a theoretical analysis demonstrating that an appropriate combination of the con- matching serves
provide
trastive loss and between the true data distribution and the model distribution. To obtain this bound, we utilize a contrastive loss construct a contrastive sampling chain to fine-tuning the pre-trained DM. In this manner, our method reduces the discretization error and thus yields smaller gap between the true data distribution and our model distribution. Moreover, the presented method can be applied to fine-tuning various pre-trained DMs, both with or without fast sampling algorithms, contributing better sample quality or slightly faster sampling speeds. To validate the efficacy of our method, we conduct comprehensive experiments. For example, on CIFAR 10, when applied to pre-trained EDM, our method improves the FID from 2.04 to 1.88 with 35 neural function evaluations (NFEs), and reduces NFEs from 35 25 to achieve the same 2.04 FID. The code is available at Contrastive-Sanma Sampling
upper bound of the KL divergence
1 Introduction
Diffusion models (DMs) 57 22 have emerged as powerful generative models, breaking records in image generation 667
26 32). and finding rapid applications in other domains such as video generation 23 3D point cloud generation [46] text-to-image generation
29
[22] speech synthesis
7. inverse problems 62 30 and lossless compression [36]. While score- score-based generative models
22 are two branches of DMs, a certain parameterization reveals an equivalence between them 45 In seminal work, Song et al. (6 generalized DMs through the lens of stochastic differential equations (SDEs). Specifically, for any given stochastic diffusion process that progressively diffuses a data point into random noise with continuum noise schedule, DM learns remove the added noise with a reverse-time SDE (T]. For SGMs, SDEs utilize deep neural networks to match the gradient of the log probability density with respect to data at each noise scale, which is dubbed denoising score matching [25] 68 59). In this manner, DMs enable not only exact likelihood computation [6]] like variational auto-encoders (VAE) [37 but also higher sample quality than widely-used generative adversarial networks (GANs) B
(SGMs) 57 58 and denoising diffusion probabilistic models (DDPMs) 47
"Corresponding author.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

In practice, directly solving the reverse-time SDE to obtain an image is intractable. Instead, numerical solvers 66 44 are utilized to discretize the problem by constructing a discrete sampling chain with many sampling steps. Concretely, numerical solvers decompose the intractable integration of an SDE into multiple integration intervals [31] and solve them iteratively. However, due to the intractability of the integration in high dimensions, numerical solvers are unable to obtain the exact solution for the integration of each interval. Instead, they provide approximate solutions, which introduce a discretization error. This enror represents the discrepancy between the approximate solution and the exact solution. It is important to note that any discretization scheme used by numerical solvers introduces discretization errors 42 in each integration interval. Consequently, there is a gap between the intermediate data distribution and its corresponding model distribution at each discretization step 775 The accumulation of these discretization errors results in larger gap between the true data distribution and its model distribution. It is worth noting that the estimation error, resulting from score matching or noise prediction during training, can also contribute the gap between these two distributions 75 17.. However, in this paper, we solely concentrate on the discretization error. From an integration point of view, the discretization error gradually decreases the step size of the interval decreases. However, it cannot be completely avoided because the step size cannot be reduced to infinitesimal values.
To alleviate this issue, more discretization steps can be used to make each approximate solution of the discretization step closer to the exact solution, but this significantly increases the computational cost during sampling. For example, DDPM [22] requires 1000 steps to produce an image, with each step requiring the evaluation of the neural network once, which is substantially slower than GANs 228 55]. While recent works have made it possible to achieve high-quality images with significantly reduced sampling steps (29 32 70 they still encounter the issue of discretization errors. On the other hand, some fast sampling DMs [27 65| 72 53] speed up sampling process for SDEs or ordinary differential equations (ODEs) 61]. but suffer from sharp decline in image quality since the discretization error grows larger. Hence, discretization errors have an severe impact on DMs.
More recently, several studies have been conducted to enhance numerical solvers which aim to reduce the discretization error. DPM-Solver [44] utilizes the semi-linear structure to avoid the corresponding discretization emror by analytically computing the linear part of the solutions. By comparison. Liu et al. 42 combine high-order methods with Denoising Diffusion Implicit Model (DDIM) [56] to solve the ODE and achieve further acceleration. However, the accuracy of this approximation is not theoretically justified and may suffer from significant discretization error if the step size is large (75). To fill the theoretical gap, Zhang and Chen 75] propose to utilize an exponential integrator to remedy the discretization error, achieving much better sample quality with smaller step numbers compared to previous fast sampling approaches. Orthogonal to this direction, our focus lies in minimizing the discretization error by optimizing the upper bound of the Kullback-Leibler (KL) divergence between the true sampling chain and a simulated chain at each time step.
The objective of our work is to establish a contrastive sampling chain to fine-tune any pre-trained DMs so to reducing the discretization error, contributing to small gap between data distribution and model distribution. Our method is motivated by the observation that minimizing the KL divergence between the true sampling chain and simulated chain at each corresponding time step effectively reduces the discretization error. This reduction in discretization error directly translates into an improvement in sample quality. To achieve this, we propose minimizing a contrastive loss [ 21 that effectively reduces the gap between intermediate data distribution and the model data distribution each time step. Specifically, by selecting instances the sampling chain of pre-trained DM for same image as positive pairs, and choosing negative instances from other training images, a contrastive loss function is formed. To keep the pre-trained model stable in its generative ability, in practice, we combine the contrastive loss with the original generative loss using dynamic weighting schedules during the fine-tuning process. Moreover, the contrastive loss is optimized with backpropagation through time (BPTT) to spread the gradients on the whole sampling chain. In this manner, our method reduces the gap between true data distribution and model distribution, which improves sample quality without increasing sampling time overhead. Comprehensive experiments validate that our method can improve generation quality for various pre-trained models when using the same neural function evaluations (NFEs), or require less NFEs to achieve the same generation quality.
In nutshell, our work makes the following contributions: 1) We demonstrate that the discretization error results in a gap between each intermediate data distribution and its corresponding model data distribution. 2) We analyze that an appropriate combination of contrastive loss and score
2

---

matching serves as an upper bound for the KL divergence between the data distribution and the model distribution. 3) We propose a contrastive sampling chain fine-tune a pre-trained DM with the assistance of our derived upper bound. 4) We present dynamic weighting schedules and BPTT as optimization techiniques for our method.
Background
As mentioned previously, SGMs and DDPMs have been considered almost equivalent when certain parameterization is applicd. For the sake of simplicity, we solely focus on utilizing denoising score matching under SDEs for further investigation in this paper. Below, we present a comprehensive review of the entire DMs process with the lens of SGMs.
Forward noising diffusion: The forward diffusion of a DM for D-dimensional data is a linear diffusion described by the stochastic differential equation (SDE) [54]
dæ F (odt + Gidw,
(I)
where F, E IR DxD denotes the linear drift coefficient, G. E DRD denotes the diffusion coefficient,
and is a standard Wiener process. The diffusion Eq. ① is initiated at the training data and simulated over fixed time schedule [0, T). Denote by (xx) ) the marginal distribution of Iy and by Pur(2g xо) the conditional distribution from To to Te, then (27o) represents the underlying distribution of the training data. The simulated stochastic process is represented by {I:DE) J ",27'
where pr(z)) is
prior π (xT) ) which is is easy to sample from, like Gaussian distribution. The parameters F, and G2 chosen such that the conditional marginal distribution por (xr an)) is simple Gaussian transition kernel, denoted as N (Pt30. ).). Three popular SDEs in DMs are summarized by Song et al. [61], which are variance preserving SDE (VP SDE), variance exploding SDE (VE SDE) and sub-variance preserving (subVP SDE). Ideally, we enable to diffuse any data distribution to a prior distribution (IT) with one of those three SDEs.
Backward denoising diffusion: Under mild assumptions 61 the forward diffusion Eq. is associated with reverse-time diffusion process
dx
F G,G. v log dt + G.dw,
(2)
where w denotes a standard Wiener process in the reverse-time direction, V log denotes the gradient of the log probability density with respect to data each time step general, with known prior distribution iT (ZT), one can model the data distribution Po (2o) with Eq. (2) as F, and G. are fixed according the forward SDEs. However, to solve Eq. (2). one needs match the score function V log Pt (z),, which is not accessible.
Training: The basic idea of DMs is
time-dependent score matching network s0 to
approximate the score V log Pr (r)). This is achieved by score matching techniques [25] 68, 59 where the score network sg is trained by minimizing the denoising score matching loss
JsM (0; (()) Ee Bp(xa) poe (w)))) (())) || log Por
so(a2,
1
(3)
(r2e(2r)) has closed form expression (ar (zo) is a simple Gaussian distribution which represents the discretized form of given SDE, w(l) denotes a time-dependent weighting function. This loss can be evaluated using empirical samples by Monte Carlo methods and thus standard stochastic optimization algorithms can be used for training.
Here V log Pox
Sampling: Once the score network s0 t) R V Pz()) is matched for almost all x E RDD and t ~ UO, 1'). it can be used to generate new samples by solving the backward SDE Eq. with log pr(2)) (æ: replaced by so (2 t). It turns out there are infinitely many diffusion processes one can use. In this work, to show the scalability of our method, we consider a general expression of SDEs
+
dx SDSDE SDE
F
SDE
G.G.T so SDE b) dt + AGGdw,
parameterized by >> O. When A = 0. Eq. reduces to an ordinary differential equation (ODE) known as the probability flow ODE [ The reverse-time diffusion Eq. (2) with an approximated score is special case of Eq. 4 with X 1.
3

---

Discretization Error Analysis
To generate image, can sample de from a standard distribution (<x)) and solve Eq. (4) to obtain an image x However, in practice, exact solutions are not attainable as it intractable to solve Eq. (4) directly. To remedy this, one needs to discretize Eq. (4) over time to get an approximated solution, which leads to a discretization error. For brevity, we next investigate the discretization error of solving the probability flow ODE (A = 0)
dx
Fix G.G. so (
SDE .(),
(5)
di
and 27 .SD represents the discretization samples solved by Eq. with numerical solver. The exact solution to this ODE is
SDE + [ ((, r) SDE (L T)27
1 G so (DDE .T) dr,
(6)
where (t, 7) satisfying @v((.T) = T),, (t, 7 known as the transition matrix from H(t)
time T to f associated with Fr. There exist many numerical solvers for Eq. ( associated with
different discretization schemes to approximate Eq. (6). As the discretization step size goes to infinitesimal, the solutions obtained from all these methods converge to that of Eq. (5).
However, the performances of these methods can be dramatically different when the step size is large. On the other hand, to achieve fast sampling in DMs, one needs to approximately solve Eq. (5) with a small number of discretization steps, and thus large step size. Concretely, the discretization of Eq. (4) equals to build discretization sampling chain, which is { SDE
tEJO,T) and iteratively convert
to a new image = with randomly initialize a sample from Zy, where is the total sampling steps. Following the philosophy of discretization sampling, small number of discretization steps is equivalent to small sampling steps T and a large step size A/ from TT to I₁. When Euler method applied to Eq. (5), the discretization form can expressed
SDE 1 Gr SDE T) At. FTO
SDE Jt
SDE
From the integral point of view, samples z SDE obtained from Eq. equals the exact solutions of Eq. (5) if and only if the step size between 27 ZSDE and 2,DE goes to infinitesimal. In practice, it's impossible to achieve it, especially the fast sampling demand in DMs that requires small T even one-step sampling [63], which means zo error is essentially gap between the approximate solution and the exact solution
SDD is approximate solution. Hence, the discretization
1 2
/
A (r
SDE SDE
v (t,
G, G. s0 (I,DEE :T) dr
(8)
SDE .T) At 2
2 F FT
SDE Gy Go SDE
In light of this, the presence of A SDE) creates a gap between (((xr) and p (EDE since each sample DiDD only approximates the exact solution xt. Consequently, one enables to optimize the KL divergence between Po(2;) and p. p By minimizing this KL term, one can effectively reduce the discretization error. This is because each Pi S (2 from the simulated chain will closely approximate Pz(z;) from the true sampling chain.
minimize this gap, denoted as DKL (pel pi PD)).
4 Theoretical Analysis
It is well-known that maximizing the log-likelihood of probabilistic model is equivalent mini- mizing the KI. divergence from the data distribution to the model distribution (60 43 Similarly, in
order improve the log-likelihood of DMs, we can optimize the KI. divergence DKL. (polled $DB) between the true distribution (o())) and its corresponding model distribution p
SSDF (2:SEE))
In what follows, demonstrate that with appropriate combination of DKL. (px ( and score matching losses JSM (t))()))), they actually becomes an upper bound on DKL (po lId SDE) [60] Notably, the weighting function (1) in Eq. B is replaced by g()) which the diffusion coefficient of SDE in Eq. II.

---

Theorem 1.
Let (o)z be the true data distribution. π(xT) be a known prior distribution. Suppose
(at is a stochastic process defined by the SDE Eq. 17) with zg ~ >Po(zo), where the
marginal distribution of Iı is denoted as Pr PZ))) By comparison, { {
J is stochastic process obtained by the reverse-SDE in Eq. (4) from a pre-trained DM with ITPE -pr(xT) and
DE
zyDE ~ ~po p (x where the marginal distribution of E.DE is denoted as pi
SIDE ( Under
some regularity conditions detailed in Appendix we have
DRL. (pollPo
) S DKL (Pr (DE)
+ JSM (0:
Sketch of proof. Let f and v denote the path measures 40) 38 of SDEs in Eq. 1 and Eq. 4 with = 1. Conceptually, a is the joint distribution of the forward diffusion process (0t ( given by Eq. and represents another joint distribution of the process 20) 2 Po pS (2.DE) simulated by Eq. Since we can marginalize fi and v to obtain distributions po (2o) and pa (~) (DSE)) the data processing inequality [60] gives DxL. (pu ()) S DRI(H,llpg where μe, and D, are subset of μ and v respectively. Intuitively, DKL (Pollpå ) is a subset of the path measure between μt and V which accumulate all the KL divergence DRL (Pallz? (plll [Eju,2] (
40]. Hence, DRL (po (polleEDE) <
DKL (Mg -: DRL (pillPEEE) ,t [0, From the chain rule for the KL divergence [40] also have (1I, [1,) DRL. (pellp,DE) Ep:(()) [Drt. (n((1 a.; 2)|l (. 5I =))] with i E 10, t). where the KL divergence in the final term can be computed by applying the Girsanov
theorem 48 to Eq. 4 and the reverse-time SDE of Eq. .T. Combining above analysis completes
the proof, detailed see in Appendix B
Though we demonstrate that the combination of score matching losses JsM (0: g+)) and KL term DKL (prllp,DE) is an upper bound of DKL (pollpğ ( it is intractable to optimize Eq. (9) due to the unknown function form of Pt- To circumvent this problem. we combined with the mutual information (MI) theory (50) that MI between (xre and p. ( can be expressed (pt SDE || pt)- Moreover, when applied the Jensen's inequal- ity to this term, we enable to obtain an upper bound of Dx (Pr. || p by a InfoNCE loss: InffNNCE( (xŞDE ,43 D) 2 (PI(Pe) ll pr ) [49), where j E [0, T] and j + (detailed in Appendix B). Hence, we obtain a new upper bound of the KL divergence of DKL (pollp (DD) when applics this term to Eq. (9). which
as (x:SD),
(10) where ((=,DE .2) .:SDE; ; = 3(1) * /INONNCE (2. SDE ,.DD)) is a scaling of InNONCE with the weight- ing function 3(t). Hence, it is obvious that the the combination of score matching losses JsM and a contrastive loss INfONCE (= .x) ()DE) in Eq. (1o is an upper bound of the KL divergence between
DKL (polleo DD S T(2
SDE SDE, :(()) JSM (0:g(t)²), STD zt
data distribution (=o) and the model distribution ph DDD (E
Importantly, the score matching term JSM (t:)()))) is stable and almost fixed in the Eq. 10). as fine-tune the pre- trairri DMs that the score network so(x2) Vlog Pe (x) is matched
for almost all x E I&D and t ~ U0, T]. In this context, optimizing the Eq. 10) equals to
minimize I( , S,,,E; B())), which is the upper bound of DKI. (Pe (pel) minimizing DeL (pe (pellesD) results in smaller gap between x, and 2 indicating that the approximate solution obtained through numerical solvers closer to the exact solution with the help of contrastive loss. On the other hand, each minimized DKL (Px IIp. will decreased the corresponding A SDD,), contributing to a smaller discretization error between step T and any step t. From this perspective, utilizing contrastive loss to increase the similarity between z SDE and each x SDE in a sampling chain is actually minimizing DKL (px llp (DEE)) )eeo,,y) Hence, by minimizing the Eq. (10), one achieve better model distribution po DE with reduction in all the discretization error present in the sampling chain. Moreover, the decreased discretization error also benefits the faster sampling speed 44
(50]. Therefore,
63
5 Methodology
Contrastive learning has recently achieved remarkable performance 4 12 21 and has made signif- icant waves in deep learning for computer vision tasks [35] 51). These influential works leverage
5

---

the contrastive loss to bring similar images closer together in high- gigh--imensional space, resulting in notable improvements in downstream tasks. Building on the insights provided in Section 4 we employ the InfoNCE loss [4 to formulate our objective function. By doing so, we effectively reduce the discretization error between xrDE and ...DP by optimizing the upper bound of PKL (pellp, Consequently, numerical solvers are capable of providing more precise solutions when solving Eq. (5). and a fine-tuned DM naturally enhances the quality of generated samples. In the following sections, we will present our method, focusing on constructing contrastive sampling chains and optimizing DMs.
5.1 Contrastive Sampling Chain
As previously mentioned, our objective is to enhance the quality of samples by reducing the dis- cretization error. Additionally, illustrate that the presence of this discretization error creates a gap between 2² and zt- Analogously, optimizing this gap is equivalent to minimizing the corresponding discretization error. While directly decreasing this gap is infeasible, minimizing its upper bound provides an alternative approach to achieve the same outcome. Leveraging Theorem T Jand Eq. (T).. we propose a contrastive sampling chain for fine-tuning pre-trained DMs using the contrastive Toss. To accomplish this, we construct the contrastive loss and combine it with the score matching loss Jsm (#: (0)) to jointly update the parameter A.
To construct the contrastive loss, we randomly select an image 2gDE and another image z; from the defined sampling chain to form a positive pair. Meanwhile, negative instances are sampled from the training images. Next, we extract 128-dimensional latent representations of these images using the pre-trained MoCov2 [II]. The contrastive loss, known as the InfoNCE loss [49), is then computed in the subsequent steps. When applying the pre-trained MoCov2 E, the InfoNCE loss can be expressed as follows:
exp ((DDDD)
((x)DE)/T)
InnfoNCr: (1. SDR SDDE log
(11)
exp (DE E(aSDB)/7) En exp ((( ((T))
where 2 25DE and 2 form a positive pair which all generated via iteratively calculating Eq.
during the fine-tuing process. By comparsion, x are negative instances which sampled from training images and r a temperature hyper-parameter. Conceptually, the InfoNCE loss crafted to bring similar features closer, thereby reducing the distance between 2r and 2 2iSDE, In this manner, the sampling chain will become tighter and the discretization error decreased accordingly because the integration interval between sampling steps decreased.
Though we show that (1 ŞDDD 3 SDE))) is an upper bound of DK (Pr |l (DE) in Section 4
directly using Eq. (IT) to fine-tune DMs will destroy the optimal result of the previous score matching in practice. To circumvent this problem, we combine the generative loss Eq. (3) and contrastive loss Eq. ( to optimize pre-trained DMs
(12) where B(t) is the weighting schedule to balance score matching term and contrastive loss term, which will be analyzed in detail in the following subsection. In Eq. (12. apply the contrastive loss via calculating x² SDE and all 2TSE in the same sampling chain with Eq. 11.. Though we amplify the JItONCE with (()), the combination of score matching term JsM (0))(t)) is still an upper bound of DKt. (pollp5PE).
La = JsM (0:g(t))) + 3(t) /IMfNCC
SDE SDE
SDE
5.2 Optimization
Considering the previous discussion, it is crucial strike balance between the InfoNCE loss and the generative loss. Placing excessive emphasis on the InfoNCE loss may disrupt the stability of the pre-trained generative task, while assigning a high weight to the generative loss may have negligible impact. To address this, we draw inspiration from the fact that the InfoNCE loss quantifies the similarity between x ,EE and 2-55 SDE, where the similarity increases as x SDE approaches 2; 2DE Consequently, it is reasonable to implement dynamic weighting schedule based on the time step t. This schedule assigns higher weights to the loss when SDE closer to x SDE and lower weights otherwise. To strike appropriate balance between the two losses during the refinement of pre- trained DMs, we have devised two dynamic weighting schedules: the linear and nonlinear weighting schedules.

---

Sample one step (B)
Shared U-net (0)
Linear Combin.
co(
contrastive loss
Sample (0c)
x
Sample (0c-1)
Sample (0+)
X
gradient flow
generative loss
[-1
Figure 1: Optimizing Diffusion Models via Back Propogation Through Time.
Linear weighting schedule: The weights in this schedule increase progressively with the discretiza- tion step as gets closer to 2SDE For instance, larger weights are assigned to 2,DE shares higher similarity with =yDEE In contrast, smaller weights are assigned to the earlier step of the sampling chain. this extreme, a linear weighting schedule can be expressed as
3(t) (T -)) t),
(13)
where a is hyper- syper-parameter to scale the weighting schedule.
Nonlinear weighting schedule: The nonlinear weighting schedule works in similar fashion to the linear weighting schedule, with the only difference being that the weights increase different rates. Specifically, calculate the noise ratio in compared to z; 22De the peak signal-to hignal-to-noise noise ratio (PSNR) [76] [4] and subsequently apply the PSNR as the elements this weighting schedule. Therefore, the nonlinear weighting schedule can be expressed as:
B(I) PS SDE SDD)).
(14)
In general, the nonlinear weighting schedule more consistent with the principle of the sampling chain in DMs, the weights gradually remove the noise in the samples. However, in practice, both weighting schedules demonstrate similar effectiveness.
Computational graph: As mentioned earlier, we construct contrastive sampling chain to fine-tune a pre-trained DM. Though we utilize Eq. (6) directly obtain sample x SDDE, in practice, x; SDE is iterately solved by Eq. (4). Hence, we need to consider the accumulated discretization error such as A (x SDE SDF) and A (x x7-11) leads to a larger gap between 2 2SDE and = To 177
address this issue and influence the parameters of previous steps while optimizing the KL divergence DKL (p:lpp SDE) for the current step t. we propose to fine-tune pre-trained DMs using a gradient
propagation mechanism similar to BPTT 73). shown in Figure
Specifically, for given sampling chain, randomly select one sample x; ..EE and another sample =
SDE and calculate the InfoNCE loss with the previously mentioned weighting function. By
combining with the generative loss (Eq. (3), we employ an optimizer to propagate the gradients of these two losses in reverse order through the chain from the current step to the final step T. In this manner, the gradients influence the entire sampling chain and update the parameters of DMs each time step t. Consequently, the training objective can be reformulated as follows:
La JsM (I)) + I(x .r) SDE, (3())); UOO,T],
(15)
where 1'<2,DE .x. SDE, B()) represents the InfoNCE loss Eq. 11 with weighting function 3(t). By following this design philosophy, the InfoNCE loss does not solely update parameters independently; unified entity. Consequently, we enable to optimize the current KL divergence DKL (pr ()))))) and simultaneously influence KL divergences in previous time steps. This approach ensures cohesive and synchronized optimization process throughout entire contrastive sampling chain. In summary, our method effectively mitigates the cumulative discretization errors, as each discretization error correspondingly reduced. Below, we demonstrate experiment results further prove analysis.
rather, it operates

---

Table I: Performance on CIEAR 10. Our method, denoted as Table 2:
Performance on fast samplers on CIFAR-10 10 and ImageNet 64x64
C++, has better quality than baselines with the same NFEs, and Jewer NFEs for the same quality.
with FID reparted. DDIM [56]. DPM Solver [1] and DEIS [75 are classical
triining-fee free fast samplers. "4" the actual NFF is smaller than the NFF
Method Unconditional
Space
NFEL
SLL
FIBI
44 given in the table. represents that the FID for this NFE is not shown in
original papers.
DDPM
VDM N0
22
Data
Data
1000
1000
2.49
3.75
3.17
7.41
Mothod
NFE
10
12
15
20
50
1DDPM
Data
1000
3.37
2.90
DDPM (CIFAR 10) 22]
DDIM
13.36
6.81
1.67
STDDPM
Data
2000
2.91
2.47
DDIM-C-+ (Ouns)
11.52
6.09
4.12
INDM 33
Latent
2000
3.09
2.28
3PPMSSaeer-3 [44]
124.77
8.50
13,33
"3.55
CLD-SGM 20
Data
312
3,31
2.25
DPM-Solwer-3-CC 3-C++ (Ours)
21.13
7.14
5.22
15.06
14.1
NCSN++ 61
Data
2000
3,45
2.20
VP (CTFAR-10)
LSGM 67
Litent
138
3.43
2.10
3elver-
51,56
6.03
3.55
1,.0
1265
LSGM-C++ (Ours)
Latent
100
3.40
2.10
DPM-Solver-3-Crr (Our))
T43.81
5.1
3.25
12.84
12.60
LSGM-C++ (Ours)
Lateat
138
3.40
1,99
[8]
3.37
2.57
EDM 29
Data
35
2.60
2.04
DEIS-ABB1CC++ (Ouns
4.02
3.20
2.77
2.48
EDM-C++ (Ours)
Data
25
2.55
2.04
IDDPM (EnugeNet) 447
EDM-C++ (Ours)
Data
35
2.55
1.88
Selver-
157.48
24.62
19.76
18.95
117.52
Conditional
3PM (Oas))
15063
22.65
19.45
118.73
117.49
NCSN++-G G
Data
2000
2.25
IDDPM (ImageNer Canditional)
DEIS-IRR3
6,65
3.99
13.67
3.10
2.69
EDM
Data
35
2.60
1.82
DEIS-ABB-C++ (Ouns)
13.60
3.07
2.67
EDM-C-+ (Ours)
Data
27
2.55
L882
DPM-Soher-2
44
7.93
5.36
14,46
3.42
2,82
RDM-C++ (Ours)
Data
35
2.55
1.73
DPM (Our))
7.78
5.22
14.38
3.36
2,80
6
Experiments
In this section, we demonstrate effectiveness of the contrastive sampling chain by experimental results, namely higher image quality, better log-likel,hood, or slightly faster sampling speed. Comprehensive experiments are conducted on various datasets, including CIFAR-10. CelebA/FFHQ 64x64, and ImageNet 64x64. We first utilize contrastive sampling chain to improve pre-trained DMs, for which we select EDM [29] and LSGM [67] on CIFAR- 10, EDM [29] on FFHQ, and STDDPM (34] on CelebA. Additionally, we further verify the performance of our method with sampling chains defined by training-free free fast sampling methods, such as DDIM [56), DPM-Solver [44], and DEIS 775., when they are combined with various pre-trained DMs. Specifically, we combine these fast sampling methods with IDDPM 477 on ImageNet 64x64, DDPM 22 and SDE VP [6]] CIFAR-10 10. Our objective is not to focus on achieving state-of-the-arr metrics in generative models, but rather to demonstrate the significant performance of our method in enhancing pre-trained DMs, either combined with or without training-free fast-sampling methods.
It is worth noting that we maintain all the training settings of the pre-trained DMs and only modify the part that constructs the contrastive loss, thereby ensuring fair comparison and demonstrating the flexibility of our method. Similarly, to showcase the applicability of our method to sampling chains defined by fast sampling methods, we replace the original chains provided by pre- pre-trained DMs with new chains for fast sampling, while retaining all other settings the same fine-tuning pre-trained DMs. Once the fine-tuning process is completed, we test the performance of the refined model by drawing 50,000 samples from it and measuring the widely adopted Fréchet Inception Distance (FID) score, Negative Log-L.kellihood values indicate better performance. Moreover, we also present the generated images for qualitative comparison, shown in Figure 2
(NLL), and Neural Function Evaluations (NFEs), where lower
6.1 Performance on Pre-trained DMs
We first showcase the performance of our method in refining the original pre-trained DMs. Tables T and 5 present the results of our method on CIFAR-10, FFHQ and CelebA respectively. Our method can achieve better generation quality than basclines when using the same NFEs. On the other hand, our method requires less NFEs to achieve the same quality as basclines. To evaluate on CIFAR-IO, we apply our method LSGM [67] and EDM (2 under unconditional or conditional settings. For fair
comparison, we report performances on EDM (29 under the random seed from 32 63 Specifically. our method improves LSGM from 2.10 FID 1.99 FID and achieves slightly better NLL with 138 NFEs, while requiring only 100 NFEs to attain the same 2.10 FID. Moreover, we enhance EDM from 2.04 FID to 1.88 FID and reduce the NLL from 2.60 to 2.55 with 35 NFEs, while requiring only 25 NFEs for the same 2.04 FID. Furthermore, we improve the performance on conditional EDM, with 1.82 FID reduced to 1.73 FID using 35 NFEs or maintaining the same 1.82 FID with only 27 NFEs.
For the evaluation CelebA and FFHQ, we apply our method to refine STDDPM 34 and EDM
8

---

Figure 2: Randomly selected ImageNet Samples from IDDPM Improved by Our Method.
29 respectively. Specifically, our method improves the performance of STDDPM on CelebA 64x64 from 1.90 FID to 1.73 FID with the same 131 NFEs, and achieves the same 1.90 FID with fewer NFEs (100). Similarly, we significantly enhance the performance of EDM on FFHQ 64x64 from 2.39 FID 2.07 FID with the same 79 NFEs, and achieve the same 2.39 FID with fewer NFEs (63).
6.2 Performance on Fast Samplers
To demonstrate the applicability of our approach to sampling chains defined by training-free fast sampling methods, carry out sequence of experiments CIFAR-10 and ImageNet 64x64, with detailed results in Table 2 We test our method on different fast samplers when combined with
various pre-trained DMs. We utilize DDIM (56), DPM-Solver
44 and DEIS [75] to replace the
original chains given by DDPM (22). SDE VP 61 and IDDPM 47). Subsequently, for a pre-trained
model with sampling chain defined by these fast samplers,
refine it with Eq. (15). In this manner,
our method remarkably improves the results as illustrated in Table 2 For example, our method obtains better sample quality when applied to refine the SDE VP with DEIS. Concretely, our method improve the FID from 4.17 to 4.02 10 NFES and increase the 2.57 FID to 2.48 FID in 50 NFEs. In comparison, the improvement on ImageNet is not significant the effectiveness of our method rather than achieving state-of-the-an performance metrics. For instance, when we apply our method IDDPM with a conditional setting and replace the original chain with DEIS, we observe improvements in FID. With 10 NFEs, the FID decreases from 6,65 6.59, and with 14 NFEs, the FID decreases from 3.67 3.60. is worth noting that the symbol indicates that the actual NFE is smaller than the NFE reported in the table provided by Lu et al. In summary, our method seamlessly integrates with training free fast sampling algorithms and enables us to achieve better overall performance.
main objective is to demonstrate
44
6.3 Ablation Study
We conduct ablation studies in Table to assess the impact of different techniques. Table compares four different settings, i.e., the proposed contrastive loss with a single pair (I,, a variant of contrastive loss with all samples E (It, 2;) in the chain, the proposed contrastive loss with BPTT, and naive fine-tuning without contrastive loss for the same number of epochs the previous

---

Table 3: FID Comparison of Ahlation Study with Different Weighting Schedules on CIFAR-10. To prove the performance of BPTT and contrastive guided technique respectively. with three different settings. 'FDM Baseline" (2] is the pre-trained model. 'Naive Fine-Tuning" mccans just
Table 4 FTD componison on FFFQ 64x64.
Method
NFE1
FID.
conduct ablation study
EDM 29
79
2.39
epochs other setting. For our techniques: "Contrastive Loss" means line-tune the model with only contrastive loss no BPTT. "Contrastive Loss Stepe)" is calculating the contrastive loss with all the discretization stop contrastive sampling chain and no BPTT, 'Contrastive Loss BPTT" our setting with hest performance.
EDM-CH (Ours)
63
2.39
EDM-C++ (Ours)
79
2.07
Tabls FID comparison on CelebA 64x64.
Method
NFE,
FID
+DPM+ [6]]
131
2.32
Weighting Schedule
Soft Diffusion 16
300
1.85
Method
Fixed
Linear
Nonlinear
INDM t33
132
1.75
EDM Baseline (29
2.04
Diffusion StyleGAN2
1
1.69
Naive Fine-Tuming
2.04
STDDPM K4
131
1.30
Contrastive Loss
2,04
1.98
1.99
Contrastive Loss (All Steps)
2.18
2.32
2.39
STDDPM-C++ (Ours)
100
1.90
Contrastive Loss BPTT
2.00
1.88
188
STDDPM-C++ (Ours)
131
1.73
settings. We also test three weighting schedules, including two previously mentioned dynamic weighting schedules and no schedule with a fixed weight during the entire process. As we mentioned in Section [.2] calculating the contrastive loss with all samples in the chain disregards the consistency of the entire chain, which can destabilize pre-trained DMs on generative tasks. Therefore, it is reasonable that obtained worse FID when fine-tuning DMs with this setting. Moreover, the two dynamic weighting schedules show equivalent performance, both of which are much better than the fixed weight. Our results demonstrate that contrastive method trained with BPTT for updating parameters can yield the best performance.
7 Conclusions
In this paper, we demonstrate the effectiveness of optimizing the KL divergence between the true sampling chain and the simulated chain each time step in reducing the discretization error associated with numerical solvers used for solving SDEs. Our theoretical analysis supports the use of our objective function as an upper bound of the KL divergence between the data distribution and the model distribution. Notably, optimizing our objective function is equivalent to minimizing the KI. divergence between the true sampling chain and the simulated chain at each time step. To address this, we propose contrastive sampling chain that leverages the derived upper bound to reduce the discretization error. Additionally, we introduce the use of backpropagation through time (BPTT) to propagate gradients in the reverse direction of the sampling chain, and we design dynamic weighting schedules to enhance the stability of the refinement process. Our empirical results demonstrate that our approach significantly improves both the sample quality and the oogllikelhood, while slightly accelerating pre-traine trained DMs without compromising image quality.
Limitations and broader impact: Although our method has shown significant improvements, there remains potential for further optimization of our method. For instance, implementing the BPTT computational graph demands significant amount of GPU memory. Additionally, obtaining analytical solution of our weighting function will undoubtedly tighten the upper bound of the KI. divergence between the data distribution and the model distribution. On the other hand, the issue of inefficient sampling remains a major obstacle to the practical application DMs. It is reasonable to expect that the sampling speed can be greatly improved by method that effectively optimizes the discretization error. However, it is important to acknowledge that the generation of deepfake images using method also carries the potential risk of negative misuse of this technology.
Acknowledgments
This work was supported by the Natural Science Foundation of China under grant 61836016. Chang Xu was supported in part by the Australian Research Council under Projects DP210101859 and FT230100549. The Al training platform supporting this work were provided by High-Flyer AI (Hangzhou High-Flyer AI Fundamental Research Co., Ltd.). This work was also supported in part by the High Performance Computing Center of Central South University. This work was supported in part by the China Scholarship Council.
10

---

References
[1] B. D. Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications, 12 (3):313-326, 1982.
F. Bao, C. Li, J. Zhu, and B. Zhang. Analytic-DPM: analytic estimate of the optimal reverse variance in diffusion probabilistic models. International Conference on Learning Representations, 2022.
[3] A. Brock, Donahue, and K. Simonyan. Large scale GAN training for high fidelity natural image synthesis. In International Conference on Learning Representations, 2018.
[4] Caron, H. Touvron, Misra, Jégou, J. Mairal, Bojanowski, and A. Joulin. Emerging properties self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9650-9660, 2021.
[5] C.-H. Chao, W.F. Sun, B.-W. Cheng, Y.-.. Lo, C.-C. Chang, Y.L.. Liu, Y.-L. Chang, C.-P. Chen, and C.-Y. Lee. Denoising likelihood score matching for conditional score-based data generation. In International Conference Learning Representations, 2022.
[6] N. Chen, Y. Zhang, H. 7en, R. J. Weiss, M. Norouzi, and W. Chan. Wavegrad: Estimating gradients for waveform generation. In International Conference Learning Representations, 2021.
[7] N. Chen, Y. Zhang. H. Zen, R. J. Weiss, M. Norouzi, N. Dehak, and W. Chan. Wavegrad 2: Iterative refinement for text-to-speech synthesis. arXiv preprint arXiv:2106.0966/ 2021.
[8] R. Т. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.
[9] T. Chen, S. Kornblith, M. Norouzi, and G. Hinton. simple framework for contrastive learning of visual representations. International conference on machine learning, pages 1597-1607. PMIR, 2020.
[10] T. Chen, G.-H. Liu, and E. A. Theodorou. Likelihood training of Schrödinger bridge using forward- backward sdes theory, arXiv preprint arXiv:21 2021.
[11] X. Chen, H. Fan, Girshick, and K. He. Improved baselines with momentum contrastive leamnig. arXiv preprint XXxv:2033.04997, 2020.
[12] X. Chen, S. Xie, and K. He. An empirical study of training self-supervised vision transformers. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pages 9640-9649, 2021.
[13] Y. Chen, T. T. Georgiou, and M. Pavon. Stochastic control liaisons: Richard sinkhorn meets gaspard monge on a Schrödinger bridge. Siam Review, 63(2):249-313, 2021.
[14] J. Choi, J. Lee, C. Shin, S. Kim, H. Kim, and S. Yoon. Perception prioritized training of diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11472- 2022.
[15] F-A. Croitoru, V. Hondru, R. T. Ionescu, and M. Shah. Diffusion models in vision: A survey. IEEE Transactions Pattern Analysis and Machine Intelligence, 2023.
[16] G. Daras, M. Delbracio, H. Talchi, A. G. Dimakis, and P. Milanfar. Soft diffusion: Score matching for general corruptions. arXiv preprint arXiv:2209.0544 2022.
[17] G. Daras, Y. Dagan, A. G. Dimakis, and C. Daskalakis. Consistent diffusion models: Mitigating sampling drift by learning be consistent. arXiv preprint arXiv:2302.09057 2023.
[18] V. De Bortoli, J. Thornton, J. Heng, and A. Doucet. Diffusson Schrödinger bridge with applications score-based generative modeling, Advances Neural Information Processing Systems. 111176695-77099 2021.
[19] P. Dhariwal and A. Nichol. Diffusion models beat GANs on image synthesis. Advances in Neural Information Processing Systems, 34:8780-8794, 2021.
[20] T. Dockhom, A. Vahdat, and K. Kreis. Score-based generative modeling with critically-damped langevin diffusion. In International Conference on Learning Representations, 2022.
[21] K. He, X. Chen, S. Xie, Y. Li. P. Dollár, and R. Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16000-16009 2022.

---

[22] J. Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6851, 2020.
[23] J. Ho. T. Salimans, A. Gritsenko, W. Chan, M. Norouzi, and D. J. Fleet. Video diffusion models. arXiv preprint 2:X4::2004.044 2022.
[24] G. Huguet, D. S. Magruder, A. Tong, O. Fasina, M. Kuchroo, G. Wolf, and S. Krishnaswamy. Manifold
interpolating optimal-transport flows for trajectory inference. Advances in Neural Information Processing Systems, 35:29705-29718,, 2022.
[25] A. Hyvärinen and P. Dayan. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research, 6(4), 2005.
[26] B. Jing, G. Corso, R. Berlinghieri, and T. Jaakkola. Subspace diffusion generative models. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Ixrael, October 23-27, 2022, Proceedings. Part XXIII, pages 279-289. Springer, 2022.
[27] A. generating data with core-based models. arXiv preprint arXiv:210. 14080, 2021.
K. Li, R. Faill-Tail,eeer, T. Kachman, and I. Mitliagkas. Gotta fast when
[28] T. Karras, S. Laine, and T. Aila. A style-base generator architecture for generative adversarial networks.
In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4401-4410, 2019.
[29] T. Karras, M. Aittala, T. Aila, and S. Laine. Elucidating the design space of diffusion-basse generative models. Advances in Neural Information Processing Systems, 15:26565-26677. 2022.
[30] B. Kawar, G. Vaksman, and M. Elad. Snips: Solving noisy inverse problems stochastically. Advances in Neural Information Processing Systems, 34:21757-21769 2021.
[31] B. Kim and J. C. Ye. Denoising MCMC for accelerating diffusion-based generative models. arXiv preprint arXiv:2209. 14593, 2022.
[32] D. Kim, Y. Kim, W. Kang, and I.-C. Moon. Renenaiigenerative scorc- score-based diffusion models. arXiv preprint arXiv:2211.1709. 2022.
generative process with discriminator guidance in
[33] D. Kim, Na, S. Kwon, D. Lee, W. Kang, and T.-c. Moon. Maximum likelihood training of implicit nonlinear diffusion model. In Advances in Neural Information Processing Systems, 2022.
[34] D. Kim, Shin, Song, W. Kang, and I.-C. Moon. Soft truncation: universal training technique of score-basee based diffusion model for high precision score estimation. In International Conference on Machine Learning, pages 1 11201-1 12228. PMLR, 2022.
[35] G. Kim, T. Kwon, and In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2426- 2435, 2022.
Diffusionclip: Text-guided diffiusion models robust image manipulation.
[36] D. Kingma, T. Salimans, B. Poole, and J. Ho. Variational diffusion models. Advances in neural information processing systems, 34:1166-211777, 2021.
[37] D. Kingma and M. Welling. Auto-encoding variational bayes. arXiv preprint arXiv: 2013.
[38] K. Lee and J. Shin. Rényicl: Contrastive representation learning with skew Rényi divergence. Advances in Neural Information Processing Systems, 35:6463-66 6477, 2022.
[39] C. Léonard. A survey of the Schrödinger problem and some of its connections with optimal transport. arXiv preprint arXiv:/308.00 008.0225. 2013.
[40] C. Léonard. Some properties of path measures. Séminaire de Probabilités XLVI, pages 207-230, 2014.
[41] X. Li, T.-K. L. Wong, R. T. Chen, and D. K. Duvenaud. Scalable gradients and variational inference for stochastic differential cquations. In Symposium on Advances in Approximate Bayesian Inference, pages 1-28. PMLR, 2020.
[42] I. Liu, Y. Ren, 7. Lin, and 7. Zhao. Pscudo numerical methods for diffusion models on manifolds. In International Conference on Learning Representations. 2022.
[43] C. Lu. K. Zheng, F. Bao, J. Chen, C. Li, and J. Zhu. Maximum likelihood training for score-based diffusion odes by high order denoising score matching. In International Conference on Machine Learning, pages 14429-144 14460. PMLR, 2022.
12

---

[44] C. Lu, Y. Zhou, F. Bao, J. Chen, C. Li, and J. Zha. DPM-solver: A last ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems, 35:5775-5787. 2022.
[45] C. Luo. Understanding diffusion models: A unified perspective. arXiv preprint arXiv:2208. 11970, 2022.
[46] Z. Lyu, Z. Kong, X. Xudong, L. Pan, and D. Lin. A conditional point diffussin--rfinemman paradigm for 3d point cloud completion. In International Conference on Learning Representations, 2022.
[47] A. Q. Nichol and P. Dhariwal. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning, pages 8162-8171. PMLR, 2021.
[48] B. Oksendal. Stochastic differential equations: an introduction with applications. Springer Science & Business Media, 2013.
[49] A. v. d. Oord, Y. Li, and O. Vinyals. Representation learning with contrastive predictive coding. arXiv preprint arXiv: nenn:0007003700 2018.
[50] B. Poole, S. Ozair, A. Van Den Oord, A. Alemi, and G. Tucker. On variational bounds of mutual information.
In International Conference on Machine Learning, pages 1101-5180. PMLR, 2019.
[51] A. Radford, W. Kim, C. Hallacy, A, Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin,
J. Clark, et al. Learning transferable visual models from natural language supervision. In International conference machine learning, pages 8748-8763. 8763. PMLR, 2021.
[52] A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen. Hicrarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.000 226.02 2022.
[53] T. Salimans and J. Ho. Progressive distillation for fast sampling diffusion models. In International Conference on Learning Representations, 2022.
[54] S. Särkkä and A. Solin. Applied stochastic differentiall equations, volume 10. Cambridge University Press, 2019.
[55] A. Sauer, Schwarz, and A. Geiger. StyleGAN-XL: Scaling StyleGAN to large diverse datasets. ACM SIGGRAPH 2022 conference proceedings, pages 1 1-10, 2022.
[56] J. Song, C. Meng, and S. Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2022.
[57] Y. Song and S. Ermon. Generative modeling by estimating gradients of the data distribution. Advances neural information processing systems, 32, 2019.
[58] Y. Song and S. Ermon. Improved techniques for training score-based generative models. Advances in neural information processing systems, 13:12488-12448, 2020.
[59] Y. Song, S. Garg. J. Shi, and S. Ermon. Sliced score matching: A scalable approach density and score estimation. In Uncertainty Artificial Intelligence, pages 574-584. PMLR, 2020.
[60] Y. Song, C. Durkan, I. Murray, and S. Ermon. Maximum likelihood training of score-based diffusion models. Advances in Neural Information Processing Systems, 34:144::44 14:1415-1428, 2021.
[61] Y. Song. J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole. Score-based generative mod- eling through stochastic differential equations. In International Conference on Learning Representations, 2021.
[62] Y. Song, L. Shen, L. Xing, and S. Ermon. Solving inverse problems in medical imaging with score-based generative models. In International Conference on Learning Representations, 2022.
[63] Y. Song, P. Dhariwal, M. Chen, and I. Sutskever. Consistency models. arXiv preprint 26310:2303.01469. 2023.
[64] X. Su, J. Song, C. Meng, and S. Ermon. Dual diffusion implicit bridges for image-to-image translation. In The Eleventh International Conference on Learning Representations, 2022.
[65] H. Tachibana, M. Go. M. Inahara, Y. Katayama, and Y. Watanabe. Itô-taylor sampling scheme for denoising diffusion probabilistic models using ideal derivatives. arXiv e-prints, pages arXiv-2112. 2021.
[66] B. Tzen and M. Raginsky. Neural stochastic differential equations: Deep latent gaussian models in the diffusion limit. arxiv preprint 190.1105 2019.
13

---

[67] A. Vahdat. K. Kreis, and J. Kautz. Score-based generative modeling in latent space. Advances in Neural Information Processing Systems, 1487133711133,, 2021.
[68] P. Vincent. A connection between score matching and denoising autoencoders. Neural computation, 23(7): 1661-16744 2011.
[69] G. Wang, Y. Jiao, Q. Xu, Y. Wang, and C. Yang. Deep generative learning via Schrödinger bridge. In International Conference Machine Learning, pages 10794-10804. PMLR, 2021.
[70] Y. Wang, X. Wang, A.-D. Dinh, B. Du, and C. Xu. Learning to schedule in diffusion probabilistic models. In Proceedings the 29th ACM SIGKDD Conference Knowledge Discovery and Data Mining, pages 2478-2488, 2023.
[71] Z. Wang, H. Zheng, P. He, W. Chen, and M. Zhou. Diffusion-GAN: Training GANs with diffusion. In International Conference on Learning Representations. 2023.
[72] D. Watson, W. Chan, J. Ho, and M. Norouzi. Learning fast samplers for diffusion models by differentiating
through sample quality. International Conference on Learning Representations, 2022.
[73] Y. Yu, X. Si, Hu, and Zhang. review of recurrent neural networks: L.TTM cells and network architectures. Neural computation, 31(7):1235-12770 2019.
[74] Q. Zhang and Y. Chen. Diffusion normalizing flow. Advances in Neural Information Processing Systems, 34: 16280-1621, 2021.
[75] Q. Zhang and Chen. Fast sampling of diffusion models with exponential integrator. International Conference on Learning Representations. 2023.
[76] R. Zhang, Isola, A. A. Efros, E. Shechtman, and O. Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586-595, 2018.
14

---

Appendix
A
Related Works
In this paper, our focus lies in reducing the discretization error by optimizing the Kullback-Leibe (KL) divergence at each time step between the true sampling chain and the simulated chain. To this end, our method involves fine-tuning pre-trained Diffusion Models (DMs) by minimizing combination of the contrastive loss and the score matching loss. Thus, our objective function aims to optimize the upper bound of each KL divergence between the true sampling chain and the simulated chain. In line with this design philosophy, our method can be regarded as solving the Schrödinger 13I. Within this context, extensive research has been conducted to enhance DMs 61 15] (74), a diffusion normalizing flow is proposed, which jointly trains the two neural stochastic differential equations (SDEs) (64] to minimize common cost function that quantifies the difference between the two. By comparison, the diffusion Schrödinger Bridge optimizes an regulaazee transport problem on path spaces, yields diffusions which generate samples from the data distribution in finite time [8] [0]. In [69], they propose to Iearn a generative model via entropy interpolation [24] with Schrödinger Bridge. This however comes with a heavy training overhead since they train the model from scratch. By contrast, our method fine-tune pre-trained DMs via reducing the gap between the true sampling chain and the simulated chain.
Bridges 39 40
55 22
On the other hand, certain fast sampling methods aim to essentially reduce the discretization error in order to enhance both the sampling speed and the quality of the results. In (56) the authors use non-Markovian forward noising. The resulted algorihtm, DDIM, achieves significant acceleration than DDPMs. More recently, the authors of 44 optimize the backward Markovian process to approximate the non-Markovian forward process and get an analytic expression of optimal variance in denoising process. Concurrent to this work, Zhang and Chen (75) propose diffusion exponential integrator sampler, which leverages a semilinear structure of the learned diffusion process to reduce the discretization error. Orthogonal to those direction, our focus lies in minimizing the discretization error by optimizing the upper bound of the Kullback-Leibe (KL) divergence between the true data distribution and the model distribution.
B Proofs
We follow the regularity assumptions in 60 43 to prove the Theorem 1 For completeness, we list all these assumptions in this section. We use C to denote all continuous functions, and let C* denote the family of functions with continuous k-th order derivatives.
Assumptions We make the following assumptions throughout the paper:
(i) p(x) E C2 and Ex~p [|] < cc.
(ii (x) E C² and Ex-r ||| < co.
(iii) Vi E 7]: f(,1) E 30 > 0Vx E RREEEREEE :1f]< & CC + [|x|2))
(iv) CC > 0, Vx, ERD [[f(x, t) f(y,,))l2&&
(v) g Candvt [o, [t() 0.
(vi) For any open bounded set O. f-" fo (x) + Dg(()² (xp((x))12 dxdt < co.
(vii) 3C > 0Vx E RD, ,,, E [0, : | logg: < C1 ||x||2).
(viii) 30 > 0, 0, RP: V (x) Vy (g)p((y)),, & Clix y|12-
(ix) 3C > 0/xx R t E [0. T || 1)1 < C1 + ||x|2]]
(x) 3C > 0, yx, E R ||,o(x,)) t) - so sy 1)1 & C'|x -yll.
(xi) Novikov's condition: E exp (1 50 | log (x) - s0(x, 1)12 at) 80.
(xii) VL E TEK 0: px()) O (c-1xxI² 0
a=||>||2 + 00.
15

---

Below we provide all proofs for our theorems.
Proof. Let μ and in the Theorem denote the path measures 40 38
of SDES in Eq. ① and Eq. (4)
with = respectively. Due to assumptions (i) (ii) (iii) (iv) (v) (ix) and (x), both l and v are uniquely
given by the corresponding SDEs. Consider Markov kemel K ((=*) he [0, 7 y) ō(20 y)-
Since 2g ~ Po(2)), and 20 Po
SDD), we have the following result
K (((
= po(
/ <((="""))E00m)) dx) (x)
(DE)
SDE)
) - 160 (20 (> )
SDE
Here the Markov kernel K essentially performs marginalization of path measures to obtain "sliced" distributions & 0. We can use the data processing inequality with this Markov kemel to obtain
DKL (po (pullro ( DKL (f
( (fachrejo.)) delll / K (10, (1=;DE) TEEoo,Ty x
SDE
de
(16)
<DRL(Hlle) = f. (pL(P (Dpp
From the perspective of the Schrödinger Bridge, the KL divergence between the true data distribution and the model distribution bounded by the integral KL at each time step f [0, T] between μ and On the other hand, the value of any KL is greater than or equal to zero. Hence, enable also to obtain
(KI(u)ll::) = J. DKL (pi lp
DEL (oollpoEE) <
SDE ydp.
(17)
Recall that by definition De Pe(tr) and 2ΩDE (2: ). and now assume Pt distribution. Leveraging the chain rule of KL divergences [40], we have
SDE is prior
(Ku(Pelle)) PKL (pellesDE) + Eazpp [PEL (M(- x, z)|l (. (11x (DP)) = 2))]
(18)
Under assumptions (i) (iii) (iv) (v) (vi) (vii) (viii), the SDE in Eq. has corresponding reverse-time SDE given by
dx Foodt G. G, V pz)) + Gidw.
(19)
Since Eq. 19 is the time reversal of Eq. it induces the same path measure μ. As result, DKL (H(- x, = z)|lP/ x F = z)) can viewed as the KL divergence between the path measures induced by the following two (reverse-time) SDEs:
dx Fordt G.G. loggp dt Gidw, 21 =
dx Firdl G. so (2.DE)) di + G.dwe, xit
The KL divergence between two SDEs with shared diffusion coefficients and starting points exists under assumptions (vii) (viii) (ix) (x) (xi) (see in [4] 66)), and can be computed via the Girsanov theorem
[48]
DKL (H,(- In = =2]l ( SDE z))
E,e,
log de
des
3 Ep 1o' g() (Vx logpi(x) so(x, dw, 1o 9()) logg(((( so(x, 1)12 di
( En 1 So (i)2 V log so(x, 1)12 di
fo Em:(x)
g
V logp sx i)]1"" i) di
(11)
5 ff Ep,co
so(xx i) di
JsM (.)))
(20)
where (i) due to Girsanov Theorem II
48 Theorem 8.6.6), and (ii) is due to the martingale 16
property of Itō integrals. Combining Eqs.
18 and (20) completes the proof of Theorem
16

---

Though we demonstrate that the combination of score matching losses JSM (0: and KL term DKL (pe () is upper bound of DKL (po ( it is intractable to optimize Eq. (9) due to the unknown function form of pt. To circumvent this problem. we combined with the mutual information (Ml) theory [50] that MI between ((( and (2D can be expressed
as I (1 ) PJ process can be shown
(SDPF(2 (25 I (pi(pp SDE pt), where j E [0.T] and j + t. The detailed
log p DD
I [Pt
SDE SDE
Pj
SDE SDE )) = By(agDE, (DDB
SDEE (z) LTDE))
ps
log p (IDDE
DDE SDE) (DD (24)
Pj
=B pla spnn)
(r(
SDE ())DE)
(21)
p(1 Pj (DE))
log
(Ri
SDE (x; pz()))
(xt)
< p (D)
DRL (p(z) SDE DDDE) 1| Pr(x)))
= -Bpgn (pt Pi)]
SDE
Therefore, we obtain the upper bound of (p (x; p3 (x; given by DEL (pi SDE pt)- However, it it still intractable since the marginal distribution (x)) is inaccessible. To remedy this, when applied the Jensen's inequality to this term, enable to obtain upper bound of
DEL (Pt Pt SDDE) by a InfoNCE loss: JInfNNCE(")) DD)B)) 2 Dx p SDD)) [4), where
j E 10. 7 and j + t. To prove this upper bound in detail, we first review the original lower bound of MI 49
I( 2
SDE SDB) 2 log N - II SDE SDE), ,
(22)
where log N means the total numbers of positive pair (x SDDD)) when calculating the InfoNCE loss. To show clearly the connection between MI and the InfoNCE loss, we demonstrate the derivation process as follows
SDR
(nfoNCe(1))
SDE
.rj
-EE log x
p (uEDE a SDEE
p( (x=TE)
p(=2DD))
EXnng
p(x
(2 SDE)
P (x SDD
E log
1 1+
P (1)
SDE
1 SDE
p ((x-)
Ex
(ZSDDE)
P (x- SDDD)
(23)
2 E log X
(N
1) E
P (x SDR
SDEE
p p(x
x
pa SDE)
E log X
(N 1)
p [ SDE 1 SDE
p (DE
2 E log
N p (1: (EED 1 SDE)
X
=-1
(1,DE
2 SDE + log(NV)
Here, x- negative instances sampled from training images. Eqs. 23 quickly becomes more accurate as N increases. At the same time log(N)- use large values of N. In practice, obtain much more samples from the given sampling chain to form the positive pairs in each training iteration. In consequence, when combined with Eq. 21 have
[2 [ 25SDE) also mcreases, so it's useful to choose a bigger batch size for the large N. Concretely,
SDE) > log N InfoNCE(1)) SDE .I)
I(2 SDE
2
SDE)
log N DKL(PSDE SDE pt).
(24)
17

---

Subsequently, we apply the Jensen's inequality to further scale Eq.
24
[1 SDE
SDE)
2 log.N - DKL (pi
SDE
I1 Pt)
SDE ( (x)DE)
-logN Pt
SDE
SDE) log
Pt
(x(tt)
(25)
SDE
SDE
SDE (1rt (x.DE)
Pt
ZlogN log
(r()
-logN 2 llogp Pi
SDE
(x SDE) + (ot):
However, the KL in Eq. (9) is DKL (pe |l Pi (DE) which not equivalent to DEL SDE the gap, scale the former as follows
||
p.). To fill
Pe(xt)
(PI(Pe || p) = (xx) (og))og log
pt D.DE ( (I)DE)
(26)
Pe (x pr
<
= po pr (z+) log SDE SDE
Pt SDR (DE
Therefore, when combined Eq.
25 and Eq. 26), we have
SDE InfoNCE (2
(ISDE)
SDE
DKL (pr pt )
= log.N 2 log Pt
SDE [z)
( + log pt (x4) - [2og)(( - log p.
SDE (1
logNN [log Pt
(2 (I,DE) - log (x:))
SDE
(27)
log.N log pi
SDE ( (x.DB)) (x(
log.N
where (a) is due to both pt ( I 1 and pr (rx) < 1 because of the attribute of and naturally there
marginal distribution. Hence, it is reasonable that pi SDE (xr (a.DE) ppp (x+) is log p (a:$E))
0. Accordingly, we have InntoNCE (a+ SDE SDD)) 2 DKL, || DPDE).
2y Under such perspective, we replace (RI(pp: ll DDD in Eq. 9 by (nnoNCE((1) have
SDE ..SDE) and
DKL (pollpp < [I
SDD
SDF) 3
+ JSM (0:g(t)²).
(28)
In practice we enable to fine-tune pre-trained DMs with Eq.
Moreover, JSM (0: g())) is almost fixed in each pre-trained DM. Similarly, optimizing Eq. (28) is equivalent to minimizing DKL p. D.D... Therefore, our objective function is essentially solving the Schrödinger Bridges.
28
C More Experiment Details
In this section, we will provide a more comprehensive account of our experiments, focusing on the implementation of DMs. To conduct our experiments, we utilized NVIDIA A100 GPUs for conducting experiments on CIFAR-I0, CelebA, FFHQ, and ImageNet datasets. Additionally, we employed Nvidia V100 GPUs for experiments involving the combination of fast sampling methods. Based on our theoretical analysis, to optimize the KL divergence DKL || pŞPE), is sufficient to add an InfoNCE loss to mitigate the discretization error. As result, this flexible method can be applied to fine-tuning various pre-trained DMs, both with and without training- free fast sampling algorithms, thereby enhancing sample quality or achieving marginally faster sampling speeds.
To achieve this, one simply needs to include sampling chains given by pre-trained DMs or defined by fast sampling algorithms in the training process. This involves first selecting one sample 2iDE and then randomly choose another sample 25 from the chain to calculate the contrastive loss and form the total loss using Eq. (15). Afterwards, mack-propagation through time (BPTT) is utilized to update the parameters of DMs, with the gradient flow directed in the opposite direction of this chain. is worth noting that we maintain all the training settings of the pre- pre- treitraieed DMs and only modify the part that constructs the contrastive loss, thereby demonstrating the flexibility of our method. Similarly, to showcase the scalability of our method on chains defined by fast sampling methods, we replace the original chains provided by pre-trained DMs with new chains defined by fast sampling methods, while retaining all other settings the same fine-tuning pre- trained DMs. To fine-tune pre-trained
18

---

DM with sampling chain of default N steps in practice, we extend it to a chain of 2N, 3., even 4N steps construct the contrastive loss. Once the fine-tuning process is completed. we test the performance of the N-step chain by drawing 50,000 samples from it and measure the widely adopted FID score, negative log-likelihood (NLL), and NFEs, where lower values indicate better performance. Ideally, default chain in pre-trained DM with more steps will demonstrate better performance when fine-tuned by our method. However, a very long sampling chain suffers from gradient disappearance when BPTT is executed. Hence, we utilize different steps to fine-tune different DMs for different target datasets and demonstrate only the best results in this paper.
19


Reviews:
[{"review_id": "gzjGCQaPAQ", "reviewer": "Reviewer_mcPT", "summary": "The paper targets the discretization error of the diffusion SDE sampling, especially when the number of sampling steps is small. \nThe authors propose a contrastive loss for diffusion sampling where instances on the same sampling chain are deemed positive pairs. The InfoNCE loss provides an upper bound on the KL divergence at time t. \nIt is shown that an appropriate combination of contrastive loss and score matching serves as an upper bound for the KL divergence between the data distribution and the model distribution. \nThe proposed training objective (equation 10) is a combination of the score matching loss and an InfoNCE loss and can be used to fine-tune any diffusion models' samplers.  \n\nHaving introduced several two hyperparameters, beta(t) for balancing the two terms and temperature $\\tau$), the authors experimented with different training strategies with ablation studies. \nExperiments on CIFAR10 and ImageNet64 generation show that the sampler tuned with their proposed method outperforms the baseline diffusion model sampler. ", "questions": "1. Is the MoCo V2 encoder pretrained on ImageNet? Is the encoder in all the experiments the same (except for changing resolutions)? \n\n2. How stable is the proposed method, regarding the randomness during the training process? How sensitive are the results to different hyperparameter choices?\n\n3. For weighting function $\\beta(t)$, is there any optimal/analytical solutions? ", "limitations": "Limitations and broader impact have been discussed. ", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The writing of the paper is clear. The motivation and presentation of the results are easy to follow. \n\nThe idea of introducing contrastive learning to DMs is interesting and novel to the best of my knowledge. The proposed methodology can be widely applicable to any pre-trained diffusion models as a post-fix, which can be valuable to the community. \n\n\n\n", "weaknesses": "\n### 1. Pre-trained encoder might be unfair \n\nFor all datasets, the authors use the pre-trained MoCo V2 encoder (I assume it was trained on ImageNet? Please correct me if I am wrong) to extract features in order to compute the infoNCE loss. This raises some concerns: \n\n(1) The MoCo is pre-trained on datasets larger than CIFAR10 (and other small datasets), so the encoder has seen more data than diffusion models. Given that, it becomes less clear whether the improvement is coming from the contrastive loss itself or the encoder has seen more data. What makes it more concerning for me is that the gain from contrastive loss seems less significant on the more complicated ImageNet64 dataset. Intuitively, as the dataset gets more complicated, the baseline performance is weaker and may leave more room for improvement (correct me if I am wrong). I suggest the authors try training using encoders trained on CIFAR10 for evaluating diffusion models on CIFAR10 to make a more convincing case. \n\n(2) For some forms of data, there is no pre-trained encoder, so the author proposed method may not be easily adapted. Of course we can train from scratch. But that introduces more computational overhead, which makes the method less appealing. \n\n### 2. Marginal performance gap on more complicated data\nOn the ImageNet64 conditional experiment, the finetuned sampler shows only a marginal performance improvement over the default sampler. No standard deviation of the method is reported so I am not sure whether the improvement is statistically significant. \n\n### 3. Concerns about training: \nThe infoNCE loss requires differentiable samples that are generated through multiple evaluations of the diffusion model's network, this may lead to a heavy computational overhead to differentiate through the entire computational graph. The extra computation cost is not very clear. \n\nThe proposed method seems brittle with many moving parts and hard to tune from task to task (different datasets, models, steps). Even only changing the sampling steps needs extra training. In comparison, the diffusion model's sampler can change flexibly with arbitrary sampling steps. ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Weaknesses\n### 1. MoCo is pre-trained on ImageNet larger than CIFAR10.\nThe MoCo V2 encoder utilized in this paper is pre-trained on ImageNet, and we use this encoder to fine-tune DMs in all our experiments. To further analysis the performance of our method, we conduct an ablation experiment on pre-trained EDM with the help of the MoCo V2 encoder pre-trained on CIFAR-10. Concretely, we keep the same training settings as previous experiments and fine-tune the pre-trained EDM with only 10 epochs. Empirically, we decrease the FID (lower is better) from 2.04 to 1.95, shown in the below Table. The 0.09 reduction of FID proved that our method effectively improves the generative performance of the pre-trained EDM.\n| Models     | FID$\\downarrow$     | Encoder |\n| :---: | :---: | :------: |\nCIFAR-10\n|   EDM | 2.04 | - |\n|   EDM-C++ (Ours)  |   1.95   | MoCo V2 (CIFAR-10) |\n|   EDM-C++ (Ours)  |   1.88   | MoCo V2 (ImageNet) |\nIDDPM (ImageNet, 14 sampling steps)\n|   DPM-Solver-2  |   4.46   | - |\n|   DPM-Solver-2-C++ (Ours)   |   4.38   | MoCo V2 (ImageNet) |\nIDDPM (ImageNet, 20 sampling steps)\n|   DPM-Solver-2  |   3.42   | - |\n|   DPM-Solver-2-C++ (Ours)   |   3.36   | MoCo V2 (ImageNet) |\n\nIn this paper, we utilize an encoder pre-trained on ImageNet to fine-tune pre-trained DMs. In this manner, our method helps the pre-trained EDM reduces the FID on CIFAR-10 from 1.95 to 1.88, seen in the above table. The 0.07 reduction of FID demonstrate that our method further improves the gerative performance. Hence, our contrastive sampling method contributes almost half of the FID reduction. \n\nOn the other hand, our method increases the FID from 3.42 to 3.36 on ImageNet with 20 steps when utilizes DPM-Solver-2 training-free sampler to sample images for evaluation, shown in Table 2. By contrast, the encoder pre-trained on CIFAR-10 decreases the FID from 2.04 to 1.95 on CIFAR-10 dataset. The 0.06 reduction of FID on ImageNet is an order of magnitude as the 0.09 reduction of FID on CIFAR-10. Hence, our method presents a consistent improvement when fine-tune pre-trained DMs with the corresponding encoder.\n### 2. For some forms of data, there is no pre-trained encoder.\nAs shown in the above table, we achieve 0.09 reduction of FID when utilize the encoder pre-trained on CIFAR-10, while further obtain 0.07 reduction of FID when utilize the encoder pre-trained on ImageNet. In this paper, we all utilize the pre-trained encoder trained on ImageNet and improve the performance on four different datasets, including CIFAR-10, FFHQ, CelebA and ImageNet. Hence, arbitrary pre-trained encoder can be utilized for improving other forms of data, which demonstrate the flexibility and scalability of our method.\n### 3.1. Only a marginal performance improvement on the ImageNet64 (conditional).\nOur method remarkably improves various DMs on CIFAR-10, FFHQ and CelebA, shown in Table 1 and 4. We also improve the generative performance on ImangeNet with very few sampling steps compared to prior DMs, see in Table 2. Moreover, our method demonstrates an consistent improvement on  class-conditioned 64 $\\times$ 64 ImageNet. For intance, when compared to DPM-Solver3 sampler, the sampler DEIS-tAB3 reduces FID from 2.72 to 2.69 with 50 sampling steps, and decrease the FID from 2.84 to 2.81 with 30 sampling steps.\n### 3.2. No standard deviation of the method is reported.\nPrior DMs have not shown the standard deviation results [1-3]. To demonstrate the stability of our approach, we conduct this experiment on the same settings for five times. Concretely, the FID results respectively are 1.8820, 1.8836, 1.8922, 1.8903, 1.8801, which fluctuate between 1.88 and 1.90.\n\n[1] T. Karras, M. Aittala, T. Aila, and S. Laine. Elucidating the design space of diffusion-based generative models. Advances in Neural Information Processing Systems 35: 26565-26577, 2022.\n\n[2] T. Salimans and J. Ho. Progressive distillation for fast sampling of diffusion models. In International Conference on Learning Representations, 2022.\n\n[3]  C. Lu, K. Zheng, F. Bao, J. Chen, C. Li, and J. Zhu. Maximum likelihood training for score-based diffusion odes by high order denoising score matching. In International Conference on Machine Learning, pages 14429-14460. PMLR, 2022.\n### 4. The extra computation cost.\nFor fine-tuning pre-trained EDM, our method allows achieving remarkable performance with only 20 epochs or even fewer, where each epoch costs about 130 seconds. For sampling images, the improved pre-trained EDM takes the similar overhead as the pre-trained EDM.\n### 5. Changing the sampling steps needs extra training.\nOur method enables to improve pre-trained DMs with arbitrary training-free samplers. For instance, we utilize the DEIS-tAB3 sampler to construct a sampling chain and fine-tune the pre-trained IDDPM via propagating gradients along opposite direction of this chain. Hence, we only needs to fine-tune IDDPM once and subsequently report experimental results on different sampling steps.\n## Questions\n### 1. How stable is the proposed method, regarding the randomness during the training process? How sensitive are the results to different hyperparameter choices?\nTo demonstrate the stability of our approach, we conduct experiments on the same settings with five times. Concretely, the FID results respectively are 1.8820, 1.8836, 1.8922, 1.8903, 1.8801, which fluctuate between 1.88 and 1.90. Actually, we only have one hyperparameter which is $\\alpha$ used for controlling the weighting schedules $\\beta (t)$. Concretely, our method achieves the similar results when $\\alpha$ is within a certain range, which is insensitive to small fluctuations.\n### 2. For weighting function beta(t), is there any optimal/analytical solutions?\nThis is a valuable question! It may exists an analytical solution since we have derived an upper bound in equation (10). An appropriate weighting function $\\beta(t)$ may change the inequality sign to equal sign in equation (10). We left it for future work.\n"}, {"review_id": "tWNnyAyBhA", "reviewer": "Reviewer_sJua", "summary": "This paper employs the contrastive loss to construct a contrastive sampling chain, which optimizes the KL divergence between the true sampling chain and the simulated chain at each time step to reduce the discretization error associated with numerical solvers used for solving SDEs. Experimental results demonstrate that this method improves sample quality and log-likelihood, while slightly accelerating pre-trained DMs.", "questions": "1. How should individual tasks choose between the linear weighting schedule and the nonlinear weighting schedule?\n\n2. What is the expression for bate(t)? \n\n3. Figure 1 provides limited information. It might be beneficial to incorporate additional details.", "limitations": "The authors refine the diffusion models by optimizing the upper bound of the KL divergence between the true sampling chain and a simulated chain. The remaining limitations and broader impact are discussed towards the conclusion of the paper.", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "++ The authors provide a comprehensive error analysis, theoretical analysis, and theoretical proof regarding the causes and upper bounds of discretization error between the true distribution and its corresponding model distribution.\n\n++ This paper conducts extensive experiments to demonstrate the performance on pre-trained diffusion models and fast samplers , and provides ablation studies to assess the impact of different techniques.\n\n++ Reducing the discretization error is an important area for diffusion models optimization. This paper offers a novel solution by minimizing discretization error through optimizing the upper bound of the Kullback-Leibler (KL) divergence between the true sampling chain and a simulated chain at each time step.", "weaknesses": "-- It is preferable to provide evaluation metrics and visual results for the outputs of generative models, rather than solely focusing on the performance of enhancing pre-trained diffusion models.\n\n-- The formatting of Tables 1, 2, and 4 could be improved, for instance, by subdividing them into several sub-tables to enhance readability, rather than separating different settings with a grey background.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Weaknesses\n### 1. It is preferable to provide evaluation metrics and visual results for the outputs of generative models, rather than solely focusing on the performance of enhancing pre-trained diffusion models.\nTo qualitatively evaluate generated images, we visualize some generated images in the one-page PDF. These generated images contain rich and coherent semantic information. It is sufficient to prove the effectiveness of our method.\n\n### 2. The formatting of Tables 1, 2, and 4 could be improved, for instance, by subdividing them into several sub-tables to enhance readability, rather than separating different settings with a grey background.\nThanks for pointing out this. We will reformat those Tables in a more clarity manner and help others understand them quickly.\n\n## Questions\n### 1. How should individual tasks choose between the linear weighting schedule and the nonlinear weighting schedule?\nThere is no limitation to choose weighting schedule for improving individual tasks. We empirically achieved similar results on various datasets when utilize those two weighting schedules. For instance, when fine-tune pre-trained EDM on CIFAR-10, we both obtain 1.88 FID.\n\n### 2. What is the expression for bate(t)?\nThank you for pointing out our typos! This is actually due to our carelessness in not expressing the $ \\beta(t)$ correctly. We apologize for this misrepresentation. The correct form of $ \\beta(t)$ is $\\beta(t)=\\alpha * (T-t)$ in equation (13) and $ \\beta(t)=\\alpha * \\operatorname{PNSR}\\left(x_{j}^{\\mathrm{SDE}}, x_{t}^{\\mathrm{SDE}}\\right) $ in equation (14).\n\n### 3. Figure 1 provides limited information. It might be beneficial to incorporate additional details.\nWe incorporate more additional details into the Figure 1 and can be seen in the one-page PDF."}, {"review_id": "aOLHzgZMAC", "reviewer": "Reviewer_e493", "summary": "This paper focuses on diffusion models (DMs). Current DMs suffer from discretization error since it leverage the numerical solvers to solve SDEs. To overcome this issue, authors propose a a contrastive loss when optimizing  DMs.  Authors present a theoretical analysis to demonstrate that combining both the generative loss and  the contrastive loss  are reasonable.  Instead of training DMs from scratch, this work optimize their model from the pratrained model.", "questions": "The method are demonstrated with both the small data and the pretrained model, which is hard to support the generalization.", "limitations": "1) I recommend to train the proposed method from scratch with large dataset.\n\n2) Showing a few images is not expensive, right?", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1) This paper firstly analyze DMs suffers from  discretization error, which sounds interesting. Then to address it, a contractive loss is introduced.\n\n2) Authors present a detail theoretical analysis about the provided problem and method.\n\n3) This paper is well-originated.", "weaknesses": "1) Why it use the pretrained model? how about the performance when optimizing it from scratch? I am not convincing about the generalization capability.\n\n2) DMs is attracting due to the large model trained on huge dataset. However, this paper only verify this proposed on small dataset, which is not convincing. I am not sure whether it still keep the advantage of the proposed method. \n\n3) Is it expensive to visualize the generated image？  even in the supplementary material.\n\n4) Also there are not the corresponding codes to reproduce the reported results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Weaknesses\n### 1.Why it use the pretrained model? how about the performance when optimizing it from scratch? I am not convincing about the generalization capability. #\nIt is traditional [1-6] to improve generative performance via fine-tuning pre-trained diffusion models (DMs). For instance, [1,4] optimize the pre-trained DMs with the help of knowledge distillation techniques. Moreover, [2] utilizes discriminator guidance to refine pre-trained EDM and achieve state-of-the-art generative performance on some datasets, while [6] proposes a consistent loss to improve pre-trained EDM and greatly improve the sampling speed of DMs. Motivated by this, we propose a plug-and-play approach to enhance different DMs. Compared to training from scratch, our method allows achieving remarkable performance with training only 20 epochs or fewer when fine-tuning pre-trained DMs.\n\nOn the other hand, our method remarkably improves generative performace of various pre-trained DMs, such as EDM, LSGM, STDDPM and IDDPM. Concretely, we reduce the FID of EDM and LSGM on CIFAR-10, and decrease the FID of STDDPM and EDM on CelebA and FFHQ respectively. Morver, we also conduct experiments on some training-free fast samplers and achieve obvious improvements on ImageNet dataset, seen in Table 2. This flexibility enables us to enhance various off-the-shelf pre-trained DMs, effectively demonstrating its generalization capability.\n\n[1] T. Salimans and J. Ho. Progressive Distillation for Fast Sampling of Diffusion Models. In International Conference on Learning Representations, 2021.\n\n[2] D. Kim, Y. Kim, W. Kang, and I.-C. Moon. Refining generative process with discriminator guidance in score-based diffusion models. arXiv preprint arXiv:2211.17091, 2022.\n\n[3] Z. Zhang, Z. Zhao, and Z. Lin. Unsupervised representation learning from pre-trained diffusion probabilistic models. Advances in Neural Information Processing Systems, 35:22117-22130, 2022.\n\n[4] C. Meng, R. Rombach, R. Gao, D. Kingma, S. Ermon, J. Ho, and T. Salimans. On distillation of guided diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14297-14306, 2023.\n\n[5] M. Careil, J. Verbeek, and S. Lathuilière. Few-shot Semantic Image Synthesis with Class Affinity Transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 23611-23620), 2023.\n\n[6] Y. Song, P. Dhariwal, M. Chen, and I. Sutskever. Consistency Models. arXiv e-prints, arXiv-2303, 2023.\n\n\n### 2. DMs is attracting due to the large model trained on huge dataset. However, this paper only verify this proposed on small dataset, which is not convincing. I am not sure whether it still keep the advantage of the proposed method. #\nThe largest dataset used in recent DMs works, such as **ADM [1]**, **EDM [2]**, **Dpm-solver [3]** and **FDM [6]**, as well as other DMs **[4-5]**, is the imagenet dataset. ImageNet dataset contains more than one million images which can not be regared as a \"small dataset\". Beyond Imagenet, these works also tend to evaluate their methods on other moderate-sized datasetes, e.g., **FFHQ** dataset, **CelebA** dataset and **CIFAR-10** dataset. We also follow this well-establsihed experiemntal setup to evaluate our algorithm. According to Table 2 in the paper, our algorithm can acheive improve performance on the ImageNet dataset. For instance, we reduce the FID from 3.67 to 3.60 with 14 steps on ImageNet when combined with DEIS-tAB3 training-free sampler. Moreover, we also we decrease the FID (lower is better) from 24.62 to 22.65 with 12 steps on ImageNet when combined with DPM-Solver-3 training-free sampler. Based on this evidence, our method is enable to keep the advantage on large dataset.\n\n[1] P. Dhariwal and A. Nichol. Diffusion models beat gans on image synthesis. Advances in Neural Information Processing Systems, 34:8780-8794, 2021.\n\n[2] T. Karras, M. Aittala, T. Aila, and S. Laine. Elucidating the design space of diffusion-based generative models. Advances in Neural Information Processing Systems 35: 26565-26577, 2022.\n\n[3] C. Lu, Y. Zhou, F. Bao, J. Chen, C. Li, and J. Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. arXiv preprint arXiv:2206.00927, 2022.\n\n[4]  C. Lu, K. Zheng, F. Bao, J. Chen, C. Li, and J. Zhu. Maximum likelihood training for score-based diffusion odes by high order denoising score matching. In International Conference on Machine Learning, pages 14429-14460. PMLR, 2022.\n\n[5] S. Wizadwongsa and S. Suwajanakorn. Accelerating Guided Diffusion Sampling with Splitting Numerical Methods. In International Conference on Learning Representations, 2023.\n\n[6] W. Du, H. Zhang, T. Yang, and Y. Du. A Flexible Diffusion Model. In International Conference on Machine Learning, pages 8678-8696. PMLR, 2023.\n\n### 3. Visualize the generated image\nTo qualitatively evaluate generated images, we randomly visualize some generated images in the one-page PDF. These generated images contain rich and coherent semantic information. It is sufficient to prove the effectiveness of our method.\n\n### 4. Also there are not the corresponding codes to reproduce the reported results.\nWe send our code to Area Chair according to rebuttal policy."}, {"review_id": "KPhByfDBqr", "reviewer": "Reviewer_AaWg", "summary": "This paper proposes to fine-tune pre-trained diffusion models using contrastive losses to reduce discretization errors. The positive pair is formed by the same image at different steps, while the negative pair is formed by different images. To better optimize the contrastive loss, dynamic weighting schedules and back-propagation through time techniques are used.", "questions": "Please address the issues in the Weaknesses.", "limitations": "The limitations are discussed. I would recommend adding a discussion about the fine-tuning cost of the proposed method.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well-motivated to handle an inherent problem of diffusion models, i.e., discretization errors. The theoretical conclusion is meaningful and well-aligned with the motivation.\n\n2. The proposed method of contrastive fine-tuning with weighting schedules and BPTT is interesting and well-aligned with motivation. One advantage of this method is that it is flexible to fine-tune various off-the-shelf pre-trained diffusion models, which makes it a good contribution to the field. And it is also compatible with fast-sampling methods.\n\n3. The experimental results are good in general, showing the effectiveness of the proposed method when combined with different baselines.", "weaknesses": "1. The term “Contrastive Sampling Chain” is a bit confusing to me, since the proposed method only fine-tunes the diffusion model without changing the sampling process. I would recommend using something like “Contrastive Diffusion Chain”. Likewise, the authors should avoid the saying “refinement of the sampling chain”, such as in L258-259.\n\n2. No generated examples are given for qualitative comparisons.\n\n3. It is unclear whether the code will be released or not.\n\n4. In L134, the analysis is based on ODE (lambda = 0). Can this analysis be applied similarly to cases where lambda is not zero?\n\n5. Minors issues:\n- In L27, “learn” should be “learns”\n- In L55, “aims” should be “aim”\n- In L91, “slightly equivalent”\n- In L104, “There” should be “Three”\n- In L106, “diffusion” should be “diffuse”\n- In L112, “modeling” should be “model”\n- You should have punctuation at the end of each equation\n- I would recommend adding short conclusions of the theory at the start of section 3 and section 4, for TL;DR purposes.\n- The math symbols are inconsistently in bold, such as x, s, and theta.\n- In L144-145, grammar errors.\n- In L211, the wrong position of citations.\n- In L263, should “j” be “t”?\n- In reference, wrong capitalization like “gans” and “Dpm”.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Weaknesses\n### 1. The term \"Contrastive Sampling Chain\" is a bit confusing to me, since the proposed method only fine-tunes the diffusion model without changing the sampling process. I would recommend using something like \"Contrastive Diffusion Chain\". Likewise, the authors should avoid the saying \"refinement of the sampling chain\", such as in L258-259.\nThanks for your sincere review and valuable comments! We will change the term \"Contrastive Sampling Chain\" into a clearer presentation, where your recommendation \"Contrastive Diffusion Chain\" is one of our considerations. Moreover, we will fix the typos in L258-259.\n\n### 2. No generated examples are given for qualitative comparisons.\nTo qualitatively evaluate generated images, we visualize some generated images in the one-page PDF. These generated images contain rich and coherent semantic information. It is sufficient to prove the effectiveness of our method.\n\n### 3. It is unclear whether the code will be released or not.\nWe send our code to Area Chair as rebuttal policy. Moreover, we will also release our code on github after this review process.\n\n### 4. In L134, the analysis is based on ODE (lambda = 0). Can this analysis be applied similarly to cases where lambda is not zero?\n\nOur analysis on ODE ($ \\lambda =0$) is a special case of SDE ($\\lambda \\ne 0$). The discretization error is caused by numerical solver which shows no connection with the term $ \\lambda \\boldsymbol{G}_{t} d \\omega $ that throwed by $ \\lambda =0 $. \n\nIn other words, the gap between the approximate solution and the exact solution is not come from this term $ \\lambda \\boldsymbol{G}_{t} d \\omega $, shown in equation (6). Concretely, the integral term of equation (6) is the factor that causes the discretization error. \n\nBy comparison, $ \\lambda \\boldsymbol{G}_{t} d \\omega $ is not an integral term which can be solved without numerical solvers. Hence, Our analysis can be applied similarly to cases where lambda is not zero. The reason we present it this way is for convenience.\n\n### 5. Minors issues.\nWe will fix all these typos in our latest paper.\n\n\n## Limitations\n### 1. The limitations are discussed. I would recommend adding a discussion about the fine-tuning cost of the proposed method.\nFor fine-tuning pre-trained EDM, our method allows achieving remarkable performance with only 20 epochs or even fewer, where each epoch costs about 130 seconds. For sampling images, the improved pre-trained EDM takes the similar overhead as the pre-trained EDM."}, {"review_id": "7Rb57EmdQF", "reviewer": "Reviewer_pguc", "summary": "This paper analysis why diffusion models need addtional contrastive loss. The main target is the reduce $D_{KL}(p_t|| p_t^{SDE})$. This paper provides a detailed theoretical analysis and solid examples to compare its method with existing ones.", "questions": "From both a theoretical and experiential perspective, what are the distinct effects of consistent loss versus contrastive loss?", "limitations": "None", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper analysis the the gap $D_{KL}(p_0|| p_0^{SDE})$ to design better diffusion models. While this is not the first analysis of this object, its solution is still interesting.\n2. This paper clearly analyzes and explains why contrastive loss is useful for diffusion models, as opposed to simply combining multiple losses without clear motivation. These analyses are beneficial for future research in this area.\n3. The main paper's experiments incorporate the contrastive loss function along with several state-of-the-art acceleration methods, demonstrating robust performance.", "weaknesses": "1. In the loss function $I_{InfoNCE}(x_t^{SDE}, x_j^{SDE})$, the first item is represented by $exp(E(x_t^{SDE}), E(x_j^{SDE}))$. This item is similar to the consistent loss, which has been proven useful for diffusion models [1]. Therefore, it would be helpful if the authors could provide further analysis on the role of the second item, which is $exp(E(x_j^{SDE}), E(x^-))$, and clarify how consistent loss and contrastive loss have different influences.\n2. In Section 3, this paper discusses the discretization error. However, it is not the only error present  in the sampling process. Another significant factor to consider is the estimation error arising from $s_\\theta$. Although this may not influence the design of $I_{InfoNCE}(x_t^{SDE}, x_j^{SDE})$, I believe that a more detailed analysis should be provided by the authors.\n3. The OOD detection results presented in Table 2 are not satisfactory. Additionally, the connection between this part and the main contribution is unclear. Are the good detection results attributed to classical diffusion models or the new loss function proposed in this paper?\n\n[1] Daras G, Dagan Y, Dimakis A G, et al. Consistent diffusion models: Mitigating sampling drift by learning to be consistent[J]. arXiv preprint arXiv:2302.09057, 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "## Weaknesses\n### 1. Provide further analysis on the role of the second item and clarify how consistent loss and contrastive loss have different influences.\nThis question is quite valuable and may provide us further research direction! The contrastive loss is designed for reducing the distance between similar features and put away the dissimilar features. Hence, the contrastive loss will reduce the distance between $x_{j}^{\\mathrm{SDE}}$ and $ x_{t}^{\\mathrm{SDE}}$, and the second item $exp(E(x_{j}^{\\mathrm{SDE}}),E(x^{\\mathrm{-}}))$ of contrastive loss contribute to put away those dissimilar images $x_{j}^{\\mathrm{SDE}}$ and $x^{\\mathrm{-}}$. In this manner, the sampling chain will become tighter and the discretization error decreased accordingly because the distance between sampling steps decreased. \n\nBy comparison, consistent loss is designed for reducing score mismatching error. In this manner, consistent loss helps DMs to generate similar images $x_{0}^{\\mathrm{SDE}}$ as $x_{0}$, which enables to minish the gap between $x_{0}^{\\mathrm{SDE}}$ and $x_{0}$. In a word, consistent loss is utlized during traing process to decrease the score mismatching error while our method enables to reduce the discretization error via fine-tuning pre-trained DMs.\n\n### 2. However, it is not the only error present in the sampling process. Another significant factor to consider is the estimation error.\nThe errors present in the sampling process are composed by discretization error and score mismatching error. Though those two errors both influence the final quality of generated images, our work focuses on solving discretization error caused by numerical solvers. Score mismatching error arises during training process which shows little connection with numerical solvers.\n\n### 3. The OOD detection results presented in Table 2 are not satisfactory. Additionally, the connection between this part and the main contribution is unclear. Are the good detection results attributed to classical diffusion models or the new loss function proposed in this paper?\nOur work aims to improve generative performance of diffusion models and OOD detection is not one of our contributions. We apologize for not comprehending this question. Could you please explain it in detail?\n\n\n## Questions\n### 1. From both a theoretical and experiential perspective, what are the distinct effects of consistent loss versus contrastive loss?\nIn a theoretical perspective, our contrastive loss is mainly focus on discretization error which caused by numerical solvers during sampling process. By contrast, the consistent loss concentrates on reducing score mismatching error during training process. On the other hand, our contrastive loss enables to combine with both deterministic sampling and stochastic sampling while consistent loss works seamless with deterministic sampling.\n\nIn an experiential perspective, our contrastive loss decreases the FID on CIFAR-10 from 2.04 (random seed) to 1.88, which demonstrates a significant improvement on the pre-trained EDM. The consistent loss also helps the pre-trained EDM to reduce the FID from 1.97 (manual seed) to 1.95. Based on the above analysis, we guess consistent loss and contrastive loss can be combined to further improve DMs."}]

Author rebuttal:
We incorporate more additional details into the Figure 1 of our paper and visualize some generated images for qualitative analysis.

Meta-review: This paper employs contrastive loss to construct a contrastive sampling chain to optimize the KL divergence between the true sampling chain and the simulated chain to reduce the discretization error associated with numerical solvers used for solving SDEs. All reviewers agree that the idea of introducing contrastive learning to DMs is interesting and novel. The authors provide a comprehensive error analysis, theoretical analysis, and theoretical proof regarding the causes and upper bounds of discretization error between the true distribution and its corresponding model distribution. Experimental results demonstrate that this method improves sample quality and log-likelihood, while slightly accelerating pre-trained DMs. All the reviewers are satisfied with the authors' responses to the questions.
Decision: Accept (poster)