Learning Energy-Based Models by Cooperative Diffusion Recovery Likelihood
Authors: Yaxuan Zhu
Jianwen Xie
Ying Nian Wu
Ruiqi Gao
Venue: ICLR 2024 spotlight (2024)
Keywords: Energy-based model
recovery-likelihood
cooperative learning
Primary area: generative models
TL;DR: We propose cooperative diffusion recovery likelihood (CDRL) model. Our model substantially improves the generation performance of EBM-based generative models.

Abstract: Training energy-based models (EBMs) on high-dimensional data can be both challenging and time-consuming, and there exists a noticeable gap in sample quality between EBMs and other generative frameworks like GANs and diffusion models. To close this gap, inspired by the recent efforts of learning EBMs by maximimizing diffusion recovery likelihood (DRL), we propose cooperative diffusion recovery likelihood (CDRL), an effective approach to tractably learn and sample from a series of EBMs defined on increasingly noisy versons of a dataset, paired with an initializer model for each EBM. At each noise level, the two models are jointly estimated within a cooperative training framework: Samples from the initializer serve as starting points that are refined by a few MCMC sampling steps from the EBM. The EBM is then optimized by maximizing recovery likelihood, while the initializer model is optimized by learning from the difference between the refined samples and the initial samples. In addition, we made several practical designs for EBM training to further improve the sample quality. Combining these advances, we significantly boost the generation performance compared to existing EBM methods on CIFAR-10 and ImageNet 32x32. And we have shown that CDRL has great potential to largely reduce the sampling time. We also demonstrate the effectiveness of our models for several downstream tasks, including classifier-free guided generation, compositional generation, image inpainting and out-of-distribution detection.

Published as a conference paper at ICLR 2024
LEARNING ENERGY-BASED MODELS BY COOPERATIVE DIFFUSION RECOVERY LIKELIHOOD
Yaxuan Zhu UCLA yaxuanzhu@g.ucla.ddd
Jianwen Xie Akool Research jianwen @ucla.edu
Ying Nian Wu UCLA ywu@stat.ucla..dn
Ruiqi Gao Google DeepMind ruiqig google.oom
ABSTRACT
Training energy-based models (EBMs) on high-dimensional data can be both challenging and time-consuming, and there exists a noticeable gap in sample quality between EBMs and other generative frameworks like GANs and diffusion models. To close this gap, inspired by the recent efforts of learning EBMs by maximizing diffusion recovery likelihood (DRL), we propose cooperative diffusion recovery likelihood (CDRL), an effective approach tractably learn and sample from a series of EBMs defined on increasingly noisy versions of dataset, paired with an initializer model for each EBM. At each noise level, the two models are jointly estimated within cooperative training framework: samples from the initializer serve as starting points that are refined by a few MCMC sampling steps from the EBM. The EBM is then optimized by maximizing recovery likelihood, while the initializer model is optimized by learning from the difference between the refined samples and the initial samples. In addition, we made several practical designs for EBM training to further improve the sample quality. Combining these advances, our approach significantly boost the generation performance compared existing EBM methods on CIFAR-10 and ImageNet datasets. We also demonstrate the effectiveness of our models for several downstream tasks, including classifier- free guided generation, compositional generation, image inpainting and out-of- distribution detection.
INTRODUCTION
Energy-based -bsed models (EBMs), class of probabilistic generative models, have exhibited their
flexibility and practicality in a variety application scenarios, such as realistic image synthesis (Xie
et al. 2016 2018a] Nijkamp et al. 2019 Du & Mordatch 2019; Arbel et al. 2021 Hill et al. 2022
Xiao et al. 2021 Lee et al. 2023] Grathwohl et al.] 2021 Cui & Han 2023). graph generation (Liu
let al. (021). compositional generation (Du et al. 2020 2023), video generation (Xie et al.
202
3D generation (Xie et al./ 2021a 2018b), simulation based inference (Glaser et al.
2022). stochastic
optimization (Kong al. 2022), out-of-distribution detection (Grathwohl et al. 2020
Liu al.
2020), and latent space modeling (Pang et al.] (020) /huu et al. [203]] Zhang et al. 2023 Yu et al.
2023). Despite these successes of EBMs, training and sampling from EBMs remains challenging,
mainly due of the intractability of the partition function in the distribution.
powerful Recently, Diffusion Recovery Likelihood (DRL) (Gao et al. 2021 has emerged framework for estimating EBMs. Inspired by diffusion models (Sohl-Dickstein et al. 2015 Ho al., 2020 [Song & Ermon 2019). DRL assumes a sequence EBMs for the marginal distributions of samples perturbed by a Gaussian diffusion process, where each EBM is trained with recovery likelihood that maximizes the conditional probability of the data at the current noise level given their noisy versions at higher noise level. Compared to the regular likelihood, Maximizing recovery likelihood is more tractable, as sampling from the conditional distribution is much easier than sampling from the marginal distribution. DRL achieves exceptional generation performance among EBM-based generative models. However, a noticeable performance gap still exists between the sample quality of EBMs and other generative frameworks like GANs diffusion models. Moreover, DRL requires around 30 MCMC sampling steps at each noise level to generate valid samples, which can be time-consuming during both training and sampling processes.

---

Published as a conference paper at ICLR 2024
To further close the performance gap and expedite EBM training and sampling with fewer MCMC sampling steps, we introduce Cooperative Diffusion Recovery Likelihood (CDRL), that jointly estimates sequence of EBMs and MCMC initializers defined on data perturbed by diffusion process. At each noise level, the initializer and EBM are updated by a cooperative training scheme (Xie et al.) 2018a): The initializer model proposes initial samples by predicting the samples the current noise level given their noisy versions at higher noise level. The initial samples are then refined by few MCMC sampling steps from the conditional distribution defined by the EBM. Given the refined samples, the EBM is updated by maximizing recovery likelihood, and the initializer is updated absorb the difference between the initial samples and the refined samples. The introduced initializer models learn to accumulate the MCMC transitions of the EBMs, and reproduce them by direct ancestral sampling. Combining with new noise schedule and a variance reduction technique, we achieve significantly better performance than the existing methods of estimating EBMs. We further incorporate classifier-fre guidance (CFG) (Ho & Salimans 2022) to enhance the performance of conditional generation, and observe similar trade-offs between sample quality and sample diversity as CFG for diffusion models when adjusting the guidance strength. In addition, we showcase that our approach can be applied to perform several useful downstream tasks, including compositional generation, image inpainting and out-of-distribution detection.
Our main contributions are follows: (1) We propose cooperative diffusion recovery likelihood (CDRL.) that tractably and efficiently learns and samples from a sequence of EBMs and MCMC initializers; (2) We make several practical design choices related to noise scheduling, MCMC sampling, noise variance reduction for EBM training; (3) Empirically we demonstrate that CDRL achieves significant improvements on sample quality compared to existing EBM approaches, on CIFAR- 10 and ImageNet 32 x 32 datasets; (4) We show that CDRL has great potential to enable more efficient sampling with sampling adjustment techniques; (5) We demonstrate CDRL's ability in compositional generation, image inpainting and
(OOD) detection, as well as its compatibility with classifier-free guidance for conditional generation.
PRELIMINARIES ON ENERGY-BASER MODELS
Let Pdata (x) be a training example from underlying data distribution. An energy-based model defines the density of x by
1
(x = exp(Jo(x)),
(1)
Zo
where So is the unnormalized log density, negative energy, parametrized by neural network with a scalar output. Zg is the normalizing constant or partition function. The derivative of the log- liliikeeihodd function of an EBM can be approximately written as
i a0
C' E Paata
88 [x()
E Pu
fo(x)
where the second term is analytically intractable and has to be estimated by Monte Carlo samples from the current model pa- Therefore, applying gradient-based optimization for an EBM usually involves an inner loop of MCMC sampling, which can be time-consuming for high-dimensional data.
COOPERATIVE DIFFUSION RECOVERY LIKELIHOOD
3.1 DIFFUSION RECOVERY LIKELIHOOD
Given the difficulty of sampling from the marginal distribution p(x) defined by an EBM, we could instead estimate sequence of EBMs defined on increasingly noisy versions of the data and jointly estimate them by maximizing recovery likelihood. Specifically, assume a sequence of noisy training examples perturbed by a Gaussian diffusion process: Xo, Xi: Xr such that Xn Pdata' X 0++1 X 0 Denote yt = 0tt+1 X, for notation simplicity. The marginal distributions of {yr: 1,.... 7} are modeled by a sequence of EBMs: Po(y.) Zo,
I exp(fo(y::t)). Then the
conditional EBM of y, given the sample X at a higher noise level can be derived as
po mo(y|\x+1)) = (x.(()
exp (To(ysst)
lly x
(3)
20
2

---

Published as a conference paper at ICLR 2024
where Zo, (x+-1) is the partition function of the conditional EBM dependent Xt+1- Compared with the marginal EBM py().), when dx+1 is small, the extra quadratic term in pe (y constrains the conditional energy landscape to be localized around X making the latter less multi-modal and easier to sample from with MCMC. In the extreme case when 0++1 is infinitesimal, po ( is approximately Gaussian distribution that can he tractably sampled from and has close connection to diffusion models (Gao et al. 2021). In the other extreme case when 0++1 + 0o, the conditional distribution falls back to the marginal distribution, and we lose the advantage of being more MCMC friendly for the conditional distribution. Therefore, we need to maintain a small 0++1 between adjacent time steps, and to equip the model with the ability of generating new samples from white noises, we end up with estimating sequence of EBMs defined on the diffusion process. We use the variance-preserving noise schedule (Song et al. 202Tb), under which case we have x, = 0x + otE, where 0g I 0g and o, = -1-00 oz
We estimate each EBM by maximizing the following recovery log- likeliiheoodd function at each noise level (Bengio et al. 2013):
[(()) 1 Σ log po (Vr.
(4)
where [yr,s, x are pair of samples at time steps t and t + 1. Sampling from (o(y)|x/411 can be achieved by running K steps of Langevin dynamics from the initialization point yp X,+ and iterating
y = 97
2i 2
(r - _
(57 (87-xx+1) S15", 1++
(5)
where s, is the step size and T is the index of MCMC sampling step. With the samples, the updating of EBMs then follows the same learning gradients as MLE (Equation ( as the extra quadratic term
20 yy - x,+1 1| po (yyrXx++) does not involve learnable parameters. is worth noting that
maximizing recovery likelihood still guarantees unbiased estimator of the true parameters of the marginal distribution of the data.
3.2
AMORTIZING MCMC SAMPLING WITH INITIALIZER MODELS
Although pa (y: (x41) is easier to sample from than po (yt). when d++1 is not infinitesimal, the ini- tialization of MCMC sampling, X may still be far from the data manifold of y. This necessitates a certain amount of MCMC sampling steps each noise level (e.g., 30 steps of Langevin dynamics in Gao et al. ))02T)) ). Naively reducing the number of sampling steps would lead to training divergence or performance degradation.
To address this issue, we propose to learn an initializer model jointly with the EBM at each noise level, which maps X closer to the manifold of yt. Our work is inspired by the CoopNets (Xie ct al. 2018a 20216 2022b), which shows that jointly training a top-down generator via MCMC teaching will help the training of a single EBM model. We take this idea and generalize it to the recovery-likellhode model. More discussions are included in Appendix A Specifically, the initializer model at noise level / is defined as
(x(xxxx-1) ~N N(go((x+1)t)
(6)
serves as a coarse approximation to po xx+1), as the former is a single-mode Gaussian distribution while the latter can be multi-modal. A more general formulation would be to in- volve latent variables z, following a certain simple prior p(z,) into 9ø. Then ( tfxx+1) = Ep(z,) (y,, z,, t[xx+1)] can be non-Gaussian (Xiao al. 2022). However,
empirically find that the simple initializer in Equation 6 works well. Compared with the more general formulation, the simple initializer avoids the inference of z, which may again require MCMC sampling, and leads to more stable training. Different from Xiao et al. (2022). samples from the initializer just serves as the starting points and are refined by sampling from the EBM, instead of being treated the final
samples. We follow (Ho et al. 2020) to set ō, 1-4/+1 0g. If we treat the sequence of initializers
1-83
as the reverse process, such choice of o, corresponds to the lower bound of the standard deviation given by Pdata being delta function (Sohl-Dickstein et al. 2015).
3

---

Published as a conference paper at ICLR 2024
3.3 COOPERATIVE TRAINING
We jointly train the sequence of EBMs and intializers in a cooperative fashion. Specifically, at each iteration, for randomly sampled noise level I, obtain an initial sample ý, from the intializer model. Then a synthesized sample Yr from p(y: x1) is generated by initializing from yt and running few steps of Langevin dynamics (Equation [5). The parameters of EBM are then updated by maximizing the recovery log- lellikkli function (Equation 4 The learning gradient of EBM is
VoJr(0) = V
foly n
Σ fo(V
To train the intializer model that amortizes the MCMC sampling process, we treat the revised sample yr by the EBM as the observed data of the initializer model, and estimate the parameters of the initializer by maximizing log- logllikelhhoo::
(x(0)
1 T1
Σ
Fe, 20
g0 (xxx
(8)
That is, the initializer model learns to absorb the difference between ý, and ý, at each iteration so that Yr is getting closer to the samples from po(yx(Xe+1) In practice, we re-weight (o()) across different noise levels by removing the coefficient 28 similar to the "simple loss" in diffusion models.
The training algorithm is summarized in Algorithm [
After training, we generate new samples by starting from Gaussian white noise and progressively samples Pa (x,,) ) at decreasingly lower noise levels. For each noise level, an initial proposal is generated from the intializer model, followed by a few steps of Langevin dynamics from the EBM. See Algorithm 2 for a summary.
3.4 NOISE VARIANCE REDUCTION
We further propose a simple way reduce the variance of training gradients. In principle, the pair of x, (or y:) and x is generated by X, ~ [āxxo T) and X ~ NN Xx, o I). Alternatielyy, can fix the Gaussian white noise NO, I). and sample pair (x/ x/+1) by
x dx + o,e
x x + 0.1c..
In other words, both x and x/4, are linear interpolation between the clean sample xo and sampled white noise image e. x and X have the same marginal distributions as x; and X But x² is deterministic given Xg and x/ while there's still variance for x given Xu and X4+1- This schedule related to the ODE forward process used in flow matching (Lipman et al. 2022) and rectified flow (Liu al. 2022b).
3.5 CONDITIONAL GENERATION AND CLASSIFIER- GUIDANCE
Ho & Salimans 2022) introduced classifier free guidance, which greatly enhances the sample quality of conditional diffusion models and balances between sample quality and diversity by adjusting the strength of guidance. Given the close connection between EBMs and diffusion models, we show that is possible to apply classifier-free guidance in our CDRL as well. Specifically, suppose e context (e.g.. a label or text description). At each noise level we jointly estimate unconditional EBM pe (y:) x (x)( t)) and a conditional EBM pe (yrlc) x exp(ff(yee) c, t)). By following the Hassifier-free guidance (Ho & Salimans| 2022).
perform conditional sampling using the following linear combination of the conditional and unconditional gradients:
Vy log (y, (c) = (w + 1)V, (y:: wVy lo Po (y::t)
(10)
= (w + 1)V fo(yri,t) - wV, fo(y;t),,
(11)
where w controls the guidance strength. We adopt a single neural network to parameterize both condi- tional and unconditional models, where for the unconditional model we can simply input null token
4

---

Published as a conference paper at ICLR 2024
Algorithm 1 CDRL Training
Input: (1) observed Xo (xm, (2) Number of noise levels 7: (3) Number of Langevin sampling steps
K per noise level; (4) Langevin step size each noise level 8e; (5) Learning rate no for EBM fo; (6) Learning rate 76 for initializer gai
Output:
Parameters 0, db
Randomly initialize 0 and @b.
repeat
Sample noise level from {0, 1,.,,,--1111111 1}.
Sample N (0, f). Let X ax 04416, Yr (441 + a.c).
Generate the initial sample y, following Equation
Generate the refined sample y: by running K steps of Langevin dynamics starting from ýt following
Equation 5
Update EBM parameter 0 following the gradients in Equation
Update initializer parameter ø by maximizing Equation
until converged
Algorithm CDRL Sampling
Input: (1) Number of noise levels T; (2) Number of Langevin sampling steps K at each noise level; (3) Langevin step size at each noise level 6,; (4) Trained EBM fe: (5) Trained initializer
Output: Samples &
Randomly initialize
N(O,,),
for = 1- to do
Generate initial proposal y; following Equation Update y: to ys by K iterations of Equation
Let x = y./0++1.
end for
Ø for the class condition e when outputing the negative energy, i.e. foly;; foly Ø,
Similarly, for the initializer model, jointly estimate an unconditional model go (x++))+1)))))))))))>1))
((ga(X++ ;; f), of I) and conditional model (y(c,, xx+1) ((gg((x+111 (), I).. We param- eterize both models a single neural network. Since both models follow Gaussian distributions, the scaled conditional distribution with classifier-free guidance is still a Gaussian distribution (Dhariwal
& Nichol 2021):
(y(l,,,, Xx41) - ((w + (x+((x));) wg.((x+1;t),)))
(12)
3.6 COMPOSITIONALITY ENERGY-BASED MODEL
One attractive property of EBMs is compositionality: one can combine multiple EBMs conditioned on individual concepts, and re-normalize to create a new distribution conditioned on the intersection of those concepts. Specifically, considering two EBMs (x(c)c)) x exp(fe (x; c1)) and p=(x)c2) x exp(fe xx 02)) that are conditioned on two separate concepts 01 and C2 respectively, Du et al. (2020): Lee al. 2023) construct new EBM conditioned on both concepts po px(x 02)
exp x; 01 + [x )2)) based on the production of expert (Hinton 2002). Specifically, suppose
the two concepts and 62 are conditionally independent given the observed example Then
have
log (x|c1, 02)
log (2 log pa (x)) const.
= log pegpelci [x) + Po x) + log (x) + const.
(x + log 02) loo po (x) + const.,
where const. a constant term independent of x. The composition can be generalized to include an arbitrary number of concepts. Suppose have M conditionally independent concepts, then
M
lx I,..,
Σ
log (x((c)) (M - log (x const.
(13)
We can combine the compositional log-density (Equation 13 with classifier-free guidance (Equa-
tion 11) to further improve the alignment of generated samples with given concepts. The sampling
5

---

Published as a conference paper at ICLR 2024
gradient of the scaled log-density function is given by
vx (x L,,., M) Vx (w + log Lo (x(c) (Mw M 1) log const
M
= (w + 1)
Vx xxfo(x - (Mw + M 1)
(14)
(a) CIFAR-10
(b) ImageNet (32 32)
Figure I: Unconditional generated examples on CIFAR-10 10 and ImageNet (32 x 32) datasets.
0.0
0.5
1.0
3.0
a. Random samples
FID
0.0
0.5
1.0
3.0
Incapptonn Scor
W
b. Samples with fixed noisc
d. Inception Score vs W
Figure 2: Conditional generation on ImageNet (32 x 32) dataset with a classifier-free guidance. (a) Random image samples generated with different guided weights 0.0, 0.5, 1.0 and 3.0; (b) Samples generated with fixed noise under different guided weights. The class label is set to be the category of Siamese Cat. Sub-images presented at the same position depict samples with identical random noise and class label, differing only in their guided weights; (c) A curve of FID scores across different guided weights; (d) eurve Inception scores across different guided weights.
4 EXPERIMENTS
We evaluate the performance of our model across various scenarios. Specifically, Section 4. demonstrates the capacity of unconditional generation. Section 4.2 highlights the potential of our model to further optimize sampling efficiency. The focus shifts to conditional generation and classifier- free guidance in Section 4.3 Section 4.4 elucidates the power of our model in performing likelihood estimation and OOD detection, and Section (.55 compositional generation. Please refer to Appendix B for implementation details, Appendix C.I for image inpainting with our trained models, Appendix DD3 for comparing the sampling time between approach and other EBM

---

Published as a conference paper at ICLR 2024
Table 1: Comparison of FID scores for unconditional generation CIFAR-10.
Models
FID
Models
FID !
EBM based method
Other likelihood based method
NT-EBM Nijkamp et al.
2022
78.12
VAE (Kingma & Welling, 2014
78.41
LP-EBM Pang et al. 2020
70.15
PixelCNN (Salimans et al.
2017
65.93
Adaptive CE Xiao & Han
2022
65.01
PixelIQN (Ostrovskı el al.
2018
49.46
EBM-SR Nijjkmmp al. 2019
44.50
Residual Flow (Chen et al.
2019
47.37
JEM (Grathoohl al. 2020
38.40
Glow (Kingma & Dhariwal
2018
45,99
EBM-IG Du & Mordatch
2019
38.20
DC-VAL CC-V (Parmar et al.
2021
17.90
EBM-FCEE (ao 2020
37.30
GAN based method
CoopVAHBMI
b
36.20
CoopNets (Xie
8018a
33.61
WGAN-GP Guuraaani et al. 2017
36,40
Divergence Irangle [am et
2020
30.10
SN-GAN Mivato 20
21.70
VARA (Grathwohl el al |024
27.50
BigGAN (Brock et al. 2019
14,80
EBM-CD (Du et al. 2021
25.10
SyylGGAN--Dillugggnnn (Zhao
2020
5.79
GEBM (Arbel ét al. 2021
19.31
Diffusion-GANN (Xiao et 20.22022
2022
3.75
НАT-EBM (Hill et all 2022
19.30
StyleGAN2-ADA (Karras el al. 2020
2.92
CF-EBM hha et al.
202T
16.71
Score based and Diffusion method
CoopFlow (Xre et ul. 2022b
15,80
CLEL-base (Lee
202.3
15.27
NCSN (Sng & Ermon 2019
25.32
VAEBM (iiao
2021
12.16
NCSN-V2 (Song & Isrmon 2020)
10.87
DRL (Gao al. 2021
9.58
NCSN+ (Song et al. 2021b
2.20
CLEEL-large (Lec
2023
8.61
DDPM Distilatin (Luhman & Luhman 2021
9.36
EGC (Unsupervised) (Cuo al. 2023
5.36
DDPM++(VP. NLL) (Kum et at. 202T
(
3.45
DDPM (Ho et al. 2020
3.17
CDRL (Ours)
4.31
DDPM+KVI, Kim et al. 2021
2.47
CDRL-large (Ours)
3.68
models, Appendix for understanding the role of EBM and initializer in the generation process and Appendix D for the ablation study. We designate our approach as "CDRL" in the following sections.
Our experiments primarily involve three datasets: (i) CIFAR-10 (Krizhevsky & Hinton 2009)
comprises images from 10 categories, with 50k training samples and 10k test samples at a resolution of 32 x 32 pixels. We use its training set for evaluating model in the task of unconditional
generation. (ii) ImageNet (Deng et al.. 2009) contains approximately 1.28M images from 1000 categories. We use its training set for both conditional and unconditional generation, focusing downsampled version (32 x 32) of the dataset. (iii) CelebA (Liu et al. (0055) consists of around 200k human face images, each annotated with attributes. We downsample each image of the dataset the size of 64 x 64 pixels and utilize the resized dataset for compositionality and image inpainting tasks.
4.1 UNCONDITIONAL IMAGE GENERATION
We first showcase model's capabilities in unconditional image generation on CIFAR-10 and ImageNet datasets. The resolution of each image is 32 x 32 pixels. FID scores (Heusel et al. 2017) on these two datasets are reported in Tables D and 3 respectively, with generated examples displayed in Figure I We adopt the EBM architecture proposed in Gao et al. (202T). Additionally, we utilize a larger version called "CDRL-large" which incorporates twice as many channels in each layer. For the initializer network, we follow the structure of (Nichol & Dhariwal 2021p, utilizing a U-Net (Ronneberger al., 2015) but halving the number channels. Compared to Gao al. (202T), CDRL achieves significant improvements in FID scores. Furthermore, CDRL uses the same number of noise levels (6 in total) as DRL but requires only half the MCMC steps at each noise level, reducing it from 30 to 15. This substantial reduction in computational costs is noteworthy. With the large architecture, CDRL achieves a FID score of 3.68 on CIFAR- and 9.35 on ImageNet (32 x 32). These results, to the best of our knowledge, the state-of-the-arr among existing EBM frameworks and are competitive with other strong generative model classes such as GANs and diffusion models.
4.2 SAMPLING EFFICIENCY
Similar to the sampling acceleration techniques employed in the diffusion model (Song al.) 202
Liu et al.) 2022a Lu et al.) (2022), we foresee the development post-training techniques to further accelerate CDRL sampling. Although designing advanced MCMC sampling algorithm could
7

---

Published as a conference paper at ICLR 2024
be standalone project, we present straightforward yet effective sampling adjustment technique to demonstrate CDRL's potential in further reducing sampling time. Specifically, we propose to decrease the number of sampling steps while simultancously adjusting the MCMC sampling step size to be inversely proportional to the square root of the number of sampling steps. As shown Table 2 while we train CDRL with 15 MCMC steps each noise level, we reduce the number of MCMC steps to 8, 5, and 3 during the inference stage, without sacrificing much perceptual quality.
4.3 CONDITIONAL SYNTHESIS WITH CLASSIFIER GUIDANCE
We evaluate our model for conditional generation on the ImageNet32 dataset, employing classifier-free free guidance as outlined Section 3.5 Generation results for varying guided weights are displayed in Figure 2 As the value of w increases, the quality of samples improves, and the conditioned class features become more prominent, although diversity may decrease. This trend is also evident from the FID and Inception Score (Salimans et al.) 2016) curves shown in Figures and While the Inception Score consistently increases (improving quality), the FID metric first drops (improving quality) and then increases (worsening quality), obtaining the optimal value of 6.18 (lowest value) at a guidance weight of 0.7. Additional image generation results can be found in Appendix CC.2
Table 2: FID for CIFAR-10 with sampling adjust- Table 3: FID for ImageNet (32 x 32) uncondi- tional generation.
ment.
Number of noise level x Number of MCMC steps
Models
FID 4
Models
4
EBM-IG IG (Du Mordatch
2019
60.23
PixelCNN Salimans et al.
201
40,51
EBM-CD et 2021
32.48
DRL Gao al. 2021
6 x 30 180
9.58
CF-EBM 7hao
2021
26.31
CDRL
6 x
4.31
CLEL-base (Lee et a.
2025
22.16 16
CDRL (step 8)
6 8 48
4.58
DRL (Gao al
converge)
CDRL (step 5)
6 5 30
5.37
DDPMFFRVE Kim al.
2021
8.42
CDRL (step 3)
6 x 3 = 18
9.67
CDRL (Ours)
9.35
4.4 LIKELIOOOD ESTIMATION AND OUT-OF-DISTRIBUTION DETECTION
distinctive feature of the EBM its ability to model the unnormalized log-likelihood directly using the energy function. This capability enables it to perform tasks beyond generation. In this section, we first showcase the capability of the CDRL in estimating the density of a 2D checkerboard distribution. Experimental results are presented in Figure 3 where we illustrate observed samples, the fitted density, and the generated samples at each noise level, respectively. These results confirm CDRL's ability to accurately estimate log-likelihood while simultaneously generating valid samples.
Moreover, we demonstrate CDRL's utility in out-of-distribution (OOD) detection tasks. For this endeavor, employ the model trained CIFAR-IO detector and use the energy the lowest
Observation
Fitted Deasity
Generated
Samples
Figure 3: The results of density estimation using CDRL for 2D checkerboard distribution. The number of noise levels in the CDRL is set to be 5, Top: observed samples at each noise level. Middle: density fitted by CDRL each noise level. Bottom: generated samples at each noise level.
8

---

Published as a conference paper at ICLR 2024
noise level to serve the OOD prediction score. The AUROC score of CDRL model, with CIFAR-10 interpolation, CIFAR-100, and CelebA data as OOD samples, is provided in Table 4 CDRL achieves strong results in OOD detection comparing with the baseline approaches. More results can be found in Table g in the appendix.
4.5 COMPOSITIONALITY
To evaluate the compositionality of EBMs, we conduct experiments on CelebA (64 x 64) datasets with Male, Smile, and Young the three conditional concepts. We estimate EBMs conditional on each single concept separately, and assume sim- ple unconditional initializer models. Classifier-free guidance is adopted when conducting compositional gener-
Table 4: AUROC scores OOD detection using CDRL and other explicit density models on CIFAR-10
Cifur-10 interpolation
Cifar-100 100
CclebA
PixeICNN (Salinans et al.
2017
0.71
0.63
Kingma Dhanwal
NVAE (Vabda & Kaini 2000
0.64
0,56
0.68
EBM-IG Di Merdaleh1
2019
0.70
0.70
VAEBM Kiao cl 2021
0.70
0.62
0.77
ation (Equation[I) [4). Specifically, we treat images with certain attribute value as individual classes. We ran-
FBM CD 11 2021
0.65
0.83
CLEL-Base [Lec
0.72
0.77
CDRL (ours)
0.75
0.78
0.84
domly assign each image in a training
batch to class based the controlled attribute value. For example, an image with Male=True and Smile=True may be assigned class if the Male attribute is picked or class if the Smile attribute is picked. For the conditional network structure, we make EBM fo conditional on attributes and use an unconditional initializer model Je to propose the imitial distribution. We focus on showcasing the compositionality ability of EBM itself, although it is also possible to conditional initializer model similar to Section 3.5 Our results are displayed in Figure[4, with images generated at a guided weight of 3.0. More generation results with different guidance weights can be found in the Appendix C.2 Images generated with composed attributes following Equation [4]oo[[[[cooe contain features of both attributes, and increasing the guided weight makes the corresponding attribute more prominent. This demonstrates CDRL's ability and the effectiveness of Equation 14
Male
Smile
Young
X
X
Figure 4: Results of attribute- cerribue-ccomositiionn generation on CelebA (64 x 64) with guided weight
= 3. Left: generated samples under different attribute compositions. Right: control attributes (* ("V", "x" and s_" indicate "True", "False" and "No Control" respectively).
5 CONCLUSION AND FUTURE WORK
We propose CDRL, a novel energy-based generative learning framework employing cooperative diffusion recovery likelihood, which significantly enhances the generation performance of EBMs. We demonstrate that the CDRL excels in compositional generation, aut-of-dixtributionn detection, image inpainting, and compatibility with classifier-free guidance for conditional generation. One limitation is that a certain number of MCMC steps are still needed during generation. Additionally, we aim scale our model for high-resolution image generation in the future. Our work aims stimulate further research on developing EBMs as generative models. However, the prevalence of powerful generative models may give rise negative social consequences, such as deepfakes, misinformation. privacy breaches, and erosion of public trust, highlighting the need for effective preventive measures.

---

Published as a conference paper at ICLR 2024
REFERENCES
Michael Arbel, Liang Zhou, and Arthur Gretton. Generalized energy based models. In The 9th International Conference on Learning Representations (ICLR), 2021.
Yoshua Bengio, Li Yao, Guillaume Alain, and Pascal Vincent. Generalized denoising auto-encoders as generative models. In Advances in neural information processing systems (NIPS), 2013.
Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale GAN training for high fidelity natural image synthesis. In The 7th International Conference on Learning Representations (ICLR), 2019.
Tong Che, Ruixiang Zhang, Jascha Sohl-Dickstein, Hugo Larochelle, Liam Paull, Yuan Cao, and Yoshua Bengio. Your gan is secretly an energy-based model and you should use discriminator driven latent sampling. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
Ricky TQ Chen, Jens Behrmann, David Duvenaud, and Jörn-Henrik Jacobsen. Residual flows for invertible generative modeling. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
Jiali Cui and Tian Han. Learning energy-based model via dual-meme teaching. In Advances in Neural Information Processing Systems (NeurIPS), 2023.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In Proceeding of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), 2009.
Prafulla Dhariwal and Alexander Quinn Nichol. Diffusion models beat gans image synthesis. In Advances in Neural Information Processing Systems (NeurIPS), 2021.
Yilun Du and Igor Mordatch. Implicit generation and generalization in energy- -base models. arXiv preprint arXiv: 1903. 08689, 2019.
Yilun Du, Shuang Li, and Igor Mordatch. Compositional visual generation with energy based models. In Advances Neural Information Processing Systems (NeurIPS), 2020.
Yilun Du, Shuang Li, Joshua B. Tenenbaum, and Igor Mordatch. Improved contrastive divergence training of energy-based models. In Proceedings of the 38th International Conference on Machine Learning (ICML), 2021.
Yilun Du, Conor Durkan, Robin Strudel, Joshua B Tenenbaum, Sander Dieleman, Rob Fergus, Jascha Sohl-Dickstein, Arnaud Doucet, and Will Grathwohl. Reduce, reuse, recycle: Compositional generation with energy-based diffusion models and meme. In Proceedings of the 40th International Conference on Machine Learning (ICML), 2023.
Bradley Efron. Tweedie's formula and selection bias. Journal of the American Statisticul Association, 160(496) 2011.
Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
Ruiqi Gao, Yang Lu, Junpei Zhou, Song-Chun Zhu, and Ying Nian Wu. Learning generative convnets via multi-ggn grid modeling and sampling. In Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
Ruiqi Gao, Erik Nijkamp, Diederik P. Kingma, Zhen Xu, Andrew M Dai, and Ying Nian Wu. Flow contrastive estimation of energy-based models. In Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
Ruiqi Gao, Yang Song, Ben Poole, Ying Nian Wu, and Diederik P. Kingma. Learning energy- based models by diffusion recovery likelihood. The 9th International Conference on Learning Representations (ICIR), 2021.
10

---

Published as a conference paper at ICLR 2024
Pierre Glaser, Michael Arbel, Arnaud Doucet, and Arthur Gretton. Maximum likelihood learning of energy-based models for simulation- based inference. arXiv preprint arXiv:2210.14756, 2022.
Ian Goodfellow, Jean Pouget-Abadics Mehdi Mirza, Bing Xu, David Warde-Farke Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications of the ACM, 63(11))139-144, 2020.
Will Grathwohl, Kuan-Chieh Wang. Jörn- Jacobsen, David Duvenaud, Mohammad Norouzi, and Kevin Swersky. Your classifier is secretly an energy based model and you should treat it like one. In The 8th International Conference on Learning Representations (ICLR), 2020.
Will Sussman Grathwohl, Jacob Jin Kelly, Milad Hashemi, Mohammad Norouzi, Kevin Swersky, and David Duvenaud. No MCMC for me: Amortized sampling for fast and stable training of energy-based models. In The 9th International Conference on Learning Representations (ICLR), 2021.
Ishaan Guirajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron Courville. Improved training of wasserstein gans. In The 6th International Conference on Learning Representations (ICLR), 2017.
Qiushan Guo, Chuofan Ma, Yi Jiang, Zehuan Yuan, Yizhou Yu, and Ping Luo. Ege: Image generation and classification via diffusion energy-basee model. arXiv preprint arXiv:2304.0200 2023.
Tian Han, Erik Nijkamp, Xiaolin Fang. Mitch Hill, Song-Chun Zhu, and Ying Nian Wu. Divergence triangle for joint training of generator model, energy-based model, and inferential model. In Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2019.
Tian Han, Erik Nijkamp. Linqi Zhou, Bo Pang, Song-Chun Zhu, and Ying Nian Wu, Joint training of variational auto-encoder and latent energy-based model. In Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In Advances in Neural Information Processing Systems (NeurIPS), 2017.
Mitch Hill, Erik Nijkamp, Jonathan Mitchell, Bo Pang, and Song-Chun Zhu. Learning probabilistic models from generator latent spaces with hat ebm. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
Geoffrey E Hinton.
Training products of experts by minimizing contrastive divergence. Neural computation, 14(8):1771-180) 2002.
Geoffrey E. Hinton. A practical guide to training restricted boltzmann machines. In Neural Networks: Tricks of the Trade Second Edition, pp. 599-619. Springer, 2012.
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598. 2022.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
Long Jin, Justin Lazarow, and Zhuowen Tu. Introspective classification with convolutional nets. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. N. Vishwanathan, and Roman Garnett (eds.), Advances in Neural Information Processing Systems (NIPS), pp. 823-833, 2017.
Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Training generative adversarial networks with limited data. In Advances Neural Information Processing Systems (NeurIPS), 2020.
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
11

---

Published as a conference paper at ICLR 2024
Dongjun Kim, Seungjae Shin, Kyungwoo Song, Wanmo Kang, and Il-Chul Moon. Soft truncation: A universal training technique of score-bases diffusion model for high precision score estimation. In Proceedings of the 38th International Conference Machine Learning (ICML), 2021.
Taesup Kim and Yoshua Bengio. Deep directed generative models with energy-based probability estimation. arXiv preprint arXiv: 130,,0339, 2016.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In The 3rd International Conference on Learning Representations (ICLR), 2015.
Diederik Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible 1x1 convolutions. In Advances in Neural Information Processing Systems (NeurIPS), 2018.
Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In The 2nd International Conference on Learning Representations (ICLR), 2014.
Diederik P. Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. In Advances in neural information processing systems (NeurIPS), 2021.
Lingkai Kong, Jiaming Cui, Yuchen Zhuang, Rui Feng, B. Aditya Prakash, and Chao Zhang. End- to-end stochastic optimization with energy-based model. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. 2009.
Rithesh Kumar, Sherjil Ozair, Anirudh Goyal, Aaron Courville, and Yoshua Bengio. Maximum /901008508, 2019.
entropy generators for energy-based models. arXiv preprini
Justin Lazarow, Long Jin, and Zhuowen Tu. Introspective neural networks for generative modeling. In IEEE International Conference on Computer Vision (ICCV), pp. 2793-2802, 2017.
Yann LeCun, Sumit Chopra, Raia Hadsell, M Ranzato, and Fujie Huang. tutorial on energy-based learning. Predicting structured data, 100), 2006.
Hankook Lee, Jongheon Jeong, Sejun Park, and Jinwoo Shin. Guiding energy-based models via contrastive latent variables. In The I 1thh International Conference on Learning Representations (ICLR), 2023.
Kwonjoon Lee, Weijian Xu, Fan Fan, and Zhuowen Tu. Wasserstein introspective neural networks. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3702-3711, 2018.
Yaron Lipman, Ricky TQ Chen, Heli Ben- Den-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747, 2022.
Luping Liu, Yi Ren, Zhijie Lin, and Zhou Zhao. Pseudo numerical methods for diffusion models on manifolds. In The 10th International Conference on Learning Representations (ICLR), 2022a.
Meng Liu, Keqiang Yan, Bora Oztekin, and Shuiwang Ji. Graphebm: Molecular graph generation with energy-based models. In ICLR workshop on Energy-Based Model, 2021.
Weitang Liu, Xiaoyun Wang, John D. Owens, and Yixuan Li. Energy-based outto--distribution detection. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning generate and transfer data with rectified flow. arXiv preprint arXiv:2209.0300 030003, 2022b.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of the International Conference on Computer Vision (ICCV), 2015.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization, In The 7th International Conference Learning Representations (ICLR), 2019.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ODE solver for diffusion probabilistic model sampling in around 10 steps. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
12

---

Published as a conference paper at ICLR 2024
Eric Luhman and Troy Luhman. Knowledge distillation in iterative generative models for improved sampling speed. arXiv preprint arXiv:2101.022 023388, 2021.
Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks. In The 6th International Conference Learning Representations
(ICLR), 2018.
Jiquan Ngiam, Zhenghao Chen, Pang Wei Koh, and Andrew Ng. Learning deep energy models. In Proceedings of the 28th international conference on machine learning (ICML), 2011.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In Proceedings of the 38th International Conference on Machine Learning (ICML), 2021.
Erik Nijkamp. Mitch Hill, Song-Chin Chun Zhu, and Ying Nian Wu. Learning non-convergent non-
persistent short-run MCMC toward energy-based model. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
Erik Nijkamp, Ruiqi Gao, Pavel Sountsov, Srinivas Vasudevan, Bo Pang, Song-Chun Zhu, and Ying Nian Wu. Learning energy-based model with flow-based backbone by neural transport meme. In The 9th International Conference on Learning Representations (ICLR), 2022.
Georg Ostrovski, Will Dabney, and Rémi Munos. Autoregressive quantile networks for generative modeling. In Proceedings of the 35th International Conference on Machine Learning (ICML), 2018.
Bo Pang, Tian Han, Erik Nijkamp, Song- Cong-Chun Zhu, and Ying Nian Wu. Learning latent space energy-based prior model. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
Gaurav Parmar, Dacheng Li, Kwonjoon Lee, and Zhuowen Tu. Dual contradistinctive generative autoencoder. In Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with clip latents. arXiv preprint arXiv:2204.0612 06112,, 2022.
Herbert E Robbins. An empirical Bayes approuch to statistics. Springer, 1992.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Cosiuter-Assisted Intervention (MICCAI), 2015.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L. Denton, Seyed Kam- yar Seyed Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, Jonathan Ho, David J. Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusion models with deep language understanding. In Advances in Neural Information Processing Systems (NeurIPS). 2022.
Tim Salimans and Jonathan Ho. Should ebms model the energy or the score? In ICLR workshop on Energy-Baad Based Model, 2021.
Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. In The 10th International Conference on Learning Representations (ICLR), 2022.
Tim Salimans, Ian Goodfellow, Wojeiech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. In Advances in Neural Information Processing Systems (NIPS), 2016.
Tim Salimans, Andrej Karpathy, Xi Chen, and Diederik P. Kingma. Pixelenn++: Improving the pixel- enn with discretized logistic mixture likelihood and other modifications. In The Sth International Conference Learning Representations (ICLR), 2017.
13

---

Published as a conference paper at ICLR 2024
Jascha Bohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamics. In Proceedings of the 32nd International Conference Machine Learning (ICML), 2015.
Jiaming Song, Chenlin Meng. and Stefano Ermon. Denoising diffusion implicit models. The 9th International Conference Learning Representations (ICLR), 2021a.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. In Advances Neural Information Processing Systems (NeurIPS), 2019.
Yang Song and Stefano Ermon. Improved techniques for training score- score-based generative models. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
Yang Song, Jascha Sohl-Dickster Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In The 9th International Conference on Learning Representations (ICI.R, 2021b.
Arash Vahdat and Jan Kautz. NVAE: A deep hierarchical variational autoencoder. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
Zhisheng Xiao and Tian Han. Adaptive multi-stage density ratio estimation for learning latent space energy-based model. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
Zhisheng Xiao, Karsten Kreis, Jan Kautz, and Arash Vahdat. Vaebm: A symbiosis between varia- tional autoencoders and energy-based models. In The 9th International Conference on Learning Representations (ICLR), 2021.
Zhisheng Xiao, Karsten Kreis, and Arash Vahdat. Tackling the generative learning trilemma with denoising diffusion gans. In The 9th International Conference on Learning Representations (ICLR), 2022.
Jianwen Xie, Yang Lu, Song- Chun Zhu, and Ying Nian Wu. theory of generative convnet. In Proceedings of the 33nd International Conference on Machine Learning (ICML), 2016.
Jianwen Xie, Yang Lu, Ruiqi Gao, Song-Chun Chun Zhu, and Ying Nian Wu. Cooperative training of descriptor and generator networks. IEEE Transactions on Pattern Analysis and Machine intelligence (TPAMI), 42(1):27 2018a.
Jianwen Xie, Zilong Zheng, Ruiqi Gao, Wenguan Wang, Song-Chun Zhu, and Ying Nian Wu. Learning descriptor networks for 3d shape synthesis and analysis. In Proceeding of the IEEE Conference on Computer Vision and Pattern Recognitio (CVPR), 2018b.
Jianwen Xie, Yang Lu, Ruiqi Gao, Song-Chus Zhu, and Ying Nian Wu. Cooperative training of descriptor and generator networks. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 42(1))27-45, 2020.
Jianwen Xie, Yifei Xu, Zilong Zheng, Song-Chun Zhu, and Ying Nian Wu. Generative pointnet: Deep energy-based learning on unordered point sets for generation, reconstruction and classification. In Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021a.
Jianwen Xie, Zilong Zheng, and Ping Li. Learning energy-based model with variational auto-encoder as amortized sampler. Proceeding of the 35th AAAI Conference on Artificial Intelligence (AAAI), 2021b.
Jianwen Xie, Song-Chur Zhu, and Ying Nian Wu. Learning energy- neatial-emmporl generative convnets for dynamic patterns. IEEE Transactions Pattern Analysis and Machine Intelligence (TPAMI), 13(2))516-531 2021c
Jianwen Xie, Zilong Zheng, Xiaolin Fang. Song- Chun-Chun Zhu, and Ying Nian Wu. Cooperative training of fast thinking initializer and slow thinking solver for conditional learning. EEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 44(8):3957-3973, 2022a.
14

---

Published as a conference paper at ICLR 2024
Jianwen Xie, Yaxuan Zhu, Jun Li, and Ping Li. A tale of two flows: Cooperative learning of langevin flow and normalizing flow toward energy-basee based model. The 10th International Conference on Learning Representations (ICLR), 2022b.
Xuwang Yin, Shiying Li, and Gustavo K Rohde. Learning energy-based models with adversarial training, In The European Conference on Computer Vision (ECCV), 2022.
Sangwoong Yoon, Young- Uk Jin, Yung-Kyun Noh, and Frank C Park. Energy-based models for anomaly detection: A manifold diffusion recovery approach. arXiv preprint 1rXiv:2310.18667,
2023.
Peiyu Yu, Yaxuan Zhu, Sirui Xie, Xiaojian Shawn Ma, Ruiqi Gao, Song-Chun Zhu, and Ying Nian Wu. Learning energy-based prior model with diffusion Information Processing Systems (NeurIPS), 2023.
liffusion-amortized amortized meme. In Advances in Neural
Jing Zhang, Jianwen Xie, Nick Barnes, and Ping Li. An energy-based prior for generative saliency. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 15(11):13100-1111, 2023.
Shengyu Zhao, Zhijian Liu, Ji Lin, Jun-Yan Zhu, and Song Han. Differentiable augmentation for data-efficient GAN training. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
Yang Zhao, Jianwen Xie, and Ping Li. Learning energy-based generative models via coarse-to- fo-fine expanding and sampling. In The 9th International Conference Learning Representations (ICIR), 2021.
Song- Chun Zhu, Ying Nian Wu, and David Mumford. Filters, random fields and maximum entropy (frame): Towards unified theory for texture modeling. International Journal of Computer Vision (IJCV), 27:107-126, 1998.
Yaxuan Zhu, Jianwen Xie, and Ping Li. Likelihood-basee generative radiance field with latent space energy-based model for aw-awere disentangled image representation. In International Conference Artificial Intelligence and Statistics (AISTATS), 2023.
15

---

Published as a conference paper at ICLR 2024
APPENDICES
Related Work
16
Training Details
17
B Network Architectures
17
Hyperparameters)
18
B.3 Noise Schedule and Conditioning Input
19
Illustration of the CDRL Framework
19
More Experimental Results
19
Image Inpainting
19
C.2
Image Generation
21
Generating Resolttcn Images
C4
Out-of-Distributiion Detection
21
D
Model Analysis
22
D.I Ablation Study
22
D.2 Effects of Number of Noise Levels and Number Langevin Steps
23
D.3
Sampling Time
23
D.4
Analyzing the Effects of the Initializer and the EBM
24
D.5
Analyzing Learning Behavior
24
A RELATED WORK
Energy-Basse -Base Learning Energy-based models (EBMs) Zhu et al. 1998 LeCun et al. 2006 Ngiam
2012, Xie et al. 2016 define unnormalized probabilistic distributions and typically trained through maximum likelihood estimation. Methods such as contrastive divergence
al. 201T: Hinton
(Hinton 2002, Du al. 2021), persistent chain (Xie et al. 2016). replay buffer (Du & Mordatchl
2019) or short-run MCMC sampling (Nijkamp al.) 2019) approximate the analytically intractable learning gradient. To scale up and stabilize EBM training for high-fidelity data generation, strategies like multi-grid sampling (Gao et al., 2018). progressive training (Zhao et al. (02T). and diffusion (Goo et al. 202I) have been adopted. EBMs have also been connected to other models, such as
adversarial training (Arbel et al. 2021 Che al. 2020), variational autoencoders (Xiao et al.) 2021
contrastive guidance (Lee et al.
2023). ), introspective learning (Lazarow et al. 2017] Jin et al. 2017
Lee al. 2018). and noise contrastive estimation (Gao et al. (2220). To alleviate MCMC burden,
various methods have been proposed, including amortizing MCMC sampling with learned networks
(Kim Bengio
2016 Xie al. 2018a; Kumar al., 2019: Xiao et al.) 2021 Han et al. 2019
Grathwohl et al.
2021 Among them, cooperative networks (CoopNets) (Xie et al. 2018 jointly
train a top-down generator and an EBM via MCMC teaching. using the generator
fast initializer
for Langevin sampling. CoopNets variants have also been studied in Xie et al. (2021b) 2022b). Our
work improves the recovery likelihood learning algorithm of EBMs (Gao al. 2021) by learning fast MCMC initializer for EBM sampling, leveraging the cooperative learning scheme (Xie et al.) 2020). Compared to Xie et al. (2020) that applied cooperative training to an initializer and an EBM for the marginal distribution of the clean data, approach only requires learning conditional initializers and sampling from conditional EBMs, which are much more tractable than their marginal counterparts. It is worth noting that Xie et al. (2022a) have investigated conditional cooperative learning, wherein a conditional initializer and conditional EBM are trained through MCMC teaching. In contrast, CDRL trains and samples from sequence of conditional EBMs and
16

---

Published as a conference paper at ICLR 2024
conditional initializers on increasingly noisy versions of a dataset for denoising diffusion generation.
Denoising Diffusion Model Diffusion models, initially introduced by Sohl--icckttinn et al. (2015) and further developed in subsequent works such as Song & Ermon (2020) Ho et al. (2020), generate samples by progressively denoising them from a high noise level to clean data. These models have demonstrated significant success in generating high-quality samples from complex distributions,
owing to range of architectural and algorithmic innovations (Ho et al., 2020; Song et al., 202 la
Kim et al. 2021; Song et al. 2021b Dhariwal & Nichol, 2021] Karras et al. 2022 Ho & Salimans 2022). Notably, Dhariwal & Nichol
(2021) emphasize that the generative performance of diffusion
models can be enhanced with the aid of a classifier, while Ho & Salimans (2022) further demonstrate
that this guided scoring can be estimated by the differential scores of conditional model versus an unconditional model. Enhancements in sampling speed have been realized through distillation techniques (Salimans & Ho. 2022) and the development of fast SDE/ODE samplers (Song et al. 2021a| Karras et |l.||2222 Lu et al, 2022). Recent advancements (Rombach et al., [022] |Saharia
et al. 2022 Ramesh et al. 2022) have successfully applied conditional diffusion models to the task of text-to-image generation, achieving significant breakthroughs.
EBM shares close relationship with diffusion models, as both frameworks can provide score to guide the generation process, whether through Langevin dynamics or SDE/ODE solvers. As Salimans & Ho (021) discuss, the distinction between these models lies in their implementation approaches: EBMs model the log-likelihood directly, while diffusion models focus on the gradient of the log-likelihood. This distinction brings advantages EBMs, such their compatibility with advanced sampling techniques (Du et al.) (023), potential conversion into classifiers (Guo et al. 2023), and capability to detect abnormal samples through estimated likelihood (Grathwohl et al. 2020
Liu et al., 2020).
The primary focus of this work is to advance the development of EBMs. Our approach connects with diffusion models (Ho et al.) 2020; Xiao et al.) 2022) by training sequence of EBMs and MCMC initializers to reverse the diffusion process. In contrast to Ho et al. (2020), our framework employs more expressive conditional EBMs instead of normal distributions to represent the denoising distribution. Additionally, Xiao et al. (2022) also suggest multimodal distributions, trained by generative adversarial networks (Goodfellow et al. 2020 for the reverse process.
B TRAINING DETAILS
B.1 NETWORK ARCHITECTURES
We adopt the EBM architecture from (Gao et al. 2021), starting with a 3 x 3 convolution layer with 128 channels (The number of channels is doubled to 256 in the CDRL-large configuration). We use several downsample blocks for resolution adjustments, each containing multiple residual blocks. All downsampling blocks, except the last one, include a 2 x 2 average pooling layer. Spectral normalization is applied to all convolution layers for stability, while ReLU activation is applied to the final feature map. The energy output is obtained by summing the values over spatial and channel dimensions. The architectures of EBM building blocks are shown in Table 5 and the hyperparameters
of network architecture are displayed in Table 6
For the initializer network, we follow (Nichol & Dhariwal 2021) to utilize a U-Net (Ronneberger
al. 2015) while halving the number of channels. This reduction effectively decreases the size of the initializer model. For an image with a resolution of 32 x 32 pixels, we have feature map resolutions 32 x 32, 16 x 16, and map resolution of 64 x 64. All feature map channel numbers are set to 64, with attention applied to resolutions of 16 x 16 and x 8. Our initializer directly predicts the noised image ýe at each noise level f, while the DDPM in (Ho et al. 2020 predicts the total injected noise
When dealing with 64 64 images, we include an additional feature
For the condiiccndd generation task, we map class labels to one-ho hot vectors and use a fully- connected layer to map these vectors to class embedding vectors with the same dimensions time embedding vectors. The class embedding is then added to the time embedding. We set the time embedding dimension 512 for EBM and 256 for the initializer in the CDRL setting. In the
17

---

Published as a conference paper at ICLR 2024
CDRL-large setting, the time embedding dimension increases to 1024 for EBM, while the one in the initializer remains unchanged.
Table 5: Building blocks of the EBM in CDRL.
(a) ResBlock
Downsample Block
(c) Time Embedding
leakyReLJU, 3 x Conv2D
ResBlocks
Sinusoidal Embedding
+ Dense((eakyReeL((emb))
Downsample 2 x 2
Dense, leakyReLU
leakyRel.U, 3 x 3 Conv2D
Dense
+ Input
Table 6: Hyperparameters for EBM architectures in different settings.
# of Downsample Blocks
N (# of Resblocks in Downsample Block)
# of channels in each resolution
Model
CDRL (32 x 32)
(128, 256, 256, 256) (256, 512, 512, 512) (128, 256, 256, 256, 256) 8 (128, 256, 256, 256, 256)
CDRL-large (32 32)
4
8
Compositionality Exp.
5
2
Inpainting Exp.
5
B.2 HYPERPARAMETERS
We set the learning rate of EBM to be f/s = le-ee and the learning rate of initializer to be Vis = le- We use linear warm up for both EBM and initializer and let the initializer to start earlier than EBM. More specifically, given training iteration iter, we have:
iter
7e m1.0 10000
le 4
iter + 500
Me min(1.0,
1e
10000
(15)
We use the Adam optimizer Kingma& & Ba [2015: Loshchilov & Hutter (2019) to train both the EBM and the initializer, with 3 (0.9,0.999) 0.999) and weight decay equal to 0.O. We also apply exponential moving average with decay rate equal to 0.9999 to both the EBM and the initializer. Training is conducted across Nvidia A100 GPUs, typically requiring approximately 400k iterations, which spans approximately 6 days.
Following (Gao et al. 202I), we use a re- paramtterization trick to calculate the energy term. Our EBM is constructed across noise levels f = 0, 1, 2, 3, 4, 5 and we assume the distribution noise level f = 6 a simple Normal distribution during sampling. Given y, under noise level f, suppose we denote the
output of the EBM network as fy f), then the true energy term is given by fo t) fo(y..)) of where s, is the Langevin step size at noise level t. In other words, we parameterize the energy as the product of the EBM network output and a noise-level dependent coefficient, setting this coefficient equal to the square of the Langevin step size. We use 15 steps of Langevin updates at each noise level, with the Langevin step size noise level given by
0.054 5 of2
(16)
where 0/-1 is the variance of the added noise at noise level +++ and o. is the standard deviation the accumulative noise at noise level During the generation process, we begin by randomly sampling
X5 ~ (0, I) and perform denoising using both the initializer and the Langevin Dynamics of the EBM, which follows Algorithm [ After obtaining samples x the lowest noise level t 0, perform an additional denoising step, where we disable the noise term in the Langevin step, to further
enhance its quality. More specifically, follow Tweedie's 's formula (Efron 2011 Robbins) 1992).
18

---

Published as a conference paper at ICLR 2024
which states that given (X) and noisy version image x' with conditional distribution p(x((x) = (x, I), the marginal distribution can be defined as p(x') = Pdata (x)p(x dx) Consequently, we have
[[(xx*) = x' + V x log ((x).
(17)
In our case, we have p(xx((xxx) (ux of and we use EBM to model the marginal distribu- tion of x, as po,)(X;), thus
B(axxo|xx) x, V..ivx log po, (xx),
Vx
E(xo(x;) Xt o. log pa. (x+)
(18)
at
Suppose the samples we obtain at = 0 are denoted as Xg- These samples actually contains a small amount of noise corresponding to ckg, thus, we may use Equation [8] to further denoise them. In practice, we find that enlarging the denoising step by multiplying the gradient term Vx, log ps, (xx) by a coefficient larger than 1.0 yields better results. We set this coefficient to be 2.0 in our experiments.
B.3 NOISE SCHEDULE AND CONDITIONING INPUT
We improve upon the noise schedule and the conditioning input of DRL (Gao et al. 2021 Let A log o² represent the logarithm of signal-to-noise ratio at noise level t. Inspired by (Kingma et al. 2021), we utilize X as the conditioning input of the noise level and feed it to the networks fa and 9s instead of directly using t.
For the noise schedule, we keep the design of using 6 noise levels as DRL. Inspired by (Nichol Dhariwal 202T), we construct cosine schedule such that A is defied as X -2 (lanaal + b)), where a and are calculated from the maximum log SNR (denoted as Amax) and the minimum log
SNR (denoted as min) using
arctan(exp(-OA. 1ooospp-0..Ammaam 0.5Amaxx)) (00xpp(00mmmmm)) - b.
(19)
32
a =
(20)
We set Amax 9.8 and Amnn = -511 to correspond with the standard deviation ãg of the accumulative noise in the original Recovery Likelihood model (T6 setting) at the highest and lowest noise levels. Figure 5 illlustrases the noise schedule of DRL alongside our proposed schedule. In contrast to the DRL's original schedule, our proposed schedule places more emphasis on regions with lower signal-to to- noise ratios, which are vital for generating low-frequency, high-level concepts in samples.
Figure 5: Noise schedule. The green line represents the noise schedule used by DRL (Gao et al. 2021) while the red line depicts the noise schedule employed by our CDRL.
B.4 IIL.USTRATION OF THE CDRL FRAMEWORK
Figure 6 illustrates the training and sampling processes of the CDRL framework, providing a comprehensive overview of the model.
C MORE EXPERIMENTAL RESULTS
C. IMAGE INPAINTING
We demonstrate the inpainting ability of our learned model on the 64 x 64 CelebA dataset. Each image is masked, and our model is tasked with filling in the masked area. We gradually add noise to the masked image up to the final noise level, allowing the model to denoise the image progressively, similar to the standard generation process. During inpainting, only the masked area is updated, while the values in the unmasked area are retained. This is achieved by resetting the unmasked area values to the current noisy version after each Langevin update step of the EBM or initializer proposal step. Our results, depicted in Figure 7 include types of masking: regular square mask and an
19

---

Published as a conference paper at ICLR 2024
observed exampies
Notey yr
Update initiaa
generated examples
Update
Clean Data
Inittal proposal with unitializer
MCMC revision gulded EBM
Notey Data
Initial proposal
Upulated example yr
(a) CDRL training process. In the training phase, start by selecting pair of images {y., x at noise levels and f respectively. The image x, noise level is then fed into the initializer generate initial proposal yr. Subsequently, this initial proposal undergoes refinement through MCMC process guided by the underlying energy function. The refined sample y, obtained from this process utilized to update both the energy function and the initializer.
FRM
IBM
Finil
MCMC
Initial
updates
Initial noise
Inttialive
Inttaalerr
Instslleer
Infastieee
Instalturr
Teitekhor
(b) CDRL Sampling process. The sampling phase starts from Gaussian noisc. Starting from the highest noisc level, an initial proposal generated by the initializer that corresponds to that noise level. Subsequently, samples undergo refinement through MCMC sampling. This denoising process is iteratively repeated to push noisy image towards lower noise levels until the lowest noise level 0 is reached.
Figure Illustration of the Cooperative Diffusion Recovery Likelihood (CDRL) framework
irregularly shaped mask. Figure 7 the first two columns respectively display the original images and the masked images, while the other columns show the corresponding inpainting results. CDRL successfully inpaints valid and diverse values in the masked area, producing inpainted results that differ from the observations. This suggest that CDRL does not merely memorize data because it fills novel and meaningful content into unobserved areas based on the statistical features of the dataset.
Figure 7: Results of Image inpainting CelebA (64 x 64) dataset. The first two rows utilize square masks, while the last two rows use irregular masks. The first column displays the original images. The second column shows the masked images. Columns three to six display inpainted images using different initialization noises.
20

---

Published as a conference paper at ICLR 2024
C.2 IMAGE GENERATION
In this section we present additional generation results. Figure showcases more compositionality results with varying guidance weights the CelebA 64 x 64 dataset. Here, #: 0.0 corresponds the original setting without guidance in Equation T3 in the main paper. In Figure 11 12 13 14
T5] we provide more results for conditional generation on ImageNet32 (32 x 32) with different guidance weights. Specifically, Figure [ showcases random samples, while each figure Figures
13
and 15 contains samples from specific class under different guidance weights w.
C.3
GENERATING HIGII-RESOLUTION IMAGES
The recent trend in generative modeling of high-resolution images involves either utilizing the latent space of a VAE, as demonstrated latent diffusion (Rombach et al., (022), or initially generating low-rcsolution image and then gradually expanding it, exemplified by techniques like Imagen (Saharia et al. 2022). This process often reduce the modeled space to dimensions such as 32 x 32 64 64, which aligns with the resolutions that used in experiments in the main text. Here, we conduct additional experiments by learning CDRL following RRobbach et al. (2022). We conduct experiments on the CelebA-HQ dataset, and the generated samples are shown in Figure g Additionally, we report the FIDs in Table 7
Table 7: Comparison of FIDs the CelebA-HQ (256 256) dataset
Model
FID score
GLOW Kingma Dhariwal 2018)
68.93
VAEBM (Xiao et al. 202T)
20.38
ATEBM Yin et al. 2022)
17.31
VOGAN+Iranstommn Transtormer (Esser et al.
2021
10.2
LDM Rombach et al.
2022)
5.11
(DRL((urs)
10.74
C.4 OUT-OF-DISTRIBUTION DETECTION
We present the results for the out-of-distributtion (OOD) detection task on additional datasets, along with incorporating recent baselines. The comprehensive results summarized in Table g
Table 8: Comparison AUROC scores in OOD detection on CIFAR-10 10 dataset. The AUROC score for DRL (Gao al., 2021) is reported by Yoon et al. (2023). For EBM-CD (Du et al. 2021) present two sets of performance results from different sources: one from Du et al. (2021) and the other from recent study YYon et al. 2023). We include both sets of results, with scores from Yoon et al. (202)) presented in brackets for comparison.
Cifar-10 interpolation
Cifar-100
CelebA
SVHN
Texture
PixeICNN Salimans et al. 2017
0.71
0.63
0.32
0.33
GLOW Kingma& & Dhariwal
2018
0.51
0.55
0.57
0.24
0.27
NVAE Vahdat Kautz 2020
0.64
0.56
0.68
0.42
EBM-IG Du & Mordatch
2019
0.70
0.50
0.70
0.63
0.48
VAEBM Xıao
0.70
0.62
0.77
0.83
EBM-CD Du
0.65 (-)
0.83 (0.53)
-0..54
0.91 (0.78)
0.88 (0.73)
CLEL Lee et al.
0.72
0.77
0.98
0.94
DRL Gao el al.
2021
0.44
0.64
0.88
0.45
MPDR-S/ Yoon Yoon et al.
2023
0.56
0.73
0,99
0.66
MPDR R Yoon et al.
2023
0.64
0.83
0.98
0.80
CDRL
0.75
0.78
0.84
0.82
0.65
21

---

Published as a conference paper at ICLR 2024
Figure 8: Samples generated by CDRL model trained on the CelebAHQ (256 x 256) dataset.
D MODEL ANALYSIS
In this section, we employ several experiments to analyze the CDRL model.
D.1 ABLATION STUDY
In this section, we conduct an ablation study to analyze the effectiveness each component of our CDRL model. We have previously described three main techniques in our main paper that contribute significantly to our CDRL model: (1) the new noise schedule design, (2) the cooperative training algorithm, and (3) noise variance reduction. We demonstrate the impact of each of these techniques by comparing our CDRL model with the following models:
1. The original diffusion recovery likelihood (DRL) model (Gao et al. 2021 as baseline.
2. A model trained without using the cooperative training. This corresponds to the DRL but using the same noise schedule and conditioning input as CDRL.
3. CDRL without using noise reduction.
4. Similar to Xiao transform it to y,. Note that our CDRL uses the initializer to directly predict ý..
2002)), we use the initializer to predict the clean image x and then
5. Similar Ho et al. 2002)) we use initializer to directly output the prediction of total added noise and then transform to ý..
6. Compared with the noise schedule used in the original D et al. 2021 paper, the proposed used in CDRL places more emphasis the high-noise area where is close to O. We train a CDRL model with the original DRL noise schedule but with additional noise levels in the high-noise region for comparison.
22

---

Published as a conference paper at ICLR 2024
7. As depicted in Equation 8 in the main paper, our cooperative training algorithm involves the initializer learning from the revised sample yı at each step. natural question arises: should we instead regress it directly on the data y To answer this, we train model, in which the initializer directly learns from Yt at each step.
We ensure that all models share the same network structure and training settings the CIFAR-10 10 dataset and differ only in the aforementioned ways. As shown in Table our full model performs the best among these settings, which justifies design choices.
Table 9: Ablation study on the CIFAR-10 dataset.
Models
FID
DRL (Gao et al. 2021)
9.58
CDRL without cooperative training
6.47
CDRL without noise reduction
5.51
CDRL with an initializer that predicts Xo
5.17
CDRL with an initializer that predicts ē
4.95
CDRL using noise schedule in DRL-T8
4.94
CDRL with an initializer that learns from Yr
5.95
CDRL (full)
4.31
D.2 EFFECTS OF NUMBER OF NOISE LEVELS AND NUMBER OF LANGEVIN STEPS
We test whether the noise level can be further reduced. The results in Table 10a show that further reducing noise level to 4 can make model more unstable, even if we increase the number of the Langevin sample steps K. On the other hand, reducing T to 5 yields reasonable but slightly worse results. In Table 10b show the effect of changing the number of Langevin steps K. The results show that, on one hand, decreasing K to 10 yields comparable but slightly worse results. On the other hand, increasing K to 30 doesn't lead better results. This observation aligns wit the finding from Gao et al. (222)) The observation of changing K implies that simply increasing the number of Langevin steps doesn't significantly enhance sample quality, thereby verifying the effectiveness of the initializer in our model.
Table 10: Comparison of CDRL models with varying numbers of noise levels T and varying numbers of Langevin steps K. FIDs are reported on the Cifar- 10 dataset.
(a) Results of CDRL models with varying T
(b) Results of CDRL models with varying K
Model
FID
Model
FID
T 4 (K 15, 20)
not converge
T 6 (K =10)
4.50
T = 5 (K = 15)
5.08
T = 6 (K =15)
4.31
T 6 (K = 15) 4.31
T 6(K =30)
5.08
D.3 SAMPLING TIME
In this section, we measure the sampling time of CDRL, and compare it with the following models:
(1) CoopFlow (Xie et al., 2022b), which combines an EBM with a Normalizing Flow model; (2) VAEBM (iiao et al. 2021] which combines a VAE with an EBM and achieves strong generation performance; (3) DRL Gao al. 2021) model with 30-step MCMC sampling at each noise level. We conduct the sampling process of each model individually on a single A6000 GPU to generate a batch of 100 samples on the Cifar10 dataset. Our CDRL model produces samples with better quality with a relatively shorter time frame. Additionally, with the sampling adjustment techniques, the sampling time can be further reduced without significantly compromising sampling quality.
23

---

Published as a conference paper at ICLR 2024
Table 11: Comparison of different EBMs in terms of sampling time and number of MCMC steps. The sampling time are measured in seconds.
Method
Number of MCMC steps
Sampling Time
FID |
CoopFlow (Xie et al.
2022b
30
2.5
15.80
VAEBM (Xaoo al.
2021
21.3
12.16
DRL Gao et al.
2021
6 30 180
23.9
9.58
CDRL
6 15 90
12.2
4.31
CDRL (8 steps)
6 x 8 = 48
6.5
4.58
CDRL (5 steps)
= 30
4.2
5.37
CDRL (3 steps)
6 3 = 18
2.6
9.67
D.4 ANALYZING THE EFFECTS OF THE INITIALIZER AND THE EBM
To gain deeper insights into the roles of the initializer and the EBM in the CDRL in image generation, we conduct two additional experiments using pretrained CDRL model on the ImageNet Dataset (32 32). We evaluate two generation options: (a) images generated using only the initializer's proposal, without the EBM's Langevin Dynamics at each noise level, and (b) images generated with the full CDRL model, which includes the initializer's proposal and 15-step Langevin updates at each noise level. As shown in Figure 9a and 9b the initializer captures the rough outline of the object, while the Langevin updates by the EBM improve details of the object. Furthermore, in Figure [9c] we display samples generated by fixing the initial noise image and sample noise of each initializer proposal step. The outcomes demonstrate that images generated with the same initialization noises share basic elements but differ in details, highlighting the impact of both the initializer and the Langevin sampling. The initializer provides a starting point, while the Langevin sampling process enriches details.
(a) Initializer only
(b) Full CDRL model
(c) Fixing the initialization noise
Figure 9: Illustration of the effects of the initializer and the EBM on the image generation process using CDRL. model pretrained on the ImageNet Dataset (32 x 32). (a) Samples generated using only the proposal of the initializer; (b) Samples generated by the full CDRL model; (c) Samples generated by fixing the initial noise image and the sample noise of each initialization proposal step. Each row of images shared the same initial noise image and the sample noise of each initialization proposal step, but differed in the noises of Langevin sampling process each noise level.
D.5 ANALYZING LEARNING BEHAVIOR
We dive into deeper understanding of the learning behavior of the cooperative learning algorithm. We follow the analysis framework of Xie et al. 2200 2022a Let Ke(y,lyt', Xx+1) the transition kernel of the K-step Langevin sampling that refines the initial output y,' to the refined output yr. Let (Kegs)(y Xx+1) = f K₂ (yzly/, X )9 (y'l xx++ ddy be the conditional disribution yv. which obtained by K steps of Langevin sampling starting from the output of the initialier qa (x+1). Let x (y,, xx+1) be the true conditional distribution for denoising x to retrieve y.. The maximum recovery likelihood for the EBM in Equation in the main paper equivalent minimizing the
24

---

Published as a conference paper at ICLR 2024
Kullback- Leibler divergence (KL) divergence (L( (144)))llg((/ll/)) Using to index the learning iteration for model parameters, given the current initializer model (y [x41),, 11, the EBM updates its parameters 0 by minimizing
0++1 arg min (yxxy1y
- (o
(21)
which is modified contrastive divergence. It is worth noting that, in the original contrastive divergence, the above (Ka, (yr((x))) ) is replaced by (Ka, (x((())))). That is, the MCMC chains are initialized by the true data. The learning shifts po(ys|xe+)) toward the true distribution т (v)])++1).
On the other hand, given the current EBM, the initializer model learns from the output distribution of the EBM's MCMC. That is, we train the initializer with y in equation 8 of the main paper. The update of the parameters of the initializer at learning iteration j approximately follows the gradient of
Φ++1 arg min (L(Kogo, (x
(22)
The initializer Is (Vr which is conditional top-down generator, learns to be the sta- tionary distribution of the MCMC transition Ko (x(x))+++1) by adjusting its mapping towards the low-energy regions of po(xx(Nr+1). In a limit, the initializer go (o(y((xe+1) minimizes and approach the conditional EBM (9(x)(N41)) The entire learning algorithm can be viewed as a chasing game, where the initialzier model go (y Xx+1) chases the EBM po ( (x+11) in pursuit of the true conditional distribution myy:(x++1).
KL[(Kas, (Ye (1x41))llo0yy)xxx...
According to the above learning behavior, we can now discuss the benefits of the cooperative learning compared to directly training the initializer with observed y.
Firstly. presented in Equation 22 the MCMC process of the EBM drives the evolution of the initializer, which seeks to amortize the MCMC. At each learning iteration, in order to provide good initial examples for the current EBM's MCMC, the initialzer needs be sufficiently close to the EBM. Therefore, training the initialzer with the MCMC outputs y is a beneficial strategy to maintain a appropriate distance between EBM and initialzer model. Conversely, if the initializer directly learns from the true distribution, even though it may move toward the true distribution quickly, it might not provide a good starting point for the MCMC. A competent initializer should assist in identifying the modes of the EBM. Consider a scenario in which the initizlier model initially shifts toward the true distribution by learning directly from y, but the EBM remains distant. Due to a large divergence between the EBM and the initialzier, the latter may not effectively assist the EBM in generating fair samples, especially with finite-step Lanegevin dynamics. A distant initializer could lead to unstable training of the EBM.
Secondly, let us consider a more general senario where our initializer is modeled using a non-Gaussian (x 2, l), where z ~ (0, I) introduces randomness through the latent vector z. In this case (V: Xx+1) = ( z p)((z) z dz is analytically intractable. Learning qo (x+11)))) directly from y independently requires MCMC inference for the posterior distribution qo (z) Xx+1 Nt). However, cooperative learning circumvents the challenge of inferring the latent variables z. That is, at each learning iteration, generate examples ý from ( (x++1) 1) by first sampling à ~ p(z) and then mapping it to y, I (X++1, 2, t). The ý is used to initialze the EBM's MCMC that produces y. The learning equation of Φ is 4 E z0.= [1,,, - (X 1, 2, I)1?,, where the latent variables Z is used. That is, we shift the mapping from 2 + ý to 2 + ý for accumulating the MCMC transition. Although our paper currently employs a Gaussian initializer, if we adopt more expressive non-Gaussian initializer in the future, the current cooperative learning strategy (i.c., training % with y) can be much more beneficial and feasible.
generator
25

---

Published as a conference paper at ICLR 2024
Male
Smile
Youry
X
X
X
aa)
0.0
Made
Smile
Young
X
x
(b) 0.5
Male
Smile
Young
X
X
(c) 1.0
Figure 10: Attribute compositional samples generated by CDRL models trained on the CelebA (64 64) dataset. We utilize guided weights 0.0, 0.5, 1.0. Images at different guidance share the same random noise. Results can also be compared with those in Figure 4 which use = 3.0.
26

---

Published as a conference paper at ICLR 2024
(a) 0.0
(b) 0.5
(c) w 1.0
(d) 1.5
(c) w 2.0
(f) 3.0
Figure 11: Conditional generated examples with various classifier-free guidance weights the ImageNet32 (32 x 32) dataset. Samples are generated using randomly selected class labels.
27

---

Published as a conference paper at ICLR 2024
(a) 0.0
(b) 0.5
(c) 1.0
(d) 1.5
(c) w 2.0
(f) w 3.0
Figure 12: Conditional generated examples with different classifier-free guidance weights on the ImageNet32 (32 x 32) dataset, using the class label "Tench".
28

---

Published as a conference paper at ICLR 2024
(a) 0.0
(b) 0.5
(c) w 1.0
(d) 1.5
(c) w 2.0
(f) 3.0
Figure 13: Conditional generated examples with different classifier-free free guidance weights on the ImageNet32 (32 x 32) dataset, using the class label "Siberian Husky".
29

---

Published as a conference paper at ICLR 2024
(a) 0.0
(h) 0.5
(c) 1.0
(d) 1.5
(c) w 2.0
(f) 3.0
Figure 14: Conditional generated examples with different classifier-free guidance weights the ImageNet32 (32 x 32) dataset, using the class label "Tow Truck".
30

---

Published as a conference paper at ICLR 2024
(a) 0.0
(b) 0.5
(c) 1.0
(d) 1.5
(c) w 2.0
3.0
Figure 15; Conditional generated examples with different classifier-free guidance weights the ImageNet32 (32 x 32) dataset, using the class label "Volcano".
31


Reviews:
[{"review_id": "JfXRWyvHBb", "reviewer": "Reviewer_9dWK", "summary": "This paper proposes a novel training algorithm for jointly training an energy-based model (EBM) and an initializer model. The initializer has a very similar form to DDPM, so the proposed method can be viewed as a cooperation of EBM and DDPM.", "questions": "See weaknesses.\n\n===\nThe authors have addressed my questions in detail.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* The overall exposition is clear to follow.\n* The proposed algorithm, CDRL, shows a clear improvement over Diffusion Recovery Likelihood (DRL), an algorithm that CDRL is based on. The sample quality (in FID) is improved while the required computation (in MCMC steps) is reduced.\n* CDRL demonstrates broad applicability over multiple tasks outside image generation, such as compositional generation and out-of-distribution detection.", "weaknesses": "* The empirical performance of CDRL is good but not very strong. It is clear that CDRL is an improvement over DRL, but it still falls behind other models in multiple tasks.\n    * The initializer is very similar to DDPM, but CDRL's unconditional CIFAR-10 FID is worse than DDPM (Table 1).\n    * Also in Table 3, CDRL is outperformed by DDPM++.\n* In the out-of-detection distribution (OOD) experiment (Table 4), OOD detection capability is evaluated on only three test datasets, and the number of test OOD datasets is too small. It is important to use diverse test OOD datasets in evaluation because an OOD detector needs to detect any possible outliers. The current practice typically uses at least five OOD datasets that have different visual characteristics. Also, SVHN is known to be a highly challenging OOD dataset, particularly for generative models [1], but not included in Table 4. It would be great to see CDRL is able to detect SVHN as OOD.\n* It would be nice if the authors could comment on the consistency of the model after adding the initializer model. Would it alter the consistency?\n\n[1] Nalisnick et al., Do Deep Generative Models Know What They Don't Know?, https://arxiv.org/abs/1810.09136", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CHO2cPQKW1", "reviewer": "Reviewer_cEiK", "summary": "This paper proposes several techniques for improved training of diffusion recovery likelihood models, such as\n- learning an initializer model for MCMC,\n- noise variance reduction for reducing gradient variance.\n\nThis paper shows application of the proposed model to\n- conditional generation and classifier-free guidance,\n- compositional generation,\n- OOD detection.", "questions": "- I am curious about the authors' opinion on the practical benefits of CDRL vs. diffusion. Specifically, tasks that can be achieved with CDRL can also be achieved by diffusion, and vice versa, using the relation that the gradient of the energy is the score. I also observe that hyper-parameter choices in this paper are heavily influenced by those of diffusion models. Moreover, while CDRL uses fewer noise levels than diffusion models (e.g., 1000 levels), recent works on fast diffusion sampling have reduced sampling time for diffusion significantly (e.g., 2.87 FID on CIFAR10 with NFE=20 [1]). I think this naturally leads us to wonder what are the strengths of CDRL compared to diffusion.\n\n[1] DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps, Lu et al., NeurIPS, 2022.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is clearly written and easy to read.\n- CDRL shows clear performance improvements over previous EBM-based generative models.\n- CDRL can be applied to tasks such as unconditional generation, conditional synthesis, likelihood estimation, OOD detection, composition, etc. Such tasks were unexplored in the original DRL paper [1].\n\n[1] Learning Energy-Based Models by Diffusion Recovery Likelihood, Gao et al., ICLR, 2021.", "weaknesses": "I am inclined to give the score \"marginal accept\" for the following reasons.\n- The proposed method is a straightforward combination of two known techniques, DRL [1] and MCMC amortization [2,3]. While the simplicity of the idea is practically appealing, the idea lacks theoretical novelty. Moreover, the tasks demonstrated in the paper are already well-explored in the diffusion model literature, and the algorithms for the tasks are straightforward extensions of diffusion-based ones, as the score is just the gradient of the energy. For instance, classifier-free guidance is explored in [4], and compositional generation is explored in [5].\n- The paper lacks a comparison of inference time for CDRL and the baselines.\n- The paper lacks a comparison of training cost (e.g., required VRAM) for CDRL and the baselines. I expect CDRL training is more expensive than diffusion or DRL training, as the former requires two networks while latter only one.\n- The paper lacks results on higher resolution (e.g., 256x256) images.\n\n[1] Learning Energy-Based Models by Diffusion Recovery Likelihood, Gao et al., ICLR, 2021.\n\n[2] Approximate Inference with Amortized MCMC, Li et al., 2017.\n\n[3] Learning Energy-Based Prior Model with Diffusion-Amortized MCMC, Yu et al., NeurIPS, 2023.\n\n[4] Classifier-Free Diffusion Guidance, Ho et al., 2022.\n\n[5] Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMC, Du et al., ICML, 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PFJWe3iZ09", "reviewer": "Reviewer_nfgg", "summary": "This paper proposes the Cooperative Diffusion Recovery Likelihood (CDRL) for training energy-based models. Compared with the baseline DRL, CDRL introduces an extra initializer model that is jointly trained with the EBM along the diffusion process. The required MCMC steps are reduced thanks to a better initial sample.  \nThe experiments show that CDRL has significantly improved over the baseline DRL method. \n\nBesides the proposed method, the paper also discusses some other aspects of EBMs such as noise scheduler designs, classifier-free guidance, etc.", "questions": "The performance gain, regardless of efficiency, seems to come from a better initial state provided by the cooperatively trained initializer. In this case, I wonder about the limit of the baseline CRL, if we increase the MCMC sampling steps significantly, say 300, what would the performance be?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper is well-organized and easy to follow. \nI think the highlight of this paper is the strong empirical performance of CDRL. The work shows the possibility of achieving an FID on image benchmarks at least comparable to other generative models such as GANs and diffusion models. \nThe ablation studies are convincing as validation of each of the components in the proposed method.\n\nBesides, the introduced classifier-free guidance for EBM is also empirically sound, which verifies many intuitions from diffusion models.", "weaknesses": "My main concern is the lack of novelty. \nThe key components are inspired by other works. The DRL is well-established in training EBMs and the idea of a trainable initializer for MCMC is also not new. The combination of them seems a bit hacky to me and the overall method is a little cumbersome.\nIt would be better if the motivation and necessity of introducing an initializer (because initializers bring additional costs) were explained more clearly with more new insights to DRL. \n\nNonetheless, making this idea work empirically is impressive and the generative performance of EBMs has been greatly improved. \nHowever, the image generation experiment is on very low resolutions. I think it would be more convincing if the method could be shown to be able to handle image generation tasks for higher resolutions;", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ttChdjyUyq", "reviewer": "Reviewer_S2Sh", "summary": "The authors propose a new method for learning energy-based models (EBMs) mimicking diffusion models. Specifically, within the diffusion-model framework, EBMs are modified to parameterize the denoising process; to accelerate the MCMC sampling process when training the EBM, the authors also use a simultaneously trained diffusion-like model as an ``initialzer model.'' Experimental results demonstrate the effectiveness of the proposed method.", "questions": "1. What are the main contributions of the proposed CDRL when compared with DRL (Gao et al., 2021)?\n\n2. In the paragraph following Eq. (5), why \"maximizing recovery likelihood still guarantees an unbiased estimator of the true parameters of the marginal distribution of the data?\"\n\n3. Eqs. (6) and (8) are quite like the fomula of the conventional diffusion model. Please elaborate on the relationships between them.\n\n4. It seems that Eq. (9) indicates a deterministic noising process, right? If so, why \"there’s still variance for xt given x0 and xt+1?\"\n\n5. How to interpret Eq. (11), especially the $\\tilde p_{\\theta}$?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The presented techniques are likely new.", "weaknesses": "The clarity should be improved. For example, consider the relationships between the proposed method and related works.\n\nThe contribution is kind of incremental. Based on the Diffusion Recovery Likelihood (DRL) (Gao et al., 2021) in Sec. 3.1, the authors proposed in Sec. 3.2 a new \"initializer model\" to amortize the expensive MCMC sampling.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "o71P8ykLYg", "reviewer": "Reviewer_KQ36", "summary": "The authors study methods for training and sampling from energy-based models (EBMs). \n\nThey propose Cooperative Diffusion Recovery Likelihood (CDRL), which is an extension of Diffusion Recovery Likelihood (DRL). The proposed CDLR aims to improve the sample quality of DLR, while also reducing the number of MCMC steps required during training and sampling. CDLR entails jointly estimating a sequence of EBMs and MCMC initializers, utilizing the cooperative training approach. Essentially, the main proposed method is a combination of DLR (Gao et al., 2021) and cooperative training (Xie et al., 2020).\n\nThe proposed method is shown to improve the sample quality of DLR in unconditional generation on CIFAR10 and ImageNet 32x32. It is also applied to some conditional generation (via classifier-free guidance), compositional generation and OOD detection experiments.", "questions": "1. I think that the proposed CDLR method makes sense, and that it seems like a promising method for improving EBM training/sampling. However, it is not entirely clear to me how the resulting EBM differs from diffusion models? What are the main pros and cons compared to diffusion models? Why should one use this type of EBM instead of a diffusion model? When / in which applications should one use this type of EBM?\n\n2. Would it be possible to compare CDLR with the DLR baseline also in the OOD detection experiment (Table 4)?\n\n3. Would it perhaps be possible to illustrate / give a schematic overview of the proposed approach in some kind of figure? (a sequence of noise levels, one EBM and initializer model per level etc. I just think that this perhaps could help illustrate the general idea)\n\n4. The experiment in Figure 7 is interesting. Could you report FID scores for (a) and (b)? I.e., for K=0 and K=15 steps of Langevin refinement. Could this experiment perhaps also be repeated for K=1, 2, 4, 8 steps of Langevin refinement? \n\n\n\nMinor things:\n- Section 3.1, \"...constrains the conditional energy landscape to be localized around y_t\": Should y_t be x_{t+1}?\n- Abstract, \"noisy versons\": versons --> versions.\n- Section 1, \"The initializer model proposes initial samples by making prediction of the samples at the current noise level given\": \"making a prediction\"? \"making predictions\"? \"by predicting the samples...\"?\n- Section 3.2, \"which may again requires MCMC sampling\": requires --> require?\n- Algorithm 1: noise level --> noise levels, sampling step --> sampling steps.\n- Algorithm 2: sampling steps L --> sampling steps K? (also some inconsistency with this in the Appendix)\n- 4.4: CDLR strong --> CDLR achieves strong?\n- G.5: doesn't gives --> doesn't give (or, \"does not give\", I suppose)?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The paper is well written overall. I found it interesting to read. The authors definitely seem knowledgeable and familiar with previous work.\n\nThe main idea of the proposed method, combining DRL with cooperative training, makes intuitive sense and is described well overall.\n\nThe proposed method seems to improve the DLR baseline in experiments.", "weaknesses": "The related work section is placed in the Appendix, which seems quite odd. As a result, the previous work on cooperative training is sort of \"hidden\" in the main paper. This should be described before Section 3.3.\n\nThe similarities and differences between the proposed approach using EBMs and diffusion models could be discussed in more detail. The \"Diffusion Model\" paragraph in the related work section is interesting, but I think this could be expanded on a lot more.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (spotlight)