Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMC
Authors: Yilun Du
Conor Durkan
Robin Strudel
Joshua B. Tenenbaum
Sander Dieleman
Rob Fergus
Jascha Sohl-Dickstein
Arnaud Doucet
Will Sussman Grathwohl
Venue: Submitted to ICLR 2023 (2023)
Keywords: Generative models
diffusion models
compositional generation

Abstract: Since their introduction, diffusion models have quickly become the prevailing approach to generative modeling in many domains. They can be interpreted as learning the gradients of a time-varying sequence of log-probability density functions. This interpretation has motivated classifier-based and classifier-free guidance as methods for post-hoc control of diffusion models. In this work, we build upon these ideas using the score-based interpretation of diffusion models, and explore alternative ways to condition, modify, and reuse diffusion models for tasks involving compositional generation and guidance. In particular, we investigate why certain types of composition fail using current techniques and present a number of solutions. We conclude that the sampler (not the model) is responsible for this failure and propose new samplers, inspired by MCMC, which enable successful compositional generation. Further, we propose an energy-based parameterization of diffusion models which enables the use of new compositional operators and more sophisticated, Metropolis-corrected samplers. Intriguingly we find these samplers lead to notable improvements in compositional generation across a wide variety of problems such as classifier-guided ImageNet modeling and compositional text-to-image generation.

Under review as a conference paper at ICLR 2023
REDUCE, REUSE, RECYCLE: COMPOSITIONAL GENER- ATION WITH ENERGY-BASED DIFFUSION MODELS AND MCMC
Anonymous authors Paper under double-blind review
ABSTRACT
Since their introduction, diffusion models have quickly become the prevailing approach generative modeling in many domains. They be interpreted as learning the gradients of a time-varying sequence of log-probability density functions. This interpretation has motivated classifier-based and classifier-fre guidance as methods for post-hoc control of diffusion models. In this work, we build upon these ideas using the score-based interpretation of diffusion models, and explore alternative ways to condition, modify, and reuse diffusion models for tasks involving compositional generation and guidance. In particular, we investigate why certain types of composition fail using current techniques and present a number solutions. We conclude that the sampler (not the model) is responsible for this failure and propose new samplers, inspired by MCMC, which enable successful compositional generation. Further, we propose an energy-based parameterization of diffusion models which enables the use of new compositional operators and more sophisticated, Metropolis- corrected samplers. Intriguingly we find these samplers lead to notable improvements in compositional generation across a wide variety of problems such as classifier-guidee ImageNet modeling and compositional text-to-image generation.
INTRODUCTION
In recent years, tremendous progress has been made in generative modeling across variety of domains (Brown et al., 2020; Brock et al., 2018; Ho al., 2020). These models now serve as powerful priors for downstream applications such as code generation (Li et al., 2022), text-to-image generation (Saharia et al., 2022), question-annswring (Brown et al., 2020) and many more. However, to fit this complex data, generative models have grown inexorably larger (requiring 10's or even 100's of billions of parameters) (Kaplan et al., 2020) and require datasets containing non-negligible fractions of the entire internet, making it costly and difficult to train and or finetune such models. Despite this, some of the most compelling applications of large generative models do not rely on finetuning. For example, prompting (Brown al., 2020) has been successful strategy to selectively extract insights from large models. In this paper, we explore an alternative to finetuning and prompting, through which we may repurpose the underlying prior learned by generative models for downstream tasks.
Diffusion Models "Sohl-Dickstein et al., 2015; Song & Ermon, 2019; Ho et al., 2020) are recently popular approach to generative modeling which have demonstrated a favorable combination of scalability, sample quality, and ligllikllhhod.. A key feature of diffusion models is the ability for their sampling be "guided" after training. This involves combining the pre-trained Diffusion Model po():) with a predictive model po (ylx) to generate samples from (zrlg) This predictive model can be either explicitly defined (such as a pre-trained classifier) (Sohl-Dickstein et al., 2015; Dhariwal implicit predictive model defined through the combination of a conditional and
Nichol, 2021) unconditional generative model (Ho & Salimans, 2022). These forms of conditioning are particularly reuse pre-trained generative models for many
appealing (especially the former) they allow downstream applications, beyond those considered at training time.
These conditioning methods are form of model composition, i.e. combining probabilistic models together to create new models. Compositional models have a long history back to early work on Mixtures-Of Of-E--errt--tt- (Jacobs et al., 1991) and Product-Of-Fxpertts models (Hinton, 2002; Mayraz &

---

Under review as a conference paper at ICLR 2023
(4)
(x)
(b)
Pa(x)
Po (pizzaa))))
PoCxipizza)
Ps (x))
Ps(x)
Z
X
"The night AND
"The Night Sky"
uoberaur ridng bosse"
"An Araadnmast rilngg hore""
"A dog"
AA out"
"A "A
X
Figure Creating new models through composition. Simple operators enable diffusion models to be composed without retraining settings such (a) products, (b) classifier conditioning, (c) compositional text-to- image generation with a product (left) and a mixture (right). All samples generated by trained models.
Hinton, 2000). Here, many simple models or predictors were combined to increase their capacity. Much of this early work on model composition was done in the context of Energy-Basse Based Mod- els (Hinton, 2002), an alternative class of generative model which bears many similarities to diffusion models.
In this work, we explore the ways that diffusion models can be reused and composed with one- another. First, we introduce a set of methods which allow pre-trained diffusion models be composed, with one-another and with other models, to create new models without retraining. Second, we illustrate how existing methods for composing diffusion models are not fully correct, and propose remedy to these issues with MCMC-derived sampling. Next, propose the use of energy- based parameterization for diffusion models, where the unnormalized density of each reverse diffusion distribution is explicitly modeled. We illustrate how this parameterization enables both additional ways to compose diffusion models, as well as the use of more powerful Metropolis-adjuste MCMC samplers. Finally, demonstrate the effectiveness of approach in settings from 2D) data to high-resolution lext-to-image generation. An illustration of our domains can be found in Figure 1,
2 BACKGROUND
2.1 DIFFUSION MODELS
Diffusion models seek model a data distribution (=x)). We augment this distribution with auxiliary variables {x, ) defining Gaussian diffusion g( TT) g( ( [aro)... ... _g( ( where each transition defined g(r ((=))= Narx; vI- B, 3,T) for some 0 < S This transition first scales down _t-1 by /I Be and then adds Gaussian noise of variance 3, For large enough T. we will have ((TT) AJ (0,
Our model takes the form po (xt-) (a2z) and seeks to learn the reverse distribution of q(<x/<=-1) which seeks denoise Ve to ----1- the limit of small 33 this reversal becomes Gaussian (Sohl-Dickstein et al., 2015) so we parameterize our model Po (2+-1 (x)) = NN (o t), Bel) with:
1 no( Var (st Vi-n,
& -i (o(xx,t)
where co( t) is neural network, and ar Ci, Rr are functions of {3:)T-]-
useful feature of the diffusion process q that we can analytically derive any time marginal ((x)(2r)) = N(=; / of T) where again o, a function of {8))-- We can sample zt from this distribution using reparameterization, i.e (200 e) = V1- + o,e where e ~ NNO I). Exploiting this, diffusion models are typically trained with the loss
((0) -f (0).
(0) (((xo)) [lle (00 e).{)
(2)
Once ea (x t) trained, we recover se (z t) with Equation to parameterize pe po( (x) and perform ancestral sampling (also known as the reverse process) to reverse the diffusion, i. sample Ir ~ (O00 I). then for = + 1, sample 2,-1 ~ Po(22,111 lx,). A more detailed description can be found in Appendix B.
2

---

Under review as a conference paper at ICLR 2023
2.2 SHERGY-BASEL MODELS AND MCMC SAMPLING
Energy-Based Models (EBMs) are a class of probabilistic model which parameterize a distribution as (*)(
po(x) 2(8) where the normalizing constant 2(0) e is not modeled. Choosing not model this quantity gives the model much more flexibility but comes with considerable limitations.
We can no longer efficiently compute likelihoods or draw samples from the model. This complicates training. as most generative models are trained by maximizing likelihood.
One popular method for EBM training is denoising score matching. This approach minimizes the Fisher Divergence! between the model and a daussian-smoothee version of the data distribution la (a) = ( (x; a', a² T)de' by minimizing the following objective
(0) (e(c))(e [|I= + V ( oe)
When minimized, this ensures that efo(*) & 40 and therefore V Vzfo(z) V (o To estimate likelihoods or sample from our model, we must rely on approximate methods, such as MCMC sampling or numerical ODE integration. MCMC works by simulating a Markov chain beginning at zo ~ p(ro) and using a transition distribution 2t ~ k( If k(---) has certain properties, namely invariance w.r.t. the target and ergodicity, then as t + co, Iy converges to sample from our target distribution.
Perhaps the most popular MCMC sampling algorithm for EBMs is Unadjusted Langevin Dynamics (ULA) (Roberts & Tweedie, 1996; Du & Mordatch, 2019; Nijkamp et al., 2020) which is defined by
(=)(==1) = N [ + of (=
This resembles step of gradient ascent (with step-size 5) with added Gaussian noise of variance a². This transition based on discretization of the Langevin SDE. In the limit of infinitesimally small a this approach will draw exact samples. To handle the error accrued when using larger step sizes, a Metropolis correction can added giving the Metropolis-Adjusted Langevin-Algoree (MALA) (Besag, 1994). With Metropolis correction, we first generate proposed update at k(ze-118)
we set It = â,, otherwise I2 = It-1- k 11) then with probability min ('-toon11) 1(2
Hamiltonian Monte Carlo (HMC) (Duane al., 1987; Neal, 1996) is more advanced MCMC sampling method which augments the state-space with auxiliary momentum variables and numerically integrates energy-conservin Hamiltonian dynamics to advance the sampler. HMC typically applied with Metropolis correction, but an approximate variant can be used without it (U-HMC) (Geffner & Domke, 2021). See Appendix C.1 for details of HMC variants we use.
2.3 RELATIONSHIP BETWEEN DIFFUSION MODELS AND EBMS
Diffusion models and EBMs closely related. For instance, Song & Ermon (2019) uses an EBM perspective to propose a close cousin to diffusion models. We can see from inspection that the training objective of diffusion models is identical (up to a constant) to the denoising score matching objective
(0) (g(e))((eo [IIe or V. oze) (.)
where we have replaced eo (xx, 1) with -a, V..oo + o.e). Thus by training c0(x, 1) to minimize Equation 2, we can recover the diffused data distribution score with V, log (x) s s0(2t)) From this, we can define (a f) (z, t)) (the derivative of an explicitly defined scalar function) to learn a noise-conditional potential function fa (z, t). We later demonstrate the benefits of this two ways; it enables the use of more sophisticated sampling algorithms and more forms of composition.
2.4 CONTROLLABLE GENERATION
It may be convenient to train model of p(z) where x is, say, the distribution of all images, but in practice we often want to generate samples from p(zly) where y is some attribute, label, or feature. This can be accomplished within the framework of diffusion models by introducing learned
'The Fisher divergence is defined: F(pllg) = E.p [I/V, logp(z) Va log {+))
3

---

Under review as a conference paper at ICLR 2023
predictive model (y)la; t). i. con-ciondiii model of the distribution of some feature y given I. We can then exploit Bayes' rule to notice that (for À = 1),
(6) In practice, when using the right side of Equation for sampling, it is beneficial to increase the "guidance scale' À to be > (Dhariwal & Nichol, 2021). Thus, we can re-purpose the unconditional diffusion model and turn it into a conditional model.
V po t) = V. lo t) + AV log pa (yla;
If instead of classifier, we have both an unconditional diffusion model V. log (a; t) and conditional diffusion model V. log po (x we can again utilize Bayes' rule to derive an implicit predictive model's gradients
V log:o(u|2: t) Vz Porlu()) V log;)(
which can be used to replace the explicit model in Equation 6, giving what is known as classifier- free guidance (Ho Salimans, 2022). This method has led to incredible performance, but comes at a cost to modularity. This contrasts with the dlassifir--guddane a single (costly) generative model. We can then attach any predictive model we would like to for conditioning. This is beneficial as it is often much casier and cheaper to train predictive models than flexible generative model. In the classifier-free setting, we must know exactly which y we would like to condition on, and incorporate these labels into model training. In both guidance settings, our (possibly implicit) predictive model to modify the learned score of our model. We then perform diffusion model sampling as we would in the unconditional setting. We will see later that even in toy settings, this is often not the optimal thing to do.
setting, where we only need to train
3 COMPOSITIONAL GENERATION BEYOND GUIDANCE
Most work conditional diffusion models has come in the form of classifier or classifier-fre free guidance, but these are far from the only ways we can compose distributions to obtain new models. These ideas have been studied primarily in the context of EBMs because most compositional operators leave the resulting distribution unnormalized. We outline various options below.
Products: We can take a product of N distributions and re-normalize create new distribution, roughly equivalent to the "intersection" of the composite distributions.
(x)) 1 (1
z= == g'(x)dr.
(8)
Regions of high probability under qrrod (z) will typically have high probability under all q gz()). A simple product model can be seen in Figure 2. These ideas were initially proposed to increase the capacity of weaker models by allowing individual "experts" to model specific features in the input (Hinton, 2002), and were recently demonstrated at scale in the image domain using Deep Energy-Bant Based Models (Du et al., 2020a).
The approaches to guidance discussed in Section 2.4 define product models with only two experts. The first models the relative density of the input data and the second models the conditional probability of y. Combining these by a product models likely inputs which have the desired property y. This form of composition has become popular for diffusion models since they do not directly model the probability, but instead the gradient of the prgobabbbyl which can also be composed in this way.
Mixtures: Complementary to the product or intersection is the mixture or union of multiple distributions. We can combine N distributions through mixture create new distribution equivalent to the union of the concepts captured in each distribution
(xx = E g² (z)
where regions of high probability consist of regions of high probability under any g (z). We cannot compose ecore-function to define mixtures (unlike products). Instead, we need a model which specifies probability. Generating from mixtures of energy based models requires knowing the ratio of normalizers between the models. In our experiments, we assume this ratio is 1. A simple compositional mixture model can be seen in Figure 2.
Negation: Finally, given two distributions Po(x) (z) and Pi (I), we can explicitly invert the density of Pr (x) with respect to po(x), which constructs a new distribution which assigns high likelihood to
4

---

Under review as a conference paper at ICLR 2023
P1(3)
Pz(x)
True Result
Reverse Diffusion
HMC
X
+
Figure 2: An illustration of product and mixture compositional models, and the improved sampling performance of MCMC in both cases. Left to right: Component distributions, ground truth composed distribution, reverse diffusion samples, HMC samples. Top: product, bottom: mixture. Reverse diffusion fails sample from composed models.
points in (o)) that are not in p1 (x) (Du et al., 2020a), where a controls the degree we invert P1 (z)
(we use 0.5 in our experiments).
(x)
(x) X
(10)
(2()"
We can combine negation with our previous operators, in nested manner construct complex combinations of distributions (Figure 5).
In section 2.3, we showed how diffusion models can be interpreted approximating the gradient V log ((x), but do not learn an explicit model of the log-likelihood log g(x). This means with the standard eu (xr, --parameterization we can, in theory, utilize product and negation composition, but not mixture composition.
4 SCALING COMPOSITIONAL GENERATION WITH DIFFUSION MODELS
While highly compositional, EBMs present many challenges. The lack of normalized probability function makes training, evaluation, and sampling very difficult. Much progress has been made to scale these models (Du & Mordatch, 2019; 9; Nijkamp et al., 2020; Grathwohl et al., 2019; Du et al., 2020b; Grathwohl et al., 2021), but EBMs still lag behind other approaches terms of efficiency and scalability. In contrast, diffusion models have demonstrated very impressive scalability. Fortuitously, diffusion models have similarities to EBMs, such as their training objective and their score-based interpretation, which makes many forms of composition readily applicable.
Unfortunately, when two diffusion models are composed into, for example, a product model (rod (x) q² (xx) issues arise if the model which reverses the diffusion uses score esti- mate obtained by adding the score estimates of the two models. We see in Figure 2 that composing two models in such a way leads indeed to sub-par samples. This is because to sample from this product distribution using standard reverse diffusion (Song et al., 2021), one would need to compute instead the score of the diffused target product distribution given by
Vz log a pood (xx) = V log (J droq't (zo) q(m))
(11)
For >> O, this quantity is not equal to the sum the scores of the two models which is given by
Va log pood (xx) = V. log (ʃ dorog' (zo)g(==(za)) + V log (ʃ (10))
(12)
Therefore, plugging the composed score function into the standard ancestral sampling procedure discussed in Section 2.1, which we refer to as "reverse diffusion." does not correspond sampling from the composed model, and thus reverse diffusion sampling will generate incorrect samples from composed distributions. This effect can be seen Figure 2, with details in Appendix D.
The score of the distribution of grod (ax)) in Equation
easy to compute, unlike that of q q.nd (z,) from prodd (xx) describes a sequence distributions which smoothly interpolate Equation 11. In addition, qf.
between (x)) at 0 and (O, I) t T, though this sequence of distributions does not correspond to the distributions that result from the standard forward diffusion process described in Section 2, leading the reverse diffusion sampling generate poor samples. We discuss how may utilize MCMC samplers, which use our knowledge of V, log gondd (æt), to correctly sample from intermediate distributions q). prod (xx), leading to accurate composed sample generation.
4.1 IMPROVING SAMPLING WITH MCMC
In order to sample from grod( using the combined score function from Equation 12, we can use annealed MCMC sampling, described below in Algorithm 1. This method applies MCMC transition
5

---

Under review as a conference paper at ICLR 2023
Model Sumpler
Product
Mixture
RAISE f L.. Var In(MMD) & L..t Var 4
Reverse
1.55
-6.47
0.063
Score
ULA
2.37
1.79
0.026
U-HMC
2.52
2.40
0.021
Reverse (equal steps)
2.27
-2,92
0.046
Reverse
1.37
-6.03
0.064
-3.84
-2.17
0.020
ULA
2.36
1.84
0.027
-4.11
0.57
0.013
EBM
MALA
2.64
2.73
0.013
-4.38
1.29
0.008
U-HMC
2.63
2.45
0.022
-4.99
1.03
0.010
HMC
2.71
2.72
0.009
-4.48
1.30
0.007
Table 1: Quantitative results on 2D composition. Energy based parameterization enables mixture
compositional models, and MCMC sampling leads to better samples from compositional diffusion models.
kemelss sequence of distributions which begins with known, tractable distribution and concludes at our target distribution. Annealed MCMC has a long history enabling sampling from very complex distributions (Neal, 2001; Song & Ermon, 2019).
We explore two types of transition kernels ku (-1--) based on Langevin Dynamics (Equation 4) and HMC. When using the standard 60 >--paaameterizatin, we do not have access to explicitly defined energy-function meaning we cannot utilize any MCMC sampler with Metropolis corrections. Thus, we only utilize the ULA and U-HMC samplers described in Section 2.2. These samplers are not exact, but can in practice generate good results. In the next section we detail how Metropolis corrections may be incorporated. Full details of our samplers can be found in Appendix C.I. While continuous time sampling in diffusion models Song et al. (2021) is also referred to as ULA, the MCMC sampling procedure is run across time (and corresponds to the same sampling procedure as discretized diffusion discussed in Section 2. 1), as opposed to being used to sample from each intermediate distribution q, prod (ax)). Thus applying continuous sampling gives the same issues as reverse diffusion sampling.
Algorithm 1 Annealed MCMC
Input: Transition kernels kx (x(-))). Initial distribution pr()), Number of steps N
xT pr # Initialize.
for t = ,,....0 0, do
for i = L...., ....,N do
end for It-1 = I
end for return 2o
We can see again in Figure 2 that applying this MCMC sampling procedure allows samples from the composed distribution be faithfully generated with no modification the underlying diffusion models. Quantitative results can be found in Table 1 which further imply that the choice of sampler may be responsible for prior failures in compositional generation with diffusion models.
4.2 ENERGY- BASED PARAMETERIZATION
As noted in Section 3, we unable to mixture composition without explicitly parameterized likelihood function. But, if we parameterize a potential function [a(2, and implicitly define (=(x,)) fo i) we can recover explicit estimate of the (unnormalized) angliikelihood enabling us to utilize all presented forms for model composition.
Additionally, an explicit estimate of log- likelhhoo enables the use of more accurate samplers. As explained above, with the standard (0 >--parammterization we can only utilize unadjusted samplers. While they can perform well in practice, there exist many distributions from which they cannot generate decent samples (Roberts & Tweedie, 1996) such as targets with teateer-tan--aautse where the ULA chain is transient. Additionally, for an accurate approximation the Langevin SDE, ULA will need increasingly small stepsizes as the curvature of the log-likelihood gradient increases which can lead to arbitrarily slow mixing (Durmus Moulines, 2019). these settings a Metropolis correction can greatly improve sample quality and convergence. Again this issue can be solved by defining eø xx, t) = V..fo (,, t) for some explicitly defined scalar potential function fa t).
tails
Energy-based parameterizations have been explored in the past (Salimans & Ho, 2021) and were found to perform comparably to setting the score parameterization is then preferable computing the gradient of the energy requires more computation. In the compositional setting, however, the additional flcxibility enabled by explicit (unnormalized) log-probability estimation motivates re-exploration of the mergy-parameterizatie
models for unconditional generative modeling. In that

---

Under review as a conference paper at ICLR 2023
a)
b)
66
MALA
Ground
64
LA
Truth
Reverse
62
Accuracy
Reverse
Diffusion
56
ERM
54
HMC
1 Shape
Shapes
Shapes
15
25
Steps of Sampling
Figure 3: (a) Composition enables the positions of multiple shapes to be simultancously controlled, while training only conditions on the location of one object per image. Reverse diffusion samples place shapes in incorrect locations. MCMC generates samples that satisfy all constraints. (b) Metropolis adjustment significantly improves generation performance across sampling steps. As more MCMC steps are run (at each timestep), generation accuracy of combinations of 5 cubes improves significantl.
We explored number of energy- basey parameterizations for diffusion models and ran a pi- lot study on ImageNet. In this study we found it best to parameterize the log probability as fo(x,t) t) (1, t)ll?, where sæ(2,t) t) is vector-output neural network, like those used in (0( t)- parameterrize diffusion models. Full details on our study can be found in Appendix E. From here on, all energy- based diffusion models take the above form.
Our parreyy-rrmeetrizeedd models enable us to use MALA and HMC samplers which produce our best compositional generation results by large margin. An additional benefit of these samplers is that, through monitoring their acceptance rates, we are able to derive an effective automated method for tuning their hyper-parameters notoriously difficult task prior) which is not available for unadjusted samplers. Details of our samplers and tuning procedures can be found in Appendix C.I.
EXPERIMENTS
We experiment with various model parameterizations and sampling schemes for compositional generation with diffusion models. We first investigate these ideas on some illustrative 2D datasets, then move to the image domain with an artificial dataset of shapes. Here, we compose model conditioned on the location of a single shape with itself to condition on the location of all of the shapes in the image. After this we experiment with classifier guidance the ImageNet dataset. Finally, we self- self-compose text-to-image models to generate from compositions of various text prompts. Full details of all experiments can be found in Appendix G. Throughout we compare our proposed improvements with core-parameterized model using standard reverse diffusion sampling. We note that this baseline is exactly the approach of Liu et al. (2022).
5.1 2D DENSITIES
We train diffusion models using both parameterizations and study the impact of various sampling approaches for compositional generation. Samples are evaluated using RAISE (Burda et al., 2015) (which gives lower bounds on log-likelihoode and MMD?, LL (log-likelihood of generated samples under composed distribution), and Var (I.2 difference of variance of GMMs fit generated samples compared to GMMs of the composed distribution). Results can be found in Table and visualizations can be seen in Figure All MCMC sampling methods improve sample quality and likelihood, with Metropolis adjusted methods performing the best. All MCMC experiments use the same number of score function evaluations. We include a baseline, labeled "Reverse (equal steps)" which is a diffusion model trained with more steps such that reverse diffusion sampling has the same cost as our MCMC samplers. We see that simply adding more time-steps does not solve compositional sampling.
5.2 COMPOSING CUBES
Next, we train models on a dataset of images containing between and 5 examples of various shapes taken from CLEVR (Johnson et al., 2017). We train our models to fit p(rly) where is the location of one of the shapes in the image. We then compose this conditional model with itself to create a product model which defines the distribution of images conditioned on shapes as
For the mixture we use MMD to replace RAISE likelihood based evaluation as encountered numerical stability issues with RAISE when applying to the mixture.
7

---

Under review as a conference paper at ICLR 2023
Pizza
Daisy
Golden Retriever
Valley
Diflusion
IIM
Figure 4: Classifier-guided generation ImageNet. HMC leads to higher fidelity and more class-identified images than reverse diffusion sampling.
Model
Sampler
Combinations
Model
Sampler
Inception
Accuracy
Reverse
29.10
30.46
18.64
Reverse
70.8
68.2
66.3
64.1
57.4
Score
29.35
30.49
65.81
Score
ULA
75.0
73.4
71.8
67.9
60.2
U-HMC
32.19
26.89
89.93
U-HMC
Reverse
71.0
67.1
62.5
58.1
51.0
Reverse
28.05
33.58
18.60
ULA
81.3
71.8
66.6
19.6
54.8
28.12
33.45
66.28
EBM
MALA
30.43
32.22
83.65
U-HMC
84.5
81.3
79.2
74.2
68.1
U-HMC
31.39
32.08
90.83
HMC
91.6
82.9
80.1
76.5
72.7
HMC
33.46
30.52
94.61
Table 2: MCMC Sampling enables more com-
Table 3: MCMC Sampling enables better classifier guidance on 128x128 ImageNet dataset.
positional cube generation CLEVR.
log po (2ly))...... Mc) log
Σ (o log (3)))
(13)
We then sample using various methods, where for each number of combination of cubes, the same number of score function evaluations are used, and evaluate each by the fraction samples which have all objects placed in the correct location (as determined by a learned classifier). Results can be found in Table 2, where we see MCMC sampling leads improvements and the Metropolis adjustment enabled by the energy-based parameterization leads to further improvements. We qualitatively illustrate results Figure 3, and see more accurate generations with more steps of sampling, with more substantial increases with Metropolis adjustment.
5.3
CLASSIFIER CONDITIONING
Next, we train unconditional diffusion models and noise-conditioned classifier on ImageNet. We compose these models
V log (xlyy, t) = V (o + Vz log t).
(14)
and sample using the corresponding score functions. We compare various samplers and model parameterizations on classifier accuracy, FID (Heusel al., 2017) and Inception Score. Quantitative results can be seen in Table 3 and qualitative results seen in Figure 4. We find that MCMC improves performance over reverse sampling, with further improvements from Metropolis corrections.
5.4 TEXT-2-IMAGE
Perhaps the most well-known results achieved with diffusion models are in text-to-image generation (Ramesh et al., 2022; Saharia et al., 2022). Here we model (Timage [Mes)). While generated images generated are photo-realistic, they can fail generate images from prompts which specify multiple concepts at time (Liu al., 2022) such as Vtext = "A horse on a sandy beach or a grass plain on a not sumny day". To deal with these issues dissect the prompt into smaller components .., ., yes parameterize models conditioned on each component po y:) and compose these models using our introduced operators. We can parse the above example into
"A horse" AND ("A sandy beach" "Grass plains") AND (NOT "Sunny")
which can be used to define the following (unnormalized) distribution
X Pa (A horse")
pe (xA sandy beach") + 1 (x) "Grass plains"))
Po (xll)ee))
comp
Po [x/ "Sunny"")
Liu et al. (2022) demonstrated that composing models this way can improve the efficacy of these kinds of generations, but was restricted composition using classifier-free guidance. We train
8

---

Under review as a conference paper at ICLR 2023
"A honse" AND
AND
A sandy beach" "Grass plains")
AND "Grass plains"
AND "A sandy beack""
A"A sandy beachi" OR
"Grass plains")
AND (NOT ("Sunry "))
Figure 5: Energy based parameterization enables high-resolution compositional text-to-image synthesis.
energy-paren paraaetrreeee diffusion model for text conditional 64x64 image generation and illustrate composed results in Figure (upsampled to 1024x 1024). We find that composition enables more faithful generations of scenes in Figure 6 with more results in Appendix A.
"A lake with purple trees."
lake with purple trees." " AND "Purple trees."
Figure 6: Composing text descriptions enables more accurate scene generation.
DISCUSSION
Limitations. Our work demonstrates that diffusion models, in combination with MCMC-based sampling procedures, can be composed in novel ways capable of generating high-quality samples. However, our proposed solutions have number of drawbacks. First, more sophisticated MCMC samplers come at higher cost than the standard sampling approach and can take 5-times longer to generate samples than typical diffusion sampling. Second, we have shown that energy-parameteerizd models enable the use of more sophisticated sampling techniques, garnering further improvements. Unfortunately, this requires second backward-pass through the model compute the derivative implicitly, leading them to have double the memory and compute cost of score- poore-parameterieed models.
While these are considerable drawbacks, we note the focus of this work is to demonstrate that such things are possible within the framework of diffusion models. We believe there is much that can be done to achieve the benefits of our sampling procedures at less cost such as distillation (Salimans Ho, 2022) and easier-to-di di-ffereeniitee neural networks (Chen & Duvenaud, 2019).
Finally we note that not all models can be effectively composed together. For example if we wanted to model the product of (-110, (10 1), the resulting distribution's 's support is far outside the support of the constituent models. To accurately model this, our constituent models would need to be near-perfeed far outside the training distribution. Thus it is unlikely for good results to be obtained. The same care should be taken in the text-2-image setting with, for example, contradicting prompts.
Conclusion. In this work we have explored the ways that pretrained diffusion models can be composed to model new distributions. We demonstrate ways that naive implementations fail, and present two ways that performance can be improved: MCMC sampling and energy parameterized diffusion models. We find our proposed methods lead to notable improvement across a variety of domains, scales, and different compositional operators.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Julian Besag. Comments "Representations of knowledge in complex systems" by U. Grenander and MI Miller. J. Roy. Statist. Soc. Ser. B, 56(591-592):4 1994.
Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale GAN training for high fidelity natural image synthesis. arXiv preprint arXiv: 10096, 2018.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models few-shot learners. Advances in Neural Information Processing Systems, 33:1877-1901, 2020.
Yuri Burda, Roger Grosse, and Ruslan Salakhutdinov. Accurate and conservative estimates of mrf oog-likelihood using reverse annealing. In Artificial Intelligence and Statistics, pp. 102-110. PMLR, 2015. 7
Ricky TQ Chen and David K Duvenaud. Neural networks with cheap differential operators. Advances in Neural Information Processing Systems, 32, 2019. 9
Prafulla Dhariwal and Alexander Quinn Nichol. Diffusion models beat GANs image synthesis. In Advances in Neural Information Processing Systems, 2021. 1,4
Yilun Du and Igor Mordatch. Implicit generation and generalization in energy-based models. arXiv preprint arXiv:1903, 08689,, 2019. 3, 5, 18
Yilun Du, Shuang Li, and Igor Mordatch. Compositional visual generation with energy based models.
Advances in Neural Information Processing Systems, 33:6637-6647, 2020a. 4, 5
Yilun Du, Shuang Li, Joshua Tenenbaum, and Igor Mordatch. Improved contrastive divergence training of energy based models. arXiv preprint arXiv:2012.01316 2020b. 5
Simon Duane, Anthony D Kennedy, Brian J Pendleton, and Duncan Roweth. Hybrid Monte Carlo.
Physies letters B, 195(2):216-2))) 222, 1987. 3
Alain Durmus and Eric Moulines. ligeesimmnsional Bayesian inference via the unadjusted Langevin algorithm. Bernoulli, 25(4A))2854-288) 2019.
Tomas Geffner and Justin Domke. MCMC variational inference via uncorrected Hamiltonian annealing. Advances in Neural Information Processing Systems, 34:639-651, 2021. 3
Will Grathwohl, Kuan-Chich Wang. Jörn-Henrik Jacobsen, David Duvenaud, Mohammad Norouzi, and Kevin Swersky. Your classifier is secretly an energy based model and you should treat it like one. arXiv preprint 1xXiv:112.00326. 2019. 5
Will Grathwohl, Jacob Jin Kelly, Milad Hashemi, Mohammad Norouzi, Kevin Swersky, and David Duvenaud. No MCMC for me: Amortized sampling for fast and stable training of energy-based models. In International Conference on Learning Representations, 2021.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge local Nash equilibrium. Advances Neural Information Processing Systems, 30, 2017. 8
Geoffrey E Hinton. Training products of experts by minimizing contrastive divergence. Neural Computation, 14(8):1771-1800, 2002. 1,
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12599. 2022. 1,
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 18564 2020. 1, 20, 21
Robert A Jacobs, Michael Jordan, Steven Nowlan, and Geoffrey Hinton. Adaptive mixtures of local experts. Neural Computation, 3(1):79-87, 1991.
10

---

Under review as a conference paper at ICLR 2023
Justin Johnson, Bharath Hariharan, Laurens Van Der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick. Clevr: diagnostic dataset for compositional language and elementary visual reasoning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 2901-2910, 2017. 7
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08366/ 2020.
Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, et al. Competition-level code generation with alphacode. arXiv preprint (rXXi:2203.078//1 2022.
Nan Liu, Shuang Li, Yilun Du, Josh Tenenbaum, and Antonio Torralba. Learning compose visual relations. Advances in Neural Information Processing Systems, 34, 2021. 20
Nan Liu, Shuang Li, Yilun Du, Antonio Torralba, and Joshua B Tenenbaum. Compositional visual generation with composable diffusion models. arXiv preprint arXiv:2206.0/714, 2022. 7, 8
Guy Mayraz and Geoffrey E. Hinton. Recognizing hand-written digits using hierarchical products of
experts. Advances in Neural Information Processing Systems, 13, 2000. 1
Radford M Neal. Monte Carlo implementation. In Bayesian Learning for Neural Networks, pp. 55-98. Springer, 1996. 3, 14, 15, 16
Radford M Neal. Annealed importance sampling. Statistics and Computing, 11(2):125-39, 2001.
Erik Nijkamp, Mitch Hill, Tian Han, Song-Chun Zhu, and Ying Nian Wu. On the anatomy of meme-based maximum likelihood learning of energy- models. In Proceedings of the AAAI Conference Artificial Intelligence, volume 34, pp. 5272-5280, 2020. 3,5, 18
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with clip latents. arXiv preprint 2rX1v:2204.06/155, 2022. 8
Gareth O Roberts and Richard L Tweedie. Exponential convergence of Langevin distributions and their discrete approximations. Bernoulli, pp. 341-363, 1996. 3, 6
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S Sara Mahdavi, Rapha Gontijo Lopes, et al. Photorealistic text-to-image diffusion models with deep language understanding. arXiv preprint 1444 2022. 1,8
Tim Salimans and Jonathan Ho. Should ebms model the energy the score? In Energy Based Models Workshop-ICL. 2021, 2021. 6, 18
Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. arXiv preprint arXiv:2202.0051: 2022.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference Machine Learning, pp. 2256-2265. PMLR, 2015. 1,2
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in Neural Information Processing Systems, 32, 2019. 1,
Yang Song, Jascha Sohl-Dickstein Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021. 6,
Hongyi Zhang, Yann N Dauphin, and Tengyu Ma. Fixup initialization: Residual learning without normalization, arXiv preprint
2019. 20
11

---

Under review as a conference paper at ICLR 2023
Appendix
In this appendix, present additional text-to-image results in Section A. We present detailed derivations diffusion models in Section B. We present additional information on MCMC sampling in Section C.I. We provide additional derivations composing diffusion models in Section D. We discuss different parameterizations of energy based diffusion models in Section E. We further provide additional example 2D compositions in Section Finally, we provide experimental details in Section G.
A TEXT-TO-IMAGE RESULTS
We present additional use cases of composing models in different text-to-image domains. First, in Figure A1, illustrate how composing separate energy parameterized diffusion models enables us to more accurately generate images that have more detailed information in the caption. Next, in Figure A2, we illustrate how composing two separate energy parameterized diffusion models further enable us to accurately generate images with the correct colors assigned to each object. We further show in Figure A3 how composing the negation of one energy parameterized diffusion model with another other enables us to generate images where one commonly occurring co-founding factor does occur (i.e. a sandy beach without coastal water). Finally, we illustrate Figure A4, how composing multiple diffusion models enables us to render the number of objects in scene accurately.
Central Park 1900s. Its snowy in New York.
Central Park 1900s. AND Its snowy New York.
Taj Mahal 1900s. Snow falling down.
Mahal 1900s. AND Snow falling down
Figure A1: By composing energy based diffusion models, we can render more detailed information in images. In the above images, we can more accurately render details such as Central Park (top) or the effect of snowing (bottom).
12

---

Under review as a conference paper at ICLR 2023
A red car in front of pink forest
A red car front of pink forest AND pink forest
A lake with purple trees
A lake with purple trees AND purple trees
Figure A2: By composing energy based diffusion models, accurately render different colors of objects a scene.
B DETAILED DERIVATION OF DIFFUSION MODELS
Diffusion models seek to model a data distribution g(zo) (written this way for notational convenience) and define series of latent variables Ti, , generated from Markov process v; g(xx (2) where
UT
( -1 = N (10.5 /1 - 11, 8.1).
(A1)
unique and useful property of this process that all time marginals q(re(zz)) can be computed in closed form and are Gaussian
(0)))=)) = N (zz) 11 of20.00 of
(A2)
where of
āg and a. IT Bt). We can see that if all 0, then as oo q(Z2) (2)) + (x,;;
We seek to train model (0 (at) which reverses (x,=)) sise with a parametric model. We can analytically derive the variance of the reversal as Br 1-a,-1
I-a. and define
po (2 (- ((1 no uo (z)
(A3)
and set p(ær) = A 1). train this model to maximize a variational bound on the marginal likelihood
log po (1o) 2 (eo poggo +
(A4)
D (
(A5)
13

---

Under review as a conference paper at ICLR 2023
Sandy Beach
Sandy Banch AND (NOT (Coastal Water))
Forest
Forest AND (NOT (Green Foliage))
Figure A3: By composing energy parameterized diffusion model with the negation of another energy parameterized diffusion model, we render images in unusual configurations not typically found in the data.
The first term lacks parameters and the final is approximately 0 from the convergence of the g-process so we focus on the middle terms.
Typically, the model [ f) is not parameterized to predict the mean of z4. Instead it is parameter- ized to predict the noise added x4 arrive 2t-1- This motivates the following parameterization
1 Var,
Be
(a t)
It
(A6)
V1
ā
In this form we can rewrite the important terms in the objective as
D
= -C. [lle - (x 1)1 = C,L(x, (A7)
where C a time-dependent constant. Typically these are dropped and objectives are weighted equally.
Once we finish training, we can draw samples from our model by first sampling 2т p(xr) and then equentially sampling p,( /Be where (O0, T).
C MCMC SAMPLING DETAILS
C.I HAMILTONIAN MONTE-CARLO AND ITS VARIANTS
Hamiltonian Monte-Care (Neal, 1996) seeks to sample from an unnormalized probability distribution log ((x) fxx log Z. To do this, augment distribution over with auxillury variables
14

---

Under review as a conference paper at ICLR 2023
Two Oranges
Two of the fruit
Two Oranges AND Two of the fruit
Figure A4: By composing multiple energy parameterized diffusion models, accurately render the underlying number objects ascene.
and define the joint distribution p(z, v) ((x) 0, .M) where the covariance M is known as the p(z, v) and since a and v are independent under
"mass-matrix."" We now seek to draw samples the joint, we can simply throw away our samples leaving us with sample x ~ p(x).
Like other MCMC methods we sequentially update particle (æ", v²) in such a way that as + co we arrive a sample from p(x,,)), For a step of HMC, starting at (x),=") we first sample n ~ (u"; 0, M) since the target distribution factorizes and p(v) is known and tractable. We then integrate a likelihood-conservvnn ODE defined on I, known as "Hamiltoninan Dynamics." We use the iikeihhood-preservinn preserving leapfrog integrator which will guarentee that the transition distri- bution symmetric, i.e k(æ', of [z, v) = k(x, u v|2', v'). Thus, the Metropolis acceptance probability simplifies min (1, p(z',²") An overview of the HMC algorithm can be found in Algorithm 2. We p(a,v) refer the reader to Neal (1996) for
complete description of the algorithm.
Algorithm 2 Hamiltonian Monte-Carlo
Input: Initial state _ Mass matrix M, Number of steps N, Number leapfrog steps L, step-size for i I,...,4 N do
Sample o² ~ (O,M)
Sample momentum
z'. " Leapfrog((z (x'"1 :ete,, # Integrate dynamics with stepsize for steps
min (1, p(z' "' With probability
# Compute acceptance probability
set x
else
set x' i-1
end for return N
Since our (z(2, t) parameterized models do not admit an explicit likelihood function, we use an unadjusted variant of HMC (U-HMC) where the accept/reject step is simply ignored.
15

---

Under review as a conference paper at ICLR 2023
We in Algorithm that at every step, the momentum is re-sampled. This can be sub-optimal, as the momentum determines the initial direction x' movement and if a good direction is found, may be beneficial to continue in that direction. To deal with this, Neal (1996) proposes variant of HMC where the momentum e is partially retained between sampling steps. We add an additional sampler parameter
(0, (known the "damping-factor") which controls the amount which is retained. When y is close to I, is mostly kept and when it is near 0, is mostly refreshed. This variant is summarized in Algorithm 3. The potentially confusing momentum negations ensure the validity of the sampler. Intuitively, when the proposal is accepted, the momentum is retained and when it is rejected the momentum flipped. For this reason, one should maintain reasonably high acceptance rate when using this approch.
Algorithm 3 Hamiltonian Monte-Carle with Partial Momentum Refreshment
Input: Initial state xº, Mass matrix M, Number of steps N, Number leapfrog steps L, step-size damping-factor
Sample e (0,
Sample initial momentum
for i .....y N do
X ~~ N M)
o 1)1 -o- + 2
# Partially refresh momentum
2', v/ = Leapfrog((
ol ;e,
# Integrate dynamics with stepsize for L steps
=-
# Negate momentum
min (1, p(z'.'")
# Compute acceptance probability
With probability set x² x', v²
else
set x' x'11, n² ylf-1)/
o² - end for return x
# Negate momentum
C.2 MCMC TUNING
crucial component ensure successful MCMC sampling in diffusion models is the choice of step sizes for samplers. We initialize step sizes for all samplers at each distribution / to be roughly proportional the 3t noise values added to distribution in the diffusion process.
To tune step sizes across timesteps for both HMC and MALA samplers, to set step sizes at each timestep to be constant multiplied by 3r. We searched different constants to multiply 3r and chose value so that the average acceptance rate of MALA and HMC samplers across timesteps is approximately 60% and 70% respectively. For un-adjusted variants of these samplers, we step sizes to be the same as adjusted samplers, and found limited gains when step sizes were specifically tuned towards the un-adjusted samplers. We utilize a mass matrix of B, for HMC samplers.
Precise details on the exact MCMC steps sizes used in experiments can be detailed in Section G.
C.3 MCMC IMPLEMENTATION DETAILS
When initially rumning MCMC sampling on diffusion models in the image domain, we found that our samplers tended to converge images which had uniform textures. After experimentation, we found that the primary cause of this issue was fact that by default, typical implementations of the reverse diffusion process clip samples at intermediate time-steps of sampling be between -1 and 1. To enable proper MCMC sampling, we found that it was important to not clip intermediate values of diffusion sampling.
When running MCMC sampling on image domains, we further found that was helpful for mixing to run single step of the reverse process to initialize MCMC sampling, before running many steps of MCMC sampling at each timestep t, and all MCMC sampling settings on the image domain, step of the reverse process before running MCMC sampling. Such MCMC sampling
16

---

Under review as a conference paper at ICLR 2023
procedure is similar to the predictor-corree corrector sampling procedure introduced in (Song al., 2021) for alleviating discretization emrors when sampling continuous time diffusion models.
D COMPOSITIONAL DIFFUSIONS
In Equations 11 and 12, demonstrate that for diffused distributions (xe)) where (xr) [ (x) (oooo the diffusion of the product of q is not the same as the product of the diffusions, meaning plugging the product diffusions into standard reverse diffusion sampling will not draw samples from the product model. We present similar results for tempering and predictive model composition.
D.1 SAMPLING FROM A TEMPERED VERSION OF q USING DIFFUSION?
It is tempting to believe that we can sample from tempered/annneled version of the data distribution
(x) x ((x)>>
using the tempered diffused data distribution AV log, ((z;) but this is incorrect. For this procedure to be correct, we would need to have V log q (x+) = AV log (:(x,) for all However, while we do have V log (20) AV log 40 (2o), this equality does not hold for / 0
Vlog / (z)))
/
f AV log
D.2 GUIDANCE
For conditional generation, we should use in the reverse diffusion the score of the diffused conditional distribution V log (xxl))) where
ge(xxle) - / 9(201))9((x)/)))d00
We also have
v (logg((z)lz) (2.)) V log ())(()))
so that
log (y) V log qe (x.ly) V log (2x))
allows you to do guidance without having to train say classifier if y is categorical. In practice, was found that using in the reverse time diffusion the score
v log (x²) + XV log (3)(2)))
generates much nicer images for À > However, it is also often claimed that it samples from modified posterior where the likelihood has been annealed. This is incorrect. For a modified posterior with annealed likelihood, would have
g (xule) q(xulv) {a(v)))))>
and it true again that
= Vlog / (2
v log (xele)
V (2)) A (yl
17

---

Under review as a conference paper at ICLR 2023
D.3 SAMPLING FROM COMPOSED DISTRIBUTIONS
We can see that products, tempering, and guidance applied to diffused distributions do not give diffusions of the modified target distributions. Thus, we should expect to arrive our desired result by applying a sampling procedure which reverses a diffusion applied to the target distribution. Thankfully, as stated in Section 4, these operators do give sequence of distributions which anneals from NV (O, I) to the composed target which means we can utilize the family of annealed MCMC sampling methods mentioned in Section 4.1 to draw samples from our composed models in all of these settings, directly using the available score estimate.
E ENERGY-BASED PARAMETERIZATIONS
As mentioned in section 4.2, when using the e0 1) parameterization, can recover an estimate of the time-conditional score function with v log (r()) =a(x.)) This estimate of the log-likelihood gradient can be used for MCMC sampling methods which only require the log-likelihood gradient- such as ULA or U-HMC. These methods can work well, but will never generate exact samples when using non-zero step-sizes. Exact samplers can be derived from approximate samplers like the above methods using Metropolis corrections. Unfortunately, even if the samplers' transition distribution kx(:)) does not require log (oxx) evaluation, the Metropolis correction probability:
o,
1,. ef)( k(((x1-1)
efo(2) 1(x)-118)
min
does. Futhermore, when we only have an estimate of the score at our disposal, we are only able to compose models using products.
To enable the use of Metropolis corrections and more compositional operators, we propose to change the parameterization of our diffusion model. Instead of using neural net (x, t) (R" x N} + Rd we define a scalar- outputt neural network fa t) {IR" x N} - R. We then compute the gradient of this function and define (z( t) Vofo( (). From here, we this implicitly-defined e0 in standard diffusion model training. As before, we can recover V, log (z)) s =0(0.1 but now we are also able to recover log Pa (x) fo(w,)) + log Z which enables the application of Metropolis corrected sampling.
Much prior work on EBMs parameterizes fo 1) using a feed-forward neural network, whose final layer has single output (Nijkamp et al., 2020; Du Mordatch, 2019). Salimans & Ho (2021) compare this approach with the standard c0 t) parameterization and find the c, t) parameterization to perform better for unconditional image generation. We believe this has to do with the relative sparsity of the gradients of feed-forward neural networks. This can cause difficulties when training to optimize a function of their implicitly computed gradients.
Intriguingly, Salimans & Ho (2021) also explore more structured energy function definition inspired by denoising autoencoders:
1 DAE (x. = --1 (o, t)1?
1
where so t): (Rd x N] + Rd is a neural network (identical to the standard co(2, t)) model. We simply evaluate the gradients of this function to obtain
V.IP (x, () (x sa(c, ()) - - so (x,
In their study, this parameterization was found to perform near identically to the cø(2, !) param- eterization while admitting an explicit energy function. We believe this energy parameterization performs better because its gradients include the feed-forward network sa (z, t), making optimization casier. Salimans & Ho (2021) conclude that the c0 t) parameterization should be favored since the computing V V.fA (a., t) requires computing (ss which requires extra backward pass through the neural network, increasing compute.
We reexamine this energy parameterization and two other choices now that application motivates having access to an explicit energy function. The other parameterizations are based different transfor- mations of the so (x, architecture; the negative 1.2 norm (I.2) and inner product (IP). They
18

---

Under review as a conference paper at ICLR 2023
defined as:
16 (2, =21l0 t)12
vx 42 (x, = -50(1, 1) V)
and:
xTsa(zzt) V n 1P (x,t) so(x, 1) + x² (
f8 (z,
We train models with each parameterization ImageNet and compare using FID for unconditional sampling. Results can be seen Table A1. We see that L2 and Inner-Product perform the best, but both outperformed by the standard parameterization. We initially experimented with these two parameterizations but found that the 1.2-norm parameterization to more stable for compositional sampling due, we believe, the fact that the energy-functicg function bounded above meaning that MCMC sampling is incapable of running off to infinity to increase likelihood.
Parameterization
(1) DAE (2) L2 Norm (3) Inner-Product
ea (I, t)
97.4
91.5
90.9
86.7
Table Al: FID (Ik samples) of various energy paaneeerraattiiiiiionns on unconditional ImageNet generation.
F SYNTHETIC DISTRIBUTION COMPOSITIONS
Mixture We provide additional 2D illustrations mixtures of two diffusion models in Figure A5. We find that HMC sampling enables more accurate mixtures of different synthetic distributions.
Px
PsO0
+
Figure A5; Examples of mixture applied to diffusion models. Left right: Component distributions, reverse diffusion, HMC sampling. Reverse diffusion fails to sample accurately from mixed distributions distributions.
Negation We provide additional 2D illustrations of negating two diffusion model with respect to each other Figure A7. find that HMC sampling enables accurate negations of different sythetic distributions.
P2(x)
Figure A6: Examples of negation applied to diffusion models. Left to right: Component distributions, reverse diffusion, HMC sampling. Reverse diffusion fails sample accurately from negated distributions.
Failure Cases Next illustrate failure of composition using approach in Figure A7. Our approach fails to generate the product of two distribution when they are disjoint with respect to each other.
19

---

Under review as a conference paper at ICLR 2023
(+)
P=(*)
Revense Dillusion
HMC
X
Figure A7: Failure Cases of Our Approach. Left to right: Component distributions, reverse diffusion, HMC sampling. Our approach fails to generate products of distributions with no overlap.
G EXPERIMENTAL DETAILS
We provide detailed experimental details including underlying quantitative metrics, training details, and architectures on 2D synthetic, CLEVR, ImageNet, and test-to-image settings below. To enable stable training of energy-based diffusion models in image settings, we clip gradient norms to be less than 10, and initialize convolutional layers using aro-initializzation (Zhang et al., 2019).
Synthetic Datasets For synthetic datasets, train both score and energy based diffusion models using& a small residual MIP model with residual blocks, with a internal hidden dimension of 128 dimensions. We train models for 1 15000 iterations (10 minutes optimizer with learning rate of 1e-3, and train diffusion models on 100 discrete timesteps with linear schedule of B values.
8 TPUv2 cores) using the Adam
When evaluating product of diffusion models, we generate two separate distributions, where train two separate diffusion models. our first distribution, we construct a GMM of Gaussians in a ring of radius 0.5 around the origin, with each Gaussian having standard deviation of 0.3. In our second dataset, we construct a uniform distribution of points with x between 10.1 and 0.1 and y between -1 and 1. When evaluating mixture diffusion models, we generate one distribution consisting of a mixture of Gaussian with standard deviation 0.03 and centers at (-0.25, 0.5), (-0.25, 0.0), (00.5 0.5), and another distribution consisting of mixture of 3 Gaussian with standard deviation 0.03 and centers at (0.25, 0.5), (0.25, 0.0), (0.25, -.55).
To construct MCMC samplers from models on synthetic datasets, we run 3 steps of HMC per timestep, with leapfrog steps per step of HMC. We run 10 steps of MALA sampling per timestep. We found that MCMC performed robustly in the 2D dimensional setting and set the step size of MALA to be 0.002 across all distributions and the step size of HMC to be 0.03 across all distributions (with a mass matrix of 1)
CLEVR For CLEVR, we generated a dataset of 200,000 64 x 64 images with between to different cubes using dataset generation code in (Liu al., 2021). To evaluate the accuracy in which generated images had cubes at each specified position, we trained a binary classifier on these images, and marked cube as correctly generated if the confidence of the binary confidence of classifier is greater than 0.5.
To parameterize our diffusion architecture, we follow the architecture of (Ho al., 2020), where we use a base hidden dimension of 128, and multiply the hidden dimensions by 2,, 4 at different resolutions of the image. We utilize 3 residual blocks at each resolution of the image. We trained diffusion models with 100 discrete timesteps with linear / schedule. CLEVR models were trained for 20000 iterations with a batch size of 1024 using the Adam optimizer with step size le-4, corresponding to roughly 8 hours on 8 TPUv2 cores.
To initialize MCMC sampling on the CLEVR domain, each timestep, before applying MCMC sampling, we one step of the reverse process in the trained diffusion model. We run 40 steps of MCMC sampling per timestep for MALA samplers, and 13 steps of HMC sampling (with 3 leapfrog step per HMC step) (with the mass matrix of HMC samplers set to B8). We use HMC with partial momentum refreshment, and use a dampening coefficient of 0.9 across HMC iterations. MALA step sizes set to 0.035 3r and HMC step sizes set to 0.1 S,
ImageNet For ImageNet, we train an unconditional diffusion model 128 x 128 images. We train diffusion models for million iterations of ImageNet with batch size of 64 (3 days 16 TPUv2
20

---

Under review as a conference paper at ICLR 2023
cores), using Adam optimizer with learning rate le-4, for 1 million iterations. We train diffusion models with 1000 discrete timesteps using the
On the ImageNet dataset, report three seperate metrics. To report classifier accuracy, we feed generated sample into a ImageNet classifier trained on clean images, and label a image as correctly generated if the classifier of generated image having the specified class is greater than 50%. We further report the Inception Score and FID, which are calculated on 50000 generated samples.
We follow the architecture of (Ho al., 2020), where multiply the hidden dimensions by [1, 1, 2, 3, 4] at the different resolution of the image. We utilize residual blocks each resolution of the image.
a base hidden dimension of 128 and
To initialize MCMC sampling on the ImageNet domain, at each timestep, before applying MCMC sampling, we run one step of the reverse process in the trained diffusion model. We run steps of MCMC sampling per timestep for MALA samplers, and 2 steps of HMC sampling (with 3 leapfrog steps per HMC step and with the mass matrix of HMC samplers set (3. MALA step sizes are set to 0.5 * 5, and HMC step sizes are set to 0.6 33
Text-to-Image For text-to-image models, we train models for one week on an internal text/image dataset consisting of 400 million images using 32 TPUv3 cores, with training data batch size of 256. We train our energy-base text-to-image model using a total of 1000 timesteps with cosine beta schedule. We follow the architecture of (Ho et al., 2020), where we use a base hidden dimension of 256, and multiply the hidden dimensions by [1, 2, ; 4] at different resolution of the image. We utilize 3 residual blocks at each resolution of the image.
To upsample images from 64 x resolution 1024 1024 resolution, we utilize two trained unconditional diffusion models, one trained to upsample from 64 x 64 resolution to 256 256 resolution and one trained to upsample from 256 x 256 resolution to 1024 x 1024 resolution.
To initialize MCMC sampling the text-to-image domain, at each timestep, before applying MCMC sampling, we run one step of the reverse process in the trained diffusion model. We ran steps of HMC sampling per timestep, with 3 leapfrog step per HMC step and mass matrix of HMC samplers set to 3). We use HMC with partial momentum refreshment, and use a dampening coefficient of 0.9 across HMC iterations. HMC step sizes set 0.1 3r.
21


Reviews:
[{"review_id": "vPi3wp4uSjQ", "reviewer": "Reviewer_P5qB", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This papers shows how diffusion models can be combined together to compose several relationships. The authors take an energy minimization perspective of diffusion models. From this perspective, sampling from the distribution can be seen as running an MCMC. Then, the authors show how different distributions can be composed together. They consider three types of distributions - product, mixture and negation. To sample from these distributions, the authors propose annhealed MCMC algorithm, which is explained in appendix. The authors then show some experimental results where their proposed sampler improves composition compared to baselines.", "summary_of_the_review": "While the paper addresses an important problem, I feel experimental rigor is lacking and the authors could have done a better job comparing with other papers.", "strength_and_weaknesses": "Strengths:\nThe idea of composing distributions is an important one. One of the fundamental limitations of the current text to image models is the inability to compose several relationships, hence this problem is an important one to address. The paper takes a principled approach for this problem by considering an energy minimization perspective. They come up with good sampling algorithms that seem to improve over baseline.\n\nWeakness:\nOne of the main weaknesses I see is the similarity between this paper and [1]. This paper addresses the exact same problem and they take an energy minimization perspective. This paper considers both product distributions and negations.  They use MCMC as well. The authors in this paper cite this work, but they don't mention the similarities and differences between this work. I find this very concerning. I think the difference lies in the samplers used. This paper uses HMC which [1] doesn't. But I'm still unable to assess how this is better than [1].\n\nWith respect to experimental validation, the authors show that using HMC improves over other baseline samplers. But, it would have been nice to include comparison with other papers like what [1] did. In fact, [1] had a table for CLEVR dataset. It would have been nice if the authors showed the comparison with that.\n\n[1] Liu et al., \"Compositional Visual Generation with Composable Diffusion Models\", ECCV 2022", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Jh051k1saNa", "reviewer": "Reviewer_8B2n", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes to use composition operators (product, mixture, negation) to modify the distribution for data generation. To do so, the authors proposed an energy-based parameterization of diffusion models. HMC (Hamiltonian Monte Carlo) is used for sampling rather than the reverse diffusion method. A simple illustrative and quantitative evaluation of the downstream tasks supports this design choice. In the experiment, the method's strength was demonstrated using many image generation tasks.  ", "summary_of_the_review": "This paper proposed a simple and effective way to compose distributions for image generation. It is strong in both theory and experiment. The paper is nicely written and easy to follow. ", "strength_and_weaknesses": "Strength:\n- The paper clearly illustrated the challenge of composing data distributions for image generation and proposed an effective way to solve it.\n- The idea is clearly described and the intuition is clear.\n- Sufficient experiments to support its effectiveness.\n\nWeakness:\n- It might be better to discuss some failure cases for using this approach.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "FMpRLtD3lv", "reviewer": "Reviewer_5cCF", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes methods to compose diffusion models as products, mixtures, and negations of distributions. The use of LMC and HMC samplers is first motivated by the analysis of the intermediate distributions given by the reverse diffusion. An energy-based parameterisation of diffusion models is presented which enables mixture composition and allows Metropolis corrections in the samplers. Empirical evaluation of compositional generation is provided on multiple datasets and tasks.\n\nLMC: Langevin Monte Carlo,\nHMC: Hamiltonian Monte Carlo", "summary_of_the_review": "Overall, the paper presents an interesting framework for compositional generation. The technical contribution is limited but the paper brings empirical insights that may be relevant to the generative modeling community. There also exist some issues with the clarity which can hopefully be addressed in a revision.", "strength_and_weaknesses": "[Strengths]\n* The paper presents an interesting study on the use of different samplers and how they significantly change the quality compositional generation. \n* The proposal of the energy-based parameterization, although computationally expensive, is well motivated.\n* Multiple qualitative and quantitative experiments have been presented that show the improvements offered by the proposed method.\n* I appreciate that the authors have clearly discussed the limitations of their proposals. \n* The paper is overall well written but there exist some clarity issues (see weaknesses).\n\n[Weaknesses]\n* Clarity:\n    * For readers unfamiliar with the background, please clearly define what is meant by reverse diffusion.\n    * The jump to \"improving sampling with MCMC\" is abrupt. It is unclear how annealed MCMC improves the results over reverse diffusion. Furthermore, the authors have completely abstracted the details (e.g., the kernel k_t) in this section which forms one of the central theses of this work. I would like to see a more detailed discussion clearly explaining how LMC/HMC helps with compositional generation. \n* With the choice of LMC/HMC samplers, I would have liked to see a more detailed discussion on continuous-time diffusion models (e.g., [1]).\n* I would have liked to see more examples/experiments of mixture and negation compositions.\n\n[1] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "8tkKFM6YB9r", "reviewer": "Reviewer_ndtR", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper aims to compose energy-based diffusion models for compositional generation. The authors first point out the deviation between sampling from the composed score function and sampling from the composed diffusion model. To this end, MCMC samplers are adopted in the sampling procedure to improve the composition generation of diffusion models, and with proper energy-based parameterizations, Metropolis corrections may be incorporated into the sampling process. The experimental results demonstrate that the proposed method realizes the composition of diffusion models in different ways and on different data.", "summary_of_the_review": "This paper is well motivated. It points out the deviation between sampling from the composed score function and sampling from the composed diffusion model. The novelty of the paper seems limited as all the building blocks of the proposed method are referred to existing works. The experimental results seem good.", "strength_and_weaknesses": "Strengths:\n1. This paper has a good motivation. For diffusion models, compositionality can be an important property for many applications. For example, in the experimental results, the proposed method has shown good performance in the compositional scene generation and text-to-image generation.\n2. This paper clearly explains the problems of diffusion models in compositional generation, and uses MCMC samplers which may incorporate Metropolis corrections by energy-based parameterized diffusion models. The effectiveness of the sampling procedure is validated in the experiments.\n\nWeaknesses:\n1. The novelty of the paper seems limited. All the building blocks of the proposed method are referred to existing works, including the composition operations in Section 3, MCMC samplers in Section 4.1 and energy-based parameterization in 4.2. \n2. The organization of the paper can be improved. Preliminary knowledge (including introduction to diffusion models in Section 2 and introduction to composition operations in Section 3) span more than three pages. In contrast, the main technical part is too short to elaborate the method.   \n3. Although the paper points out the deviation between sampling from the composed score function and sampling from the composed diffusion model; however, this gap is not demonstrated on all tasks/datasets in the experiments.\n4. In the abstract, it says “explore alternative ways to condition, modify, and reuse diffusion models” and “we investigate why certain types of composition fail using current techniques and present a number of solutions”; however it seems that there are no corresponding content in the main body of the paper, do I miss something or is it overstated in the abstract?\n\nSome comments:\n- Figure 3 lacks some necessary explanations, for example, what do the boxes with different colors indicate? \n- In Table 2 and Table 3, are EBM and Energy the same model?\n- Some good experimental results of text-to-image are shown in Figure 5. Some failure cases of this task can also be shown and discussed.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject