Classifier-Free Guidance is a Predictor-Corrector
Authors: Arwen Bradley
Preetum Nakkiran
Venue: Accepted by TMLR ()

Abstract: We investigate the theoretical foundations of classifier-free guidance (CFG). CFG is the dominant method of conditional sampling for text-to-image diffusion models, yet unlike other aspects of diffusion, it remains on shaky theoretical footing. In this paper, we first show that CFG interacts differently with DDPM (Ho et al., 2020) and DDIM (Song et al., 2021), and neither sampler with CFG generates the gamma-powered distribution $p(x|c)^\gamma p(x)^{1-\gamma}$. Then, we clarify the behavior of CFG by showing that it is a kind of predictor-corrector method (Song et al., 2020) that alternates between denoising and sharpening, which we call predictor-corrector guidance (PCG). We prove that in the SDE limit, CFG is actually equivalent to combining a DDIM predictor for the conditional distribution together with a Langevin dynamics corrector for a gamma-powered distribution (with a carefully chosen gamma). Our work thus provides a lens to theoretically understand CFG by embedding it in a broader design space of principled sampling methods.

Published in Transactions on Machine Learning Research (08/2025)
Classifier-Free Guidance is a Predictor-Corrector
Arwen Bradley
Apple, Cupertino CA,, USA
Preetum Nakkiran
Apple, Cupertino CA,, USA
Reviewed on OpenReview: https: // optp//veee.rreeeeee net/ forum? id= 2S22
Abstract
We investigate the theoretical foundations of classifier-free guidance (CFG). CFG is the dominant method of conditional sampling for text-to-image diffusion models, yet unlike other aspects of diffusion, it remains on shaky theoretical sooting, In this paper, we first show that CFG interacts differently with DDPM (Ho et al., 2020) and DDIM (Song et al., 2021), and neither sampler with CFG generates the gamma-powered distribution p(z]cc"p(())' Then, we clarify the behavior of CFG by showing that it is kind of predictor-corrector method (Song et al., 2020) that alternates between denoising and sharpening, which we call predictor-correct equivalent to combining DDIM predictor for the conditional distribution together with a Langevin dynamics corrector for a gamma-powered distribution (with a carefully chosen gamma). Our work thus provides a lens to theoretically understand CFG by embedding it in a broader design space of principled sampling methods.
guidance (PCG), We prove that in the SDE limit, CFG is actually
1 Introduction
Blassfferr-froe-guidee (CFG) has become an essential part of modern diffusion models, especially in text-to- image applications (Dieleman, 2022; Rombach et al., 2022; Nichol et al., 2021; Podell al., 2023). CFG is intended to improve conditional sampling, e.g. generating images conditioned on a given class label text prompt (Ho & Salimans, 2022). The traditional (non-CFG) way to do conditional sampling is simply train model for the conditional distribution p(x|c), including the conditioning the model. In the context of diffuusio,, this means training model to approximate the conditional score (x),tcc)== == V. log Pt (x/cc at every noise level f, and sampling from this model via a standard diffusion sampler (e.g. the sampler from Ho et al. (2020) known as DDPM, for Denoising Diffusion Probabilistic Models). Interestingly, this standard way of conditioning usually does not perform well for diffusion models, for reasons that are unclear. In the text-to-image case for example, the generated samples tend to be visually incoherent. and not faithful to the prompt, even for large-scale models (Ho & Salimans, 2022; Rombach et al., 2022).
auxiliary input to
Guidance methods, such CFG and its predecessor classifier guidance (Sohl-Dickstein et al., 2015; Song et al., 2020; Dhariwal & Nichol, 2021), are methods introduced to improve the quality of conditional samples. During training, CFG requires learning & model for both the unconditional and conditional scores (V, log Pr (x) and V, log pr (x)c)). Then, during sampling, CFG runs any standard diffusion sampler (like DDPM or the sampler from Song et al. (2021) known as DDIM, for Denoising Diffusion Implicit Models), but replaces the true conditional scores with the "CFG scores"
[(z e.) == V logpe(zlc) + (1 V (o))
(1)
for some 7 > 0. This turns out to produce more coherent samples in practice, and so CFG is used almost modern text-to-image diffusion models (Dieleman, 2022). common intuition for why CFG works starts by observing that Equation (1) is the score of a namma-powerss distribution:
(xlc) px(x))="),(()))",
(2)

---

Published in Transactions on Machine Learning Research (08/2025)
CFGDDPM
y= 2
y=5
y= 10
y = 20
PCGDDIM
7'=3
/=9
= 19 y=
Figure CFG vs. PCG. We prove that the DDPM variant of classifier-free guidance (top) equivalent to a kind of predictor-correcte method (bottom), in the continuous limit. We call this latter method "predictor-correctorr guidance" (PCG), defined in Section 4.1. The equivalence holds for all CFG guidance strengths 7, with corresponding PCG parameter Y (27 1), as given in Theorem 4.4. Samples from SDXL with prompt: "photograph of a cat cating sushi using chopsticks". Note, samples provided only for qualitative comparison; our formal results are in Sections 3 and 4.
which is also proportional (())), by Bayes' rule. Raising (clz)) to power y 1 sharpens the classifier around its modes, thereby emphasizing the "best" exemplars of the given class or other conditioner at each noise level. Applying CFG that is, running a standard sampler with the usual score replaced by the CFG score at each denoising step is supposed to increase the influence of the conditioner on the final samples.
However, CFG does not inherit the theoretical correctness guarantees of standard diffusion (Chen et al., 2022), because the CFG scores do not necessarily correspond a valid diffusion forward process. The fundamental issue (which is known, but still worth emphasizing) that (zly( not the same the distribution obtained applying a forward diffusion process to the gamma-poweres data distribution (xle)) That is, letting Ne(p] denote the distribution produced by starting from distribution p and running the diffusion forward process up to time t, we have
(x/() = N(Ppo(=)c) N=]po
(3)
# No (oppll(llo
(4)
Since the distributions ( do not correspond to any known forward difffusion process, we cannot properly interpret the CFG score (1) as a denoising direction; and using the CFG score in sampling loop like DDPM or DDIM is no longer theoretically guaranteed to produce sample from (2ye( or any other known distribution. The theoretical foundations of CFG are thus unclear, and important. questions remain
2

---

Published in Transactions on Machine Learning Research (08/2025)
open: Is there principled way to think about why CFG works? And what does it even mean for CFG to "work" - what problem is CFG solving? We make progress towards understanding the foundations of CFG, and in the process we uncover several new aspects and connections to other methods.
1. First, disprove common misconceptions about CFG by counterexample. We show that the DDPM and DDIM variants of CFG can generate different distributions, neither of which is the gamma-powered data distribution Po(z))""((zz)))
2. We define a family of methods called predictor-corrector guidance (PCG), natural way to approximately sample from gamma- gamma-powered distributions. PCG alternates between denoising steps and Langevin dynamics steps. In contrast to Song et al. (2020), where the predictor and corrector both target the conditional distribution, PCG the prodictor anncals using conditional diffusion paths, while the corrector mixes towards the (sharpened) nanma--powered dist.iiuttio..
3. We prove that in the contimuuu--time limit, CFG is equivalent to PCG with a careful choice of parameters. This gives a principled way to interpret CFG: it is implicitly an annealed Langevin dynamics.
4. For demonstration purposes, we implement the PCG sampler for Stable Diffusion XT. and observe that it produces samples qualitatively similar to CFG, with guidance scales determined by our theory. Further, we explore the design axes exposod by the PCG framework, namely guidance strength and Langevin iterations, to clarify their respective effects.
We do not intend to propose PCG as a practical sampling method (since with certain parameters it is equivalent to CFG, but far less efficient)) but rather as a tool for understanding CFG. The main goal of this paper is demonstrate theoretical equivalence between CFG and PCG, in order place CFG on a firmer theoretical footing as kind of annealed Langevin dynamics the gamma-poweres distribution.
2 Preliminaries
We adopt the continuous-timm stochastic differential equation (SDE) formalism of diffusion from Song et al. (2020). These contimoous-time results can be translated to discrete-time algorithms; we give explicit algorithm descriptions for our experiments.
2.1 Diffusion Samplers
Forward diffusion processes start with conditional data distribution (z(z)c and gradually corrupt it with Gaussian noise, with pr(c)c) denoting the noisy distribution al time 4. The forward diffusion runs up lo a time T large enough that pr is approximately pure noise. To sample from the data distribution, we first sample from the Gaussian distribution pr and then run the diffusion process in reverse (which requires an estimate of the score, usually learned by neural network). A variety of samplers have been developed perform this reversal. DDPM (Ho et al., 2020) and DDIM (Song al., 2021) are standard samplers that correspond to discretizations of reverse-SDE and reverse- revErse-ODE, respectively. Due to this correspondence, reverse-SDE as DDPM and the ODv as DDIM for short. The forward process, reverse-SDE, and equivalent reverse- Orerss--DDE (Song et al., 2020) for the variance variance-preserving preserving (VP) (Ho et al., 2020) conditional diffusion are
refer to the
1 Brrdl + / B,dw.
Forward SDE: dr
(5)
2
DDPM SDE: dr = Bex - 3 V logre()le)dtt + / B,d
(6)
2
1
1 3. (xgp)
DDIM ODE dr
2 Bax dt
2
where is a Brownian motion, is a Brownian motion with time reversod, and {B)TT the variance- preserving schedule defined in Ho et al. (2020) (8, is increasing in t). The unconditional version of each
3

---

Published in Transactions on Machine Learning Research (08/2025)
sampler simply replaces pe(a)c) with pz(z). Note that the score Vz pe appears in both (6) and (7). Intuitively, the score points in direction toward higher probability, and so it helps to reverse the forward diffusion process. The score is unknown in general, but can be learned via standard diffusion training methods.
2.2 Classifier-Free Guidance
CFG replaces the usual conditional score Va logp((z)c) in (6) or (7) at each timestep with the alternative score Vz log (x)c). In SDE form, the CFG updates are
CFGDDPM : de BI - B B.V. (rledd + v Bidw
(8)
2
CFGDDIM : dr 1
B,x dt
81 B.V Po (x)e)dt
(9)
2
2
where V (x.()))))) (1 V log + 7 V log
2.3 Langevin Dynamics
Langevin dynamics (Rossky et al., 1978; Parisi, 1981) is another sampling method, which starts from an arbitrary initial distribution and iteratively transforms it into a desired one. Langevin dynamics (LD) is given by the following SDE (Robert et al., 1999)
dx = v log p(x)ddt + Vedw.
(10)
LD converges (under some assumptions) to the stcady-state p(x) (Roberts & Tweodie, 1996). That is, letting (z) denote the solution of LD at time s, we have lim, Pa(x)) p(z). Similar diffusion sampling, LD requires the score of the desired distribution p (or learned estimate of it).
3 CFG versus gamma-sharpening
We first observe that the exact definition of CFG matters: specifically, the sampler with which it used. Without CFG, DDPM and DDIM generate equivalent distributions. However, we will prove that with CFG, DDPM and DDIM can generate different distributions. We provide informal statements of our claims below, to convey the main intuitions. The formal statement and proof is provided in Appendix A.1, as Theorem A.3.
Theorem 3.1 (CFGDDIM + CFGDDPM: informal). Consider generating sample via CFG using either DDPM or DDIM as the sampler. There erists a particular data distribution for which the generations of CFG differ depending on the choice of sampler. In particular, for large guidance scale 7 > 1, CFGDDPM and CFGDDPM approcimately generale the following distribuuioos, respectively:
Pddpm R ~N(0,T=)), Padim Re NO,2277).
Next, we disprove the misconception that CFG generates the gamma- camma-powered distribution data:
Theorem 3.2 (CFG + gamma-sharpening, informal). There exists a dala distribution po such that for any distribution (o,x( x po(x)) pp())e)
> 0, neither CFGDDIM nor CFGDDPM produces the
Both claims are proven using a simple Gaussian construction, as outlined in the next section.
3.1 Counterexample
We first present setting that allows us enactly solve the ODE and SDE dynamics of CFG closod-form, and hence find the exact distribution sampled by rumning CFG. This would be intractable in general, but it is possible for specific problem, as follows.
4

---

Published in Transactions on Machine Learning Research (08/2025)
Counterexample y
Counterexample y y
3.5
(o(x) AT0, 2)
A.li -M
Ao(M()) MtD.
Ps, (xic) =A10. gix)
Pl, [x, 3.1).
1.2
CFG DOM
DDIM
2.5
CFG DDPM
CFG CFG_DDPM
1.0
2.0
0.8
1.5
0.6
1.0
0.4
0.2
0.0
0.0
-6
-6
-4
Figure 2: Counterexamples: CFGDDIM * CFGDDPM gamma-powered. CFGDDIM and CFGDDPM do not generate the same output distribution, even when using the same score function. Moreover, neither generated distribution is the gamma-powered distribution (zlc). (Left) Counterexample 1 (section 3.1): CFGDDIM yields a sharper distribution than CFGDDPM, and both are sharper than (xlc()) (Right) Counterexample 2 (section 3.2): Neither CFGDDIM nor CFGDDPM yield even a scaled version of the camma--powers distribution po,, (x = (),, The CFGDDPM distribution is mean-shifted relative to po,x(z/c), The CFGDDIM distribution mean-shifted and not even Gaussian (note the asymmetrical shape).
Consider setting where po( and po(alee 0) both zero-mean Gaussians, but with different variances. Specifically, (xr c) are jointly Gaussian under po, with p(c) = (O 1), po(2/c) + 1). Therefore
po(x) = _O,
Po(a/c= 0) (00,
(x/c 0) = NNO0, 7+1
(11)
solve CFGDDIM (9) and CFGDDPM (8) analytically; that is, we solve initial-value problems for the reversed dynamics to find the sampled distribution of &, in terms of the initial-value ær. Applying these results to = 0 and averaging over the known Gaussian distribution of zT gives the exact distribution of dp that CFG samples. The full derivation is in Appendix A.1. The final CFG-sampled distributions are:
For this problem,
to N (0,
2 22-2-2
CFGDDPM
(12)
27
CFGDDIM: 4o (0.22~7)
(13)
This shows that for any 1, the CFGDDIM distribution is sharper than the CFGDDPM distribution, and both are sharper than the gamma-powered distribution (xrlc = 0). (Even though the distributions all have the same mean, their different variances make them distinct.) In fact, for y > 1, the variance of DDPM-CFG is approximately 2y-1: which about half the variance of (a,y(1/c = 0). In Figure 2, we compare the CFGDDIM and CFGDDPM distributions - sampled using an exact denoiser (see Appendix A.6) within DDIM/DDPM sampling loops to the unconditional, conditional, and camma-powered distributions.
3.2 Counterexample 2
In the above counterexample, the CFGDDIM, CFGDDPM: and gamma-powered distributions had different variances but the same Gaussian form, might wonder whether the distributions differ only by scale

---

Published in Transactions on Machine Learning Research (08/2025)
factor in general. This is not the case, qualitative differences, in particular a symmetry-breaking behavior of CFG.
in different counterexample that reveals greater
In Counterexample 2, the unconditional distribution is a Gaussian mixture with two clusters with equal weights and variances, and means at =μc
CE (0,1), p(c=0)= 2
po(a|c= 0) = =W(-,, 1), Po(a 1) (N,11)
po(#) = ( 0) p
(14)
If the means are sufficiently separated (n > 1), then the poweeed distribution for 7 2 is approximately equal to the conditional distribution, i.c (x,,( R po(zlc), due to the nean-zcro-probabiiiy valley between the conditional densities (see Appendix A.2). However, for sufficiently high noise the clusters begin to merge, and (xlc)) # pe(x)c). In particular, (x,y( is approximately Gaussian with mean Ip, but (x/c) + pt(x)c) is not. Although we cannot solve the CFG ODE and SDE in this case, we empirically sample the CFGDDIM and CFGDDPM distributions using exact denoiser and compare them to the gamma--pwered distribution. In particular, see that neither CFGDDIM nor CFGDDPM is Gaussian with mean łu, hence neither is scaled version of the camma-pooweeed distribution. The results are shown in Figure 2. Concurrent work by Chidambaram et al. (2024) offers a theorctical analysis confirming our qualitative observations the two-cluster case, while Wu et al. (2024) conduct an analysis of similar GMM settings.
4 CFG as predictor-corrector
The previous sections illustrated the subtlety in understanding CFG. We can now state our main structural characterization, that CFG is equivalent to special kind of predictor-commedtor method (Song et al., 2020).
4.1
Predictor-Corrector Guidance
As warm-up, suppose actually wanted to sample from the ganma-powered distribution:
(x(e) p(x)'=(p((p)) "((x)))"
(15)
A natural strategy is to run Langevin dynamics w.r.t. Py, This is possible in theory because can compute the score of Py from the known scores of p(z) and p(zle):
V (x)cc) = (1 >) logp(x) + V. logp(x/)c)
However this won't work in practice, due to the well-known issue that vanilla Langevin dynamics has impractically slow mixing times for many distributions interest (Song & Ermon, 2019). The usual remedy for this is to use some kind of annealing, and the success of diffusion teaches us that the diffusion process defines a good annealing path (Song et al., 2020; Du et al., 2023). Combining these ideas yields an algorithm remarkably similar to the predictor-correctt methods introduced in Song al. (2020). For example, consider the following diffusion-like iteration, starting from FT NN σT) t T. At timestep f,
Predictor: Take one diffusion denoising step (e.g. DDIM or DDPM) w.r.t. pr(x)c), using score V. log (x)c),, to move to time t' =t At.
Corrector: Take K 2 1 Langevin dynamics steps w.r.t. distribution Perm using the score (=l (1 v, (x) + logpe
It is reasonable to expect that running this itcration down to t 0 will produce a sample from approximately P-y (z(c), since the iteration can be thought of as a type of annealed Langevin dynamics, with time / playing the role of temperature (c.f. Remark 4.1 below). We name this algorithm predictor-corrette guidance (PCG). Remarkably, it turns out that for specific choices of the denoising predictor and Langevin step size, PCG is equivalent (in the SDE limit) CFG, but with a different y. We will formalize and prove this in the subsequent section.
6

---

Published in Transactions on Machine Learning Research (08/2025)
DDPM
(logp(xle\pp/TT
denolse
L
Langevin dynamics on Pa-a cÿ Pr-a
DDIM score Vl(ee(()le)
sharpen
Xp-Ar
denoise
Figure 3: CFG is equivalent to PCG for particular parameter choices.
Algorithm PCGDDIM Theory. (See Algorithm for practical implementation.)
Require: Conditioning guidance weight 2 0.
Require: Schedule Br == B()) from Song et al. (2020), number of Langevin iterations K E N.
1: 31 _N(00 I).
for (t (t=11 At: t 2 0; t At) do
St4rr v log Pt+ Pr (214arl)
24 TitAt + ( S
DDIM step for Pt+ (x
:: B,AL
Langevin step size, matching DDPM noise scale B,
6:
for k = I...KK do
> Langevin dynamics on (x,y(
7:
17 ~~N(0,11 Ia)
8:
$1,7 = (1 - -7)) v logpi (x:) + V log
3t 4- de + 58.,7 Ven
10: end for
11: end for
12: Return: 20
Remark 4.1 (Langevin Dynamics). The standard annealed Langevin dynamics corresponds a predictor- corrector where the predictor is an identity function: only reduces the "temperature" 6+ I At without changing the current sample Fz. The iteration above uses different predictor, which moves 2, along the diffusion path.
Remark 4.2 (Mixing). Why do we expect PCG to sample from approximately (#/c)? For the same reason we expect annealed Langevin dynamics to work: in the limit many Langevin steps (K - o0), the corrector will fully mix to the distribution pr,, each time t'. In reality we may take only K Langevin stop each iteration, which will at least move the sample distribution towards the target distribution pr,, (x(c), even if it does not fully mix.
Remark 4.3 [Predictor-Correcttr]] Corrector). PCG technically differs from the predictor-corrett corrector algorithms in Song et al. (2020), bocause our predictor and corrector operate w.r.t. diffcrent distributions (pr vs. Pt,y)- (Similarly, some early works with classifier guidance (Chen al., 2023; Bansal et al., 2023) also used different predictor and corrector distributions.) However, conceptually all of these methods can be thought of as variants of annealed Langevin dynamics (as described in Remark 4.1), with different annealing choices.
4.2 SDE limit of PCG
Consider the version of PCG defined Algorithm 1, which uses DDIM as predictor and particular LD on the gamma-powerod distribution as corrector. We take K = 1, i.c. a single LD step per iteration. Crucially, we set the LD step size such that the Langevin noise scale exactly matches the noise scale of a (hypothetical) DDPM step at the current time (similar to Du et al. (2023)). In the limit Al 0, Algorithm becomes

---

Published in Transactions on Machine Learning Research (08/2025)
the following SDE (see Appendix B):
dx = (DDIM () + ALDG(1,447)
(16)
Predictor
Corrector
APCGDDIM (2,
(17)
where ADDIM t) = (x (ogpe(rle))dt
2
ALD =- V Pt, + BBdw.
Above, (DD t) is the differential of the DDIM ODE (7), i.e. the ODE can be written as da ADDIM(,, t). And 1,, 1, where stands for "guidance", is the limit as At of the Langevin dynamics step in PCG, which behaves like differential of LD (see Appendix B).
We now show that the PCG SDE (17) matches CFG with DDPM, but with different 7. In the statement, ACFGDDPM (x, (z) denotes the differential of the CFGDDPM SDE (8), similar to the notation above. This result is trivial to prove using our definitions, but the statement itself appears to be novel.
Theorem 4.4 (CFG is predictor-correcloon In the SDE limil, CFG with DDPM is equivalent to predictor- corrector. That is, the following differentials are equal:
t, 2 1) ACFGDDPM (I, (, = ADDIM(I, t) (L,
=: A.PAPDDOIIM (z, t, 27 -1
(18)
Notably, the guidance scales of CFG and the above Langevin dynamics are not identical.
Proof.
APCGOOIM t, 7)
ADDIM(I, (2,
(1- (o-
(x +
Be
2
1 3 + log))()lcddl + Bidi
2
-2 B.rSt 3V log (xlc) + Vidw
(2,
where 7 :=
5
Discussion
5.1
Open Questions and Limitations
There are number of other questions left open by our work. First, we study only the stochastic variant of CFG (i.e. CFGDDPM), and it is not clear how to adapt our analysis to the more commonly used deterministic variant (CFGDDIM). This subtle because the two CFG variants can behave very differently in theory, but appear to behave similarly in practice. It thus open to identify plausible theoretical conditions which explain this similarity?; we give suggestive experiment in Figure 5. More broadly, it open to find more explicit characterizations of CFG's output distribution, in terms of the original p(x) and p(z/c).
Notice that taking interleaved with LD (c.g. Karras et al. (2022)). This because for 1, CFGPDPM is just DDPM, Theorem 4.4 reduces to: ADDPM(z, t) ADDIM(,, ALDg(a, 2Curiously, CFGDOM the correct probability-fio ODE for CFGDDPM if and only if the intermediate distribution time is Pt.y- However we know this the distribution in general, from Section
in Theorem 4.4 recovers the standard fact that DDPM is equivalent, in the limit, to DDIM
8

---

Published in Transactions on Machine Learning Research (08/2025)
Finally, presented PCG primarily tool to understand CFG, not practical algorithm in itself. Nevertheless, the PCG framework outlines broad family of guided samplers, which may be promising to explore in practice. For example, the predictor can be any diffusion denoiser, including CFG itself. The corrector can operate on any distribution with a known score, including compositional distributions as Du et al. (2023), the sharpening distribution proposed by Karras el al. (2024a), which replaces the unconditional score with the score of a "weaker" model, or any other distribution that might help sharpen or otherwise improve on the conditional distribution. Finally, the number of Langevin steps could be adapted to the timestep, similar to Kynkäänniemi et al. (2024), or alternative samplers could be considered (Du et al., 2023; Neal, 2012; Ma et al., 2015).
5.2 Stable Diffusion Examples
CFG
PCG
CFG
PCG
Guidance Strength
Guidonce Strength
1.5
1.5
3
Langevin Steps:
5
10
Langevin Steps:
10
Figure 4: Effect of Guidance and Correction. Each grid shows SDXL samples using PCGDDIM, the guidance strength y and Langevin iterations K are varied. Left: "photograph of a dog drinking coffee with his friends". Right: "a tree reflected in the hood of a blue car". (Zoom in to view).
We include several examples running predictor-corrector guidance on Stable Diffusion XL. (Podell et al., 2023). These serve primarily to sanity-check our theory, not as suggestion for practice. For all experiments, we use PCGDDIM implemented explicitly in Algorithm 2. Note that PCG offers more flexible design space than standard CFG; e.g. we can run multiple corrector steps for each denoising step to improve the quality of samples (controlled by parameter K in Algorithm 2).
CFG vs. PCG. Figure 1 illustrates the equivalence of Theorem 4.4: we compare CFGDDPM with guidance Y to PCGDDIM with exponent 7 == (2> - 1). We run CFGDDPM with 200 denoising steps, and PCGDDIM with 100 denoising steps and K 1 Langevin step per denoising step. Corresponding samples appear to have qualitatively similar guidance strengths, consistent with our theory.
Effects of Guidance and Corrector. In Figure 4 we show samples from PCGDDIM. varying the guidance strength and Langevin iterations (i.e. parameters 7 and K respectively in Algorithm 2). We also include standard CFGDDIM samples for comparison. All samples used 1000 denoising steps for the base predictor. Overall, we observed that incrcasing Langevin steps tends improve the overall image quality, while incrcasing guidance strength tends to improve prompt adherence. particular, sufficiently many Langevin steps can sometimes yield high-quality conditional samples, even without any guidance (7 = 1); see Figure in the
9

---

Published in Transactions on Machine Learning Research (08/2025)
Method
?
7
1.1
Y
1.3
7 = 1.5
CFGDDPM
5.99
3.90
2.71
3.33
CFGDDIM
7.11
4.61
2.55
2.47
PCGDDIM (K
5.87
4.72
3.37
3.16
PCGDDIM (K 3)
4.79
3.49
3.71
6.10
PCGDDIM (K = 5)
4.51
3.35
4.87
8.86
Table 1: FID Scores on ImageNet (lower better), using DDPM, DDIM, and PCG samplers. We vary the guidance weight y and the number of LD steps K. We used direct discretization of Algorithm I for y= 1.3, 1.5 (all K) and an alternative discretization described in Appendix for y = 1.0, 1.1 (all K) (we found that the latter empirically performs better low 7). Complete FID, FD-DINOV2, and Inception Scores provided for both discretizations in Tables 2 and 3.
Appendix for another such example. This is consistent with the observations of Song et al. (2020) on unguided predictor-corrector methods. It is also related to the findings of Du et al. (2023) on MCMC methods: Du et al. (2023) similarly use annealed Langevin dynamics with reverse-diffusion annealing, although they focus on general compositions of distributions rather than the specific gamma-powered distribution of CFG.
Notice that in Figure 4, increasing the number of Langevin steps appears to also increase the "effective" guidance strength. This is because the dynamics does not fully mix: one Langevin step (K = 1) does not suffice to fully converge the intermediate distributions to Pt.y
5.3 ImageNet Experiments
For completeness, we also include experiments comparing variants PCG and CFG on ImageNet (Rus- sakovsky et al., 2015). Table 1 shows FID scores (Heusel et al., 2017) on ImageNet, using EDM2 pre- trained diffusion models (Karras al., 2024b). Metries are calculated using 50,000 samples and 200 sampling steps, generated using EDM2 checkpoints edm22-ing5122--2244483 02025 (conditional) and -ott--xxssumooo221144-4-5
(unconditional).
For all samplers, there is a "sweet spot" of guidance scale 7; quality starts to degrade when 7 is too low or too high. This well-known behavior of CFG, and also occurs for PCG.
For PCG methods, increasing the number of LD steps does not always improve FID depends on the guidance scale. More L.D steps helps at 7 = 1.1 for example, but starts to hurt higher y. This may seem surprising, but is explained by the same mechanism we in Figure 4: increasing the LD steps corresponds to increasing the "effective" guidance strength, because single step does not fully mix the Langevin dynamics.
CFGDDPM and PCGDDIM (LD=1) have different optimal guidance scales The FID of CFGDDPM is minimized at y x 1.3, while PCGDDIM is minimized at 2 1.5. This is roughly in line with Theorem 4.4, where the equivalence between PCG and CFG requires rescaling 7.
6 Conclusion
We have shown that while CFG is not a diffusion sampler on the gamma-powered data distribution po(z)' "ore)'=Po((ee (=((=))) it can be understood is DDIM denoiser, and the corrector at each step t is one step of Langevin dynamics the gamma- powered noisy distribution (x))1="pp((at) (xlc))",, with 7 = (2y 1). Although Song et al. (2020)'s Predictor-Corroctoo algorithm has not been widely adopted in practice, perhaps due to its computation expense relative to samplers like DPM++ (Lu et al., 2022b), turns out to provide lens to understand the unreasonable practical success of CFG. On a practical note, PCG encompasses a rich design space of possible predictors and correctors for future exploration, that may help improve the prompt-alignment, diversity, and quality of diffusion generation.
particular kind of predictor-corrector,, where the predictor
10

---

Published in Transactions on Machine Learning Research (08/2025)
7 Extensions and Impact
Since the original release of this preprint, a number of other works have built upon it, filling in some of the questions we left open, and proposing interesting new directions. For example, subsequent to our work, there have been theoretical advancements in understanding guidance (Chidambaram et al., 2024), geometric temporing (Chchab et al., 2024), and denoising Markov models (Chchab et al., 2024). Our PCG framework and discussion of "misconceptions about. CFG" have also helped inform the development of many new practical methods to improve conditional sampling (unlike PCG, which was originally intended primarily as a theorctical tool, and perhaps an inspiration for new methods). Some works have explored Sequential Monte Carlo/ Feynman-Kac approaches to enable precise sampling from desired distribution (as CFG fails to do): Lee et al. (2025); Thornton et al. (2025); Skreta et al. (2024; 2025). Others have developed guidance methods for flow and bridge models: Jiao et al. (2025); Fan et al. (2025); Zhu et al.. Finally, many new methods have been proposed as alternatives to CFG (helping to address some of its shortcomings discussed in this paper): Gao et al. (2025); Xia et al. (2024); Kadkhodaie et al. (2024); Noroozi et al. (2025); Zhou al. (2025).
References
Arpit Bansal, Hong-Min Chu, Avi Schwarzschild, Soumyadip Sengupta, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Universal guidance for diffussio models. In Proceedings of the IEEE/CV Conference on Compuler Vision and Pallern Recognition, pp. 843 852, 2023.
Omar Chehab, Anna Korba, Austin Stromme, and Adrien Vacher. Provable convergence and limitations of geometric tempering for langevin dynamics. arXiv preprint arXiv:2410.09697 2024.
Hansheng Chen, Jiatao Gu, Anpei Chen, Wei Tian, Zhuowen Tu, Lingjie Liu, and Hao Su. Single-stage diffusion nerf: A unified approach to 3d generation and reconstruction. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 2416-2425, 2023.
Sitan Chen, Sinho Chewi, Jerry Li, Yuanzhi Li, Adil Salim, and Anru Zhang. Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions. In The Fleventh International Conference on Learning Representations, 2022.
Muthu Chidambaram, Khashayar Gatmiry, Sitan Chen, Holden Lec, and Jianfeng Lu. What does guidance do? fine-grained analysis in simple setting, arXiv preprinl arXiv:2409. 13074, 2024.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-877 8794, 2021.
Sander Dieleman. Guidance: a cheat code for diffusion models, 2022. URL https://benanng .ith.b. 1o/ 2022/05/22 20:000////////// /ttp////////////eeeeeee//// gww./ncdddcccccmm html.
Yilun Du, Conor Durkan, Robin Strudel, Joshua B Tenenbaum, Sander Dieleman, Rob Fergus, Jascha Sohl-Dickstein, Arnaud Doucet, and Will Sussman Grathwohl. Reduce, reuse, recycle: Compositional generation with energy-based diffusion models and meme. In International conference on machine learning, pp. 8489 8510. PMLR, 2023.
Weichen Fan, Amber Yijia Zheng, Raymond Ych, and Ziwei Liu. Cfg-zero*: Improved classifier-free guidance for flow matching models. ar preprinl arXiv:2503. 18886, 2025.
Zhengqi Gao, Kaiwen Zha, Tianyuan Zhang, Zihui Xue, and Duane S Boning. Reg: Rectified gradient guidance for conditional diffusion models. arXiv preprint
18865, 2025.
Martin Hensel, Hubert Ramsaner, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. or preprint ar Xix:2207..12556 2022.
11

---

Published in Transactions on Machine Learning Research (08/2025)
Jonathan Ho, Ajay Jain, and Picter Abbeel. Denoising diffusion probabilistie models. Advonces in newral informalion processing systems, 33:6840-660 (1884066551,, 2020.
Guanlong Jiao, Biqing Huang, Kuan-Chieh Wang, and Renjie Liao. Uniedit-flow: Unleashing inversion and editing in the era of flow models. arXie preprint 13100 2025.
Alexia
Ke Li, Rémi Piché- Taillefer, Tal Kachman, and Ioamnis Mirliagkas. Gotta go fast when generating data with score-based modols. arXiv preprint 100.:20000,00 2021.
Zahra Kadkhodaie, Stéphane Mallat, and Eero P Simoncelli. Feature-guided score diffusion for sampling conditional densities. orXiv preprint ar Xrv: 10. 11646, 2024.
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space diffusion- generalive models, 2022.
Tero Karras, Miika Aittala, Tuomas Kynkäänniemi, Jaakko Lehtinen, Timo Aila, and Samuli Laine. Guiding a diffusion model with bad version of itself. arXiv preprint arXiv:2406.0r 200004406002077, 2024a.
Tero Karras, Milka Aittala, Jaakko Lehtinen, Janne Hellsten, Timo Aila, and Samuli Laine. Analyzing and improving the training dynamics of diffusion models. In Proc. CVPR, 2024b.
Tuomas Kynkäämiemi, Miika Aittala, Tero Karras, Samuli Laine, Timo Aila, and Jaakko Lehtinen. Applying guidance a limited interval improves sample and distribution quality in diffusion models. arXiv preprint urXiv:2404.0772 2024.
Cheuk Kit Lee, Paul Jeha, Jes Frellsen, Pietro Lio, Michael Samuel Albergo, and Francisco Vargas. Debiasing guidance for discrete diffusion with sequential monte carlo. arXie preprint orXiw:2502.06079 2025.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems, 35:5775 5787, 2022a.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver+++: Fast solver for guided sampling of diffusion probabilistic models. arXiv preprial or Niw:2211.01099 2022b.
Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Lue Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11461-11471, 2022.
Yi-An Ma, Tianqi Chen, and Emiiy Fox. A complete recipe for stochastic gradient meme. Advances in neural information processing systems, 28, 2015.
Radford M Neal. Mome using hamiltonian dynamics. arXiv preprint arXic: 1901, 2012.
Alex Nichol, Prafulla. Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, 1lya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2112.1074 2021.
Mehdi Noroozi, Alberto Gil Ramos, Luca Morreale, Ruchika Chavhan, Malcolm Chadwick, Abhinav Mehro- tra, and Sourav Bhattacharya. Guidance free image editing via explicit. conditioning. arXio preprint arXiv:2503.1155 7593, 2025.
Giorgio Parisi.
Correlation functions and computer simulations. Nuclear Physics B, 180(3):378 384, 1981.
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint XXX:::2307.0195, 2023.
Christian P Robert, George Casella, and George Casella. Monte Carlo statistical methods, volume Springer, 1999.
12

---

Published in Transactions on Machine Learning Research (08/2025)
Gareth O Roberts and Richard L Tweedie. Exponential convergence of langevin distributions and their discrete approximations. 1996.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10684 10695, 2022.
Peter J Rossky, Jimmie D Doll, and Harold L Friedman. Brownian dynamics as smart monte carlo simulation. The Journal of Chemical Physics, 69(10):4628 4633, 1978.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 115(3):211 211-252,, 2015. doi:
Marta Skreta, Lazar Atanackovic, Avishek Joey Bose, Alexander Tong, and Kirill Neklyudov. The superposition of diffusion models using the Itô density estimator. arXiv preprint orXic:2412.177000 2024.
Marta Skreta, Tara Akhound-Sadegh, Viktor Ohancsian, Roberto Bondesan, Alán Aspuru-Guzik, Arnand Doucet, Rob Brekelmans, Alexander Tong, and Kirill Neklyudov. Feynman-kac correctors in diffusion: Annealing, guidance, and product of experts. ar preprint 2xx2:2200.002819, 2025.
Jascha Sohk-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya. Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference machine learning, pp. 2256-226662 2265. PMLR, 2015.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2021. URL
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13450 2020. URL https://arxi:: org/pdf /3/////.....1......... pdf.
James Thornton, Louis Béthune, Ruixiang Zhang, Arwen Bradley, Preetum Nakkiran, and Shuangfei Zhai. Composition and control with distilled energy diffusion models and sequential monte carlo. arXiv preprint MxXi::550.112880 2025.
Yuchon Wu, Minshuo Chen, Zihao Li, Mengdi Wang, and Yuting Wei. Thcorctical insights for diffusion guidance: A case study for gaussian mixture models. arXio preprinl K5w:2403.01633 2024.
Mengfei Xia, Nan Xue, Yujun Shen, Ran Yi, Tieliang Gong, and Yong-Jin Liu. Rectified diffusion guidance for conditional generation. arXi preprint Xrxin:24 10. 10.1101.11111 18737, 2024.
Zhenyu Zhou, Defang Chen, Can Wang, Chun Chen, and Siwei Lyu. Dice: Distilling classifier free guidance into text embeddings. arXiv preprint Xrx::22002. 03726, 2025.
Kaizhen Zhu, Mokai Pan, Yuexin Ma, Yanwei Fu, Jingyi Yu, Jingya Wang, and Ye Shi. A unified diffusion bridge framework via stochastic optimal control. In ICLR 2025 Workshop on Deep Generative Model in Machine Learning: Theory, Principle and Efficacy.
13

---

Published in Transactions on Machine Learning Research (08/2025)
Variance
1.25
2.00
:
CFG BFG_ y=
CFG CFG FFG_DDIM:PM (G)--
1.00
DMMM y
1.75
DP__DDPM: 2-21-
0.75
CFG DDIM: empirical
0.50
DDPM: ompirical
1.25
Variance
0.00
Variance
0.75
0.50
0.15
0.00
0.05
10 12
14
Figure 5: (Left) For Counterexample 1 (section 3.1), we plot the empirical and theoretical variance of the gamma-powered, CFGDDIM, and CFGDDPM distributions, over a range of values of 7. The theoretical predictions are given by equations (13) and (12), and the empirical distributions are sampled using an exact denoiser. This verifies the theoretical predictions and illustrates the decreasing variance from po,y CFGDDPM to CFGDDIM- (Right) For counterexample 3 (section A.3 with different choices of variance (a = and 2), compare CFGDDIM and CFGDDPM- Increasing the variance makes the two CFG samplers more similar. Also note that the CFGDDIM distribution is symmetric around the center cluster, but asymmetric around the side clusters. This experiment suggests that multiple clusters and greater overlap between classes can help symmetrize and reduce the difference between CFGDDIM and CFGDDPM
A 1D Gaussian Counterexamples
In this section, we formalize and prove Theorems 3.1 and 3.2. We will work with pariance-erploding (VE) process, so we begin by defining CFG for the VE process (analogous to the SDE (8) and ODE (9) for the VP process).
Definition A.1 (CFG, variance-expllddnng). Given data distribution Po(x, c). define the noisy distribution (z( for any t E R_ as the result of running the VE forward diffusion SDE dr du, up to time t, with initial distribution Po(x) at t = 0. Explicitly, this is the convolution Pt == * (0, Similarly define pr(rcc) == po(ccc * (0,
For all 7 E R and e E R, define the CFG SDEs for DDPM and DDIM, respectively,
CFGDDPM: dr V x Po, da 1 Va log
( (xle)dt du,
(19)
CFGDDIM
(20)
dt 2
where := p:)0===p)p))ppp)))>>>>>>))) and Z E R2 is the appropriate normalization constant.
The SDE and ODE above specify the dynamics of the CFG sampler in the VE setting. Specifically, in order to sample via. CFG, start with Gaussian sample Ør NO 7) for some T > 0, and then run the SDE or ODE from time t = T down to time t = 0, to generate sample zo. We call the resulting distribution of samples zo the generated distribution, and adopt the following notation:
Definition A.2 (CFG generated distributions). Denote by (xlpm (2)c), PDDIM (æ|c) the probability densities of the distributions generated by the CFGpDPM SDE (19), CFGDOIM ODE (20), respectively; that is, the solutions the SDE, ODE, respectively, time with initial conditions or _N(0,)) T), for any terminal times T R4 and conditioning E R.
(T)
14

---

Published in Transactions on Machine Learning Research (08/2025)
We will mainly be interested in the limits of the generated distributions we let the terminal time T + 00; which corresponds to allowing the diffusion process to fully mix. We can now formalize Theorems 3.1 and 3.2 as follows:
Theorem A.3 (Counterexample for which CFGDDIM + CFGDDPM + gamma--haarpninng.) the setting of Definitions A.1 and A.2, there ezists data distribution such that the distributions generated by CFGDDPM and CFGDDIM different, and neither is equal to the gamma-powered distribution. Specifically, define data distribution po po(z, c), over inputs E IR and conditioning E E.
(o(c) = Oc po(a|c) (x; 1).
In particular, (x, c) E RRR is jointly Gaussian and po(a/c= = 0) = =a;=0,, 1) Then, for all 2,, E RR the limiting generated distributions for c. = are:
lim ( 0) = 2; 0, 2 - 22-21 27
(21)
lim (olc 0) = W (r;0,2' 0,) T
(22)
Furlhermore, the gammaa-poered distribulion for c is given by po,, 0) -N(x;; 7 Therefore,
P + lim T T-
lim (T)
(alm 0) + pa, P 0). T-
Note that variance of the generated distributions depends on the guidance weight y (Equations 21 and 21), and is exponentially different between DDIM and DDPM when 7 > 1. The proof follows directly from the caleulations in the next section (A.1), which characterize the density evolution of CFG in this setting.
A.1 Counterexample 1
Countercxample (equation 11) has
p(c) (),
Po (z|c) .Nc,11
po(x) ~ 2(0,2)
Po(a/e 0) ().
The y-powered distribution
(0,y = (2=)))>>==
v0² x e e
11-7)02
(rt1)
2
N(O, 7
We consider simple variance-exploding (VE) diffusion defined by the SDE
dr = dw.
(23)
The DDIM sampler discretization of the reverse ODE
dr - (x),
dt
and the DDPM sampler is discretization of the reverse SDE
dr - (o dt dw.
15

---

Published in Transactions on Machine Learning Research (08/2025)
For CFGDDIM CFGDDPM, we replace the score with CFG score V (t)
At inference time we choose an imitial sample Fr ~ TT) and run CFGDDIM from t = T' - 0 to obtain a final sample zo- Note that the true distribution generated by the forward process in our setting is pr = (0, 1), which becomes close to our inference-time time terminal distribution (0, T) for large T. Taking the limit of T oo our setting thus corresponds to allowing the forward diffusion process to fully mix.
CFGDDIM For Counterexample 1, the CFGDDIM ODE has a closed-forn solution (derivation in section A.5):
dr
CFGDDIM
= V. loggot Pt, (x)
dt
7
+ (1-7)
Dt
2(1 + 2(2 +t)
(++ +2)
I V
(1+1))(T + 2)1-
That is, for a particular initial sample æT, CFGDDIM produces the sample 22 at time t. Evaluating at = 0 and taking the limit as T + oo yields the idcal denoised ty sampled by CFGDDIM given an initial sample ay:
21-7
CFGrrM 40
(IT)
2
(T+ 2)) 2)1
221-7
+ bTr
as T - x0.
T
To get the denoised distribution obtained by reverse-sampling with CFGDDIM, we need to average over the distribution of IT:
21-7
CFGOOM - = DN (0.2
N
T
which is equation 13 in the main text.
CFGDDPM CFGDDPM also has closed-form solution (derived in section A.5):
dr = Vv log Pt, (x) + du
? +
(1 71-
= (1 +)
dl + diō
(2+
(1+ + +t) 7
(t+1
1-27
T +
1-27
x(t) = JT + ()7 (2) (1+
- 27
iV
(7+2)
T +
g.
Similar to the CFGDDIM argument, we can obtain the final denoised distribution as follows:
21-7
T (T+1)
1-27
CFGDDPM (IT) = FT
+ 211
227-1
E
(1 (1+ (T>
T)
2n -
T T
21-r
2 22-27
XT +
as 000
27
22-7)
2 - 22-21 2y
CFGorrs
(ZT))
N 0,T
+
((,7)
T
27
2 - 22-27
N 0,
27
16

---

Published in Transactions on Machine Learning Research (08/2025)
which is equation 12 in the main text, and for > becomes approximately
E
F0
GoooM (xT)) = N
N0,,,
2
24
In Figure 5, confirm results (12, 13) empirically.
A.2 Counterexample 2
Counterexample 2 (11) is Gaussian mixture with equal weights and variances.
C {0,1), p(c= 0)
Po(oolc) ~ N(1 1), 11 -1, 10 1 po(zo) Po = 0) 200(20lc 1).
We noted in the main text that if y is sufficiently large enough that the clusters are approximately disjoint, and y 2 1, then (x/c) = po(z)c). To see this note that
po( ~ 100(0010)us20 + (1020
(x)() po(o)c) po(=))
(P((ale)
po(x))
px(xx)
x po(x)) (Isign(z)=p(e) = po(()c) for 2 1.
However, (xlc( * (x(ccc since the noisy distributions do overlap/interact.
We don't have complete closed-form solutions for this problem like we did for Counterexample We have the solution for conditional DDIM for the basic VE process dæ dw (using the results from the previous section):
dx V, log ((rle)
DDIM on pr(c)c): dt
2(1 (1(()) - -xt)
1+t 1 T
x(1) = ( + (xT (1))))
but otherwise have to rely on empirical results. We do however have access the ideal conditional and unconditional denoisers via the scores (Appendix A.6):
V logp((z(c) 2(1 + t) (p(0) x1)
VaP((T) Ec (ape(alc) Pr(x)
V logg((z)=
Pe(x)
A.3 Counterexample
We consider 3-cluster problem investigate why CFGDDIM and CFGDDPM often appear similar in practice despite being different. in theory. Counterexample 3 (11) is Gaussian mixture with equal weights and
17

---

Published in Transactions on Machine Learning Research (08/2025)
variances. We vary the variance to investigate its effect CFG.
C (0,1,22, p(c) 1
Vc
3
po(zol)) ~ N(u .),, 00 = -3, (11) = 0, u1²) =
1 Po(au) 3 po(zolc + 3 poo =] po 2).
We run CFGDDIM and CFGDDPM with 3, for o = 1 and σ = 2. Results are shown in Figure 5.
A.4
Generalization Example 4
30
ecect uncond Score cand soore
1.75
1.00
0.75
-10-
0.50
0.25
0
-4
-3
-2
Without CFG
With CFG
Palkic
Px, y
ddpm leamned score
ddpm leamned score
ddpm Exact score
ddpm
Figure 6: An example where guidance benefits generalization. (Top left) Conditional po(a/c 0) (purple) and unconditional Po(x:) (green) distributions for Example 4 (equation 24). The unconditional distribution is approximately uniform, while the conditional distribution for e 0 is GMM with several clusters with equal variances, and equal weights except for single "dominant" cluster with a higher weight. (Top right) We train small MLPs to predict the conditional and unconditional scores, with carly-stopping so that the fit imperfect. We plot the exact (orange) vs. learned (blue) conditional and unconditional scores: the unconditional scores are learned accurately, while the conditional scores are learned accurately near the dominant cluster but poorly elsewhere. (Bottom left) We sample with DDPM on the conditional distribution (no guidance) using learned scores (blue) vs. exact scores (orange). We expect DDPM to generate the conditional distribution Po(a1e= = 0) (purple). However, DDPM-witt--eaaeed-com samples less accurately away from the dominant cluster (where the learned scores are inaccurate) (compare the increased blue vs. orange sampling in low-probability regions). (Bottom right) With guidance
than
= 3, Po,y (red) and both samplers concentrate around the dominant cluster (where the learned scores are accurate), reducing the generalization gap between the learned and exact models.
18

---

Published in Transactions on Machine Learning Research (08/2025)
We consider multi-chuster problem to explore the impact of guidance on generalization:
po(x) (N0 10)
po(x/c= = 0) ~ Σ wnNiuun
(24)
(-3, -2.,- 2, -15, -0.5 0, 0.0.5 1,15,,,,,
0² 0.0476
Vi #6 w6 = 0.446
0 0= 0.1
Note that the unconditional distribution wide enough to be essentially uniform within the numerical support of the conditional distribution. The conditional distribution is a GMM with evenly spaced clusters of equal variance, and all equal weights, except for "dominant" cluster in the middle with higher weight. The results are shown in Figure 6.
A.5 Clomed-form ODE/SDE solutions
First, want to solve equations of the general form t aa() + b(t), which will encompass the ODFs and SDEs of interest to us. All need for the ODEs is the special b(t) a(t)c, which is easier.
The main results are
dr a(t)(c- x)
di
x(t) c + (ØT A(T)--A)))
(25)
where A(t) / a(t)dt
and
dr dt a(t)x x b(1)
x(l) e A(0) (B()) - B(T)) +Tpe A(T)
(26)
where A(f) = / a(t)dt, B(t) / (a())b(tddt.
First let's consider the special case b(t) = a(t)c, which is easier. We can solve it (formally) by separable equations:
dr
di a(I((c
/ da / aa())tt = A(t)
log(c x) = A(t) +C
c -((--C
r(t) = c Ce A(t)
(27)
Next we need to apply initial conditions get the right constants. Remembering that we are actually sampling backward time from initialization Ir, we can solve for the constant C as follows, to obtain result (25):
FT = c + Ce A(T)
- c) = A(T) ((T) (xT
x(t) = c (IT A(T)))))
19

---

Published in Transactions on Machine Learning Research (08/2025)
We will apply this result to CFGDDIM shortly, but for now note that for VE diffusion dr vidw
Gaussian data distribution po(z) ~ (1,,))) the above result implies the exact DDIM dynamics:
pc(x) N(, +t)
dr 1 Va logp((z)
DDIM on (rr)::
dt
2
(11 x)
2(0² +t)
1
A(I) log(o 2 +1)
2t + (IT 11) 1)) ((T))A(((()
0² +t p + (.T V 0² + T I==0 1t).
(which makes sense since Tt=T Fr and
Vo*+T
Now let's return to the general problem with arbitrary b(t) (we need this for the SDEs). We can use an integrating factor to get formal solution:
dx =a( b(1))
dt
d Integrating factor: A()), A(I) = / a(()dl
dt (z(()) = (d((1) + =a())x(1)) e(0))
=(())=
= el())(()) = eAlD)(t)dt+CC
x(t) e / ((Ob(())tt CC A(0) Note that if b(t)= a(t)c this reduces to (27):
(28)
/e (ltt)) fe-At)e2l0bbb)) ce -e(1) / a(t)) dt =c
x(t) c A(1)
Again, we need to apply boundary conditions to get the constant, and remember that we are actually sampling backward in time from initialization Fy to obtain result (26):
dæ
dl -a(()x + b(t)
B(I) = / (AAO
-A(T) B(T) + Ce-A(T))
C ^00) xT - B()
ztt) -A()) B(t) (C xT B(T))c A(1)
e A(0) (B(t) - B(T)) + (x
Note that for b(() = all) this reduces (25):
b(t) a(t)c = B(t) A(1)
x(1) -ce -A(0) (EA()) - -AA(T)) + xre A(T)-AA)
(IT c) (AT))
20

---

Published in Transactions on Machine Learning Research (08/2025)
Counterexample 1 solutions
To solve the CFGDDIM ODE for Counterexample (Equation 11) apply
result (25):
dr
di a()(cc x) = x(1) = c+ (eT AT)
7
(1-7)
a(t)
c=0
2(1 + t)
2(2 +t)
1
/ (1+ 1)
(1 (1-1) dl
A(I)
2
t2+
= 2 (> log(l +1) (7 1) log(t - 2))
2))
Ft V (T+1))(T + 2)1-7
To solve the CFGDDPM SDE for Counterexample (Equation 11), we first apply (26) to the SDE with b(t) = C(t):
dx
dt = a(t)x - E(t), (E(t)) = 0, (E()),((t)) = 5(t - +')
= x(t) = ITE A(T)-A(1)) te -10) (B(t) B(T)), A(t) = a(t)dt, B(t) = / (()g())
-ITA((T)-A() + +e - S. 22A0)
Now, plugging in the DDPM drift term we find that
(1 (1-7)
a(t) (1+t) (2 t)
A(t) - log(1 1 + t) (1 - 7) log(2 2 + t)
A(t) = (1 + 1) 72++1)
f 24 di / + (1+1)- 2) 2) 2+ dt
(1+1) 1-27
-2 -1 (++2
Sr T
a(t) A(1) + te -A()),
c2A(1)
(1 (1+
1
((++2
1-27
(T+1)
1-27
IT (2+7))(++7)11) +1+
2
5.
T 2
A.6 Exact Denoiser for GMM
For the experiments Figure 2, we used an exact denoiser, for which we require exact conditional and unconditional scores. Exact scores are available for any GMM as follows. This well-known (e.g. Karras et al. (2024a)) but repeated here for convenience.
21

---

Published in Transactions on Machine Learning Research (08/2025)
1
p(x) Σ (1, where (EE = V2TO
2n²
Vxp(z)
V logog((z)
p(x)
Σ ws Vno(us.
Σ (uo((ui σi)
Σ wi 2-
b12; of)
Σ (i
B PCG SDE
We want to show that the SDE limit of Algorithm with K = 1 is
dx ADDIM(2, t) (L,
To see this, note that a single iteration of Algorithm with K = 1 expands to
3
2t = Z1+ 7 Be(I+AA Vlog Plog
V. (ogle, B.AANN(O,
2
step on Pc+aa(
Langevin dynamics (a.-())le)
dr lim At-0
- (zslp( + 3. (1aledd + BBdw.
ADDIM(a.t)
ALDe(z,t.)
This concludes the proof.
A subtle point in the argument above is that ALD (,7) represents the result of the Langevin step in the PCG corrector update, rather than the differential of an SDE. In Algorithm 1, remains constant during the LD iteration, and so the SDE corresponding to the LD iteration
dx B. (ogees + v Bedw
(29)
where s is an LD time-axis that distinct from the denoising time i, which is fixed during the LD iteration. Thus At, is not the differential of (29) (the difference is dt vs ds). However, when we take an LD step of length as required for the PCG corrector, the result
dt
81 V log ds Vi,do 2
3t V. log Bidw ALD
so ALDo(z,t,7) represents the result of the PCG corrector update in the limit At 0.
C Additional Samples and Metrics
An alternative discretization
In this section we empirically study an alternative discrelization of PCG. The equivalence between PCG and CFG holds in the SDE limit as At + 0, so PCG should be thought of as an SDE for which Algorithm 1 is one choice of discretization. However, other discretizations are possible. In this section we explore one of these. In particular, we make single change Algorithm we modify the LD loop by changing the order of operations: first. add noise, and then compute and step in the direction of the score; specifically, the
22

---

Published in Transactions on Machine Learning Research (08/2025)
Increasing # Langevin Steps (PCGpDPM)
0
1
2
5
Figure 7: Effect of Langevin Dynamics. PCG generations with 7 = 1 (no guidance) fixed and number of Langevin steps K varied. The prompt is "photograph a panda cating pizza". Increasing the mumber of Langevin steps can qualitatively improve image quality, even without guidance.
Table 2: Metrics for DDPM, DDIM, and PCG over y and number of LD steps. LD discretization per A Algorithm See also alternative discretization metrics in Table 3.
FID
Y =
Y
1.1
Y = 1.3
Y = 1.5
CFG
5.99
3.90
2.71
3.33
CFGDDIM
7.11
4.61
2.55
2.47
PCG (K = 1)
7.77
5.54
3.37
3.16
PCGDDIM (K 3)
7.42
3.71
6.10
PCGEDIM (K = 5)
7.23
3.80
4.87
8.86
FD-DINOV2
Y
1.1
1.3
1.5
DDPM-CFG
161.72
125.71
84.65
65.44
DDIM-CFG
189.76
152.04
104.17
79.07
PCG LD STEPS 188.83
155.19
109.11
83.50
PCG LD STEPS 3
174.97
119.87
73.38
70.80
PCG STEPS
166.38
110.27
71.08
93.21
INCEPTION SCORE
Y
TY 1.1
1.3
YY 1.5
DDPM-CFG
108.20 108.2628
126.8507
157.0371
178.0676
DDIM-CFG
100.0823
116.3814
144.7761
164.6486
PCG LD STEPS =
101.2306
113.6755
133.1969
147.5756
PCG LD STEPS 105.2118
126.9752
152.2398
160.9198
PCG LD STEPS 5
107.1457
139.8954
155.7239
149.6180
inner loop LD becomes:
Kt + zi + Ven. ~~ ~~~100,1 1a)
81,7 = (1 7) Voggp( +VV (ogp)
Zt zt + $1,7
(30)
This is similar to the "churn" operation in Karras et al. (2022)'s stochastic sampler, and conceptually similar noiso-then-denoie step in Lugmayr et al. (2022). We generally find that this change improves the PCG metrics (more closely matching the DDPM metrics) for smaller 7's, while worsening the metrics for larger
23

---

Published in Transactions on Machine Learning Research (08/2025)
Table Metrics for DDPM, DDIM, and PCG over y and number of LD steps. Alternative LD discretization (Equation 30).
FID
Y =
7 7=
y = 1.3
y 1.5
CFGDDPM
5.99
3.90
2.71
3.33
CFGDDIM
7.11
4.61
2.55
2.47
PCG LD STEPS 5.87115
4.72043
4.15484
4.74044
PCG LD STEPS 3
4.79793
3.49296
4.82135
7.69348
PCG LD STEPS 5
4.51476
3.35029
6.04134
10.6716
FD-DINOv2
1.1
1.3
1.5
DDPM-CFG
161.72
125.71
84.65
65.44
DDIM-CFG
189.76
152.04
104.17
79.07
PCG LD STEPS 156.854
132.605
102.107
88,2433
PCG LD STEPS 3
137.502
100-912
76.9214
86.1473
PCG LD STEPS
129.782
89.3722
79.0756
112.229
INCEPTION SCORE
Y=1
T
7= 1.3
T
DDPM-CFG
108.2628
126.8507
157.0371
178.0676
DDIM-CFG
100.0823
116.3814
144.7761
164.6486
PCG LD STEPS 107.7871
117.3694
132.3872
141.6556
PCG LD STEPS 115.4412
131.1285
148.9654
152.2574
PCG LDD STEPS 5
117.5658
136.5819
150.1884
138.9601
Algorithm PCGDDIM explicit.
Require: Conditioning c, guidance weight 0.
Require: Constants: {ar), (āx), {B,} from Ho et al. (2020)
1: I1 ~ (N,
2: for (t (f=11 1- At; t 12 0; L 1- At) do
3: E == NosiePerdccionMMddeezzpsee c)
èo == (I++Ar /1 04+) / Vat+At
DDIM step (=+ar(#lc) > pt(x)cc
Ft = vā,ão + /1 ā.Ec
for 1.... K do
Langevin dynamics (r.((
E. == Noise NoisePredictionModelli... c) B. + 2vT ((1 7) + 7Ee) + /Bn
8:
end for 10: end for
11: Return: 20
y's, as shown Table 3. We are not sure why this is, but it well-known that diffusion models are sensitive discretization choices in practice Policoeur-Martinee al., 2021; Lu et al., 2022a).
E
Algorithms
Algorithm provides an explicit, practical implementation of PCG. Note that Algorithm and have slightly different DDIM steps, but this just corresponds two different discretizations of the same process. Algorithm 1 uses the first-order Euler-Maruyama discretization known "reverse SDE" (Song et al., 2020), which is convenient for our mathematical analysis. Algorithm 2 uses the original DDIM discretivation ion (Song et al., 2021), equivalent to more sophisticated integrator (Lu et al., 2022a), which is more common in practice.
24


Reviews:
[{"review_id": "ctE3OAx9wF", "reviewer": "Reviewer_AUGV", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The work provides interesting theoretical characterizations of an important generative technique. It has already been cited and built upon by latter or concurrent works.\n2. The technical analysis appears correct, though I admit I was not able to check all of the details carefully.\n3. The experiments are sufficient to demonstrate their conclusions.\n\nWeaknesses:\n1. As noted, there is not an immediate practical benefit to the findings. There is a chance that the predictor-corrector framework may prove fruitful for later exploration though", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper presents a theoretical characterization of classifier-free guidance (CFG) and its application to the DDPM and DDIM sampling\nframeworks. In particular, they show that they do *not* produce the gamma-powered distribution as some have intuited. They also\ncharacterize CFG applied to DDPM as equivalent in the SDE limit to a predictor-corrector guidance (PCG) method, realized as alternation of a diffusion denoising step and a Langevin dynamics step (note: this method is not practical). Empirically, they show these behaviors and equivalences on simple one- or two-dimensional examples, and qualitatively on real image generations from Stable Diffusion XL.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I just have small technical requests/questions:\n1. After Eq. 2, you might want to note Bayes' rule at play. At first glance reading quickly, I didn't notice the swapped order of c and x, and it confused me for a bit.\n2. What is the difference between $dw$ and $d\\bar{w}$ in Eqs. 5 and 7? What is $\\beta_t$ and its dependence on time? This would be helpful for the uninitiated. \n3. Just above Eq. 11, it should be $(x,c)$ are jointly Gaussian. There's an $x_0$ for some reason currently. $x_0$ also seems to be used in places in Eq. 14 for some reason. \n4. Second sentence below Eqs. 12 and 13, \"twice\" should be \"half\".", "broader_impact_concerns": "These generative models have all sorts of ethical implications to consider, but I do not believe that this work needs to address these in\nparticular, or exacerbates any of these in a concerted fashion. There is also some hope that deeper understanding will help to mitigate\npotential ill effects in general.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "5GzPc6Uvlm", "reviewer": "Reviewer_Zore", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "# Strengths\n1. The paper addresses and corrects a common misunderstanding in the community—that classifier-free guidance (CFG) generates a gamma-powered distribution. It supports this clarification through both theoretical analysis and experimental visualization using a toy model.\n\n2. It introduces a novel framework, predictor-corrector guidance (PCG), to better understand CFG. This framework could be valuable for developing improved generative methods.\n\n3. The paper is well-organized and clearly written. The logical flow is easy to follow: (1) CFG does not produce the gamma-powered distribution; (2) PCG converges to the gamma-powered distribution as the number of correction steps approaches infinity; (3) PCG is equivalent to CFG when the correction loop is performed once.\n\n# Weaknesses\n1. As a theoretical paper, it provides limited justification for why PCG leads to the gamma-powered distribution. The paper lacks rigorous theoretical analysis and supporting experiments to substantiate this key claim.\n\n2. The practical contribution of the paper is limited. While the theoretical insights are interesting, it remains unclear how they can be translated into improved sampling algorithms based on PCG.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper investigates the theoretical foundation of classifier-free guidance (CFG). It begins by clarifying a common misconception, that CFG generates a gamma-powered distribution. To establish a connection between CFG and the gamma-powered distribution, the paper introduces a novel framework called predictor-corrector guidance (PCG). It demonstrates that when the correction step of PCG is performed once, the method is equivalent to CFG. As the number of correction iterations approaches infinity, PCG converges to the gamma-powered distribution. This framework offers a new perspective for understanding CFG through the lens of predictor-corrector methods.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. If possible, could the authors provide a theoretical analysis explaining why PCG can generate a gamma-powered distribution? If a complete analysis is intractable, the paper should at least include both theoretical reasoning and experimental validation using the toy model in Section 3 to support this key claim.\n\n2. Could the authors elaborate on the derivation behind Equations (3) and (4)? A more detailed explanation would greatly enhance clarity.\n\n3. The authors use Figure 1 to support Theorem 4.4, but it is unclear how the figure visually demonstrates the equivalence between CFG and PCG. The two distributions shown in the figure are hard to say are identical. Therefore, the figure alone provides weak support for Theorem 4.4. I also recommend using the toy example from Section 3 to reinforce the theorem's validity.", "broader_impact_concerns": "No concern about the ethical implications of the work.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "yXWfi3hFsS", "reviewer": "Reviewer_tzBY", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "This work raises an important and timely question about the theoretical foundations of Classifier-Free Guidance (CFG). However, despite providing reasonable interpretation and arguments, the results of the work remain inconclusive. In particular, the proposed interpretation raises the following major concerns.\n- The theoretical motivation of the proposed method is not sound because of the arbitrary choice of the predictor step. Indeed, in theory, the corrector step ensures convergence to the target density $p_t(x)^{(1-\\gamma)}p_t(x|c)^\\gamma$ for any initial $x_t$. Furthermore, one can use $(1-\\gamma)\\nabla \\log p_t(x) + \\gamma\\nabla \\log p_t(x|c)$ in the predictor step instead of $\\nabla \\log p_t(x|c)$; thus, getting $\\gamma' = \\gamma$ in Theorem 4.4. Hence, the result of Theorem 4.4. has no specific meaning and depends completely on the choice of the predictor step, which is not motivated.\n- Experiments do not suggest the proposed theoretical interpretation of CFG. Indeed, in Table 1, the authors demonstrate the performance on ImageNet for different values of the guidance scale $\\gamma$ and the number of Langevin steps $K$. For all the values of $\\gamma$ except $\\gamma = 1$ (which means no guidance) and $\\gamma = 1.1$, the performance deteriorates with the number of Langevin dynamics steps $K$. These results suggest the opposite conclusion that CFG cannot be considered as an approximate sampling from $p_t(x)^{(1-\\gamma)}p_t(x|c)^\\gamma$.\n- Equation (19) is not a decomposition but a trivial identity. Indeed, the equation reads\n$$f(\\gamma) = g(\\gamma) - (g(\\gamma) - f(\\gamma)),$$\nwhere functions $f(\\gamma)$ and $g(\\gamma)$ are the perceived qualities of different distributions depending on the guidance scale $\\gamma$. Clearly, no conclusions can be made about $g(\\gamma)$ or $g(\\gamma) - f(\\gamma)$ based on the monotonicity of $f(\\gamma)$ since this identity holds for absolutely any function $g(\\gamma)$.\n\nMinor comments:\n- Although the geometric average of densities is often used as an intuitive interpretation of CFG, calling it a common misconception and providing the counterexamples does not seem necessary. Furthermore, multiple concurrent works are improving, interpreting, and correcting CFG, which suggests that no misconception exists in the community.\n- Some abbreviations are not introduced (e.g., DDIM, DDPM), which complicates the reading of the paper.\n- There is inconsistency in the differentiation notation --- some nablas miss $x$ (see eq. (9,10)), some nablas have it. I suggest choosing one type of notation and using it throughout the paper.\n- The following sentence is confusing: \"The iteration above uses an intuitively better predictor that moves $x_t$ along the diffusion path, which is the “correct” way to reduce temperature (at least in the conditional diffusion setting).\" What is temperature in this context? What is the \"correct\" way to reduce temperature for the conditional diffusion?\n- In Table 1, the reported metrics have different numbers of significant digits.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work studies Classifier-Free Guidance (CFG), a standard technique in the diffusion models literature used at inference time for image generation. While significantly improving the quality of generations, CFG lacks a theoretical foundation, which does not allow for establishing guarantees and building on top of it. This work proposes interpreting CFG as an integration scheme, which interleaves two steps: the predictor step simulates the corresponding reverse-time SDE, and the corrector step employs Markov Chain Monte Carlo to ensure convergence of samples to the target marginal density. The only difference with the predictor-corrector scheme proposed in [1] is the target marginal density of the corrector step. Namely, the authors proposed to target the geometric average $p_t(x)^{(1-\\gamma)}p_t(x|c)^\\gamma$, where $p_t(x)$ is the marginal density at time $t$ and $p_t(x|c)$ is the corresponding conditional density with the condition $c$.\n\nTo support their interpretation of CFG the authors provide two arguments: theoretical and empirical. The authors start their analysis by providing 1-d Gaussian examples where CFG fails to sample from $p_t(x)^{(1-\\gamma)}p_t(x|c)^\\gamma$. These examples motivate the introduction of the corrector step: the unadjusted Langevin dynamics that converges to $p_t(x)^{(1-\\gamma)}p_t(x|c)^\\gamma$ with the number of iterations. Furthermore, in the limit $\\Delta t \\to 0$, the authors show that the proposed predictor-corrector scheme with a single step of the Langevin dynamics is equivalent to CFG. For the empirical argument, the provided experiments study the proposed scheme for different guidance scales $\\gamma$ and numbers of steps $K$, and demonstrate its equivalence to CFG by generating visually similar images. \n\nThroughout the work the authors highlight that the proposed framework serves as an explanation of CFG rather a way to improve its performance. However, they hypothesize that the proposed interpretation potentially allows for designing much more efficient inference procedures.\n\n[1] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "To address my main concerns, I suggest the following changes:\n1. Extend the proposed PCG framework to a broader family of predictor steps and empirically demonstrate their necessity (e.g., showing that using just annealed Langevin for any number of steps does not produce high-quality generations).\n2. Discuss why the empirical results are not in accordance with the proposed theoretical model.\n3. Remove equation (19) from the outline of the big picture and adjust the claims correspondingly.\n4. Address the list of minor comments.", "broader_impact_concerns": "The paper does not raise any concerns regarding its broader impact.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)