REG: Rectified Gradient Guidance for Conditional Diffusion Models
Authors: Zhengqi Gao
Kaiwen Zha
Tianyuan Zhang
Zihui Xue
Duane S Boning
Venue: ICML 2025 poster (2025)
Keywords: Diffusion models
classifier-free guidance
conditional generation
Primary area: deep_learning->generative_models_and_autoencoders
TL;DR: Albeit their empirical success, the practical implementation of guidance diverges significantly from its theoretical motivation. Our work reconciles this discrepancy by building the correct guidance theory for conditional diffusion models.

Abstract: Guidance techniques are simple yet effective for improving conditional generation in diffusion models. Albeit their empirical success, the practical implementation of guidance diverges significantly from its theoretical motivation. In this paper, we reconcile this discrepancy by replacing the scaled marginal distribution target, which we prove theoretically invalid, with a valid scaled joint distribution objective. Additionally, we show that the established guidance implementations are approximations to the intractable optimal solution under no future foresight constraint. Building on these theoretical insights, we propose rectified gradient guidance (REG), a versatile enhancement designed to boost the performance of existing guidance methods. Experiments on 1D and 2D demonstrate that REG provides a better approximation to the optimal solution than prior guidance techniques, validating the proposed theoretical framework. Extensive experiments on class-conditional ImageNet and text-to-image generation tasks show that incorporating REG consistently improves FID and Inception/CLIP scores across various settings compared to its absence.

REG: Rectified Gradient Guidance for Conditional Diffusion Models
Zhenggi Gao Kaiwen Zha
Tianyuan Zhang' Zihui Xue Duane S. Boning
Abstract
models now dominate a wide range of tasks, such as class- conditional image generation (Peebles & Xie, 2023; Karras et al., 2024b), text-to-image generation (Ramesh et al., 2022; Rombach et al., 2022), video generation (Ho et al., 2022), and audio and speech synthesis (Kong et al., 2021). De- spite the varied theoretical foundations, such as DDPM (Ho et al., 2020), score matching (Song et al., 2021), Schrödinger Bridge (De Bortoli et al., 2021), and flow matching (Lip- man al., 2023), diffusion models converge on unified implementation: forward process progressively corrupts data by adding Gaussian noise, while reverse process, pa- rameterized by a neural network, is trained to denoise and reconstruct high-quality samples from pure noise.
Guidance techniques are simple yet effective for improving conditional generation in diffusion models. Albeit their empirical success, the prac- tical implementation of guidance diverges signif-- icantly from its theoretical motivation. In this paper, we reconcile this discrepancy by replacing the scaled marginal distribution target, which we prove theoretically invalid, with a valid scaled joint distribution objective. Additionally, we show that the established guidance implementa- tions are approximations to the intractable opti- mal solution under no future foresight constraint. Building on these theoretical insights, we pro- pose rectified gradient guidance (REG), a versa- tile enhancement designed to boost the perfor- mance of existing guidance methods. Experi- ments on ID and 2D demonstrate that REG pro- vides better approximation to the optimal so- lution than prior guidance techniques, validating the proposed theoretical framework. Extensive experiments on class-conditional ImageNet and text-to- -moge generation tasks show that incor- porating REG consistently improves FID and In- ception/CLIP scores across various settings com- pared to its absence. Our code is publicly avail- able at:
One pivotal factor behind the success of diffusion models is the guidance technique (Dhariwal & Nichol, 2021; Ho Salimans, 2022; Kynkäänniemi al., 2024; Karras et al., 2024a). Concretely, guidance is a sampling-time method that balances mode coverage and sample fidelity (Ho & Sali- mans, 2022) by updating the noise prediction network output as a weighted sum of its original output and a user-defined guidance signal, with the mixing coefficient (i.e., guidance strength) controlled by hyper-parameters. When initially proposed by Dhariwal & Nichol (2021), the guidance signal is provided by an auxiliary classifier trained alongside the diffusion model, giving rise to the name classifier guidance. Subsequently, Ho & Salimans (2022) eliminate the need for an additional classifier, relying instead on an implicit Bayes posterior classifier to produce the guidance signal. This classifier-free guidance method has since become a pervasive component of modern diffusion models, enabling effective conditional generation across various applications.
1. Introduction
Generative machine learning endeavors to model the under- lying data distribution, enabling the synthesis of new data samples that closely mirror the characteristics of the original dataset. While many generative model families (Kingma, 2013; Goodfellow et al., 2020) have emerged over time, the recent surge in diffusion models (Ho et al., 2020; Song al., 2021) has marked significant breakthrough, allow- ing for diverse and high-quality generation. These diffusion
Despite the practical effectiveness of the guidance technique, its motivation and theoretical formulation remain poorly understood and, at times, conflicting in existing literature. Specifically, guidance is originally stated as constructing new noise prediction network post-training correspond- ing to sampling from scaled distribution (Ho & Salimans, 2022). However, recent works (Bradley & Nakkiran, 2024; Chidambaram et al., 2024) using Gaussian mixture case studies reveal that this newly constructed noise prediction network does not result in sampling from the intended scaled distribution. This fundamental inconsistency weakens the theoretical foundation of current guidance techniques, rais- ing misinterpretations and concerns about the implemen-
'Massachusetts Institute of Technology. Uniierriity of Texas at Austin. Correspondence to: Zhengqi Gao <zhengqi@mit.cduu mii.
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

REG: Rectified Gradient Guidance for Conditional Diffusion Models
tation optimality. In this paper, we reconcile this conflict and build unified theoretical framework for understanding guidance techniques in conditional diffusion models. Our main contributions include:
is fixed in DDPM (Ho al., 2020), but can also be learned as a function of () (Nichol & Dhariwal, 2021). The mean Mø, E IR is parameterized by noise prediction network Eg Eg(x,, v) X x R+ y TR as follows:
(1) We establish that scaling the marginal distribution at the denoising endpoint, as used current guidance literature, conflicts with the inherent constraints of the reverse denoising process of diffusion models ($ 3).
1
1 az (o(x,tt)))))) VI-
Mo.t = Val
X,
(3)
where 02 IT az. The noise prediction network eg (xe, y) is trained by minimizing the weighted L2 norm between its output and the actual noise motivated by vari- ational evidence lower bound (Ho et al., 2020). From now on, unless explicitly stated, we will omit the arguments and use the shorthand notation 60,z = <a <x(x,, y) for brevity.
(2) We ground our theory in scaling the joint distribution associated with the entire denoising chain, interpreting guidance methods as approximations to the intractable optimal solution with quantified error bounds ($ 4).
(3) Finally, we present rectified gradient guidance (REG), versatile enhancement compatible with various guid- ance techniques ($ 4), Comprehensive experiments validate our theoretical framework and justify the ef- fectiveness of the proposed REG method ($ 5).
Applying guidance techniques in conditional diffusion mod- els is achieved by replacing Eg,, with a refined version Eg, in the reverse denoising process Eq. (3) during sampling. Be- low we delincate three prominent forms of Eg correspond- ing to classifier guidance (CG) (Dhariwal & Nichol, 2021), classifier free guidance (CFG) (Ho & Salimans, 2022), and auto auto-guidance (AutoG) (Karras et al., 2024a):
2. Preliminary
We briefly review guidance techniques (Dhariwal & Nichol, 2021; Ho & Salimans, 2022) developed for conditional image generation in diffusion models. For simplicity, we adopt the notation of discrete-time DDPM (Ho et al., 2020) throughout this paper, while noting that all of our results can be similarly derived in other diffusion settings (Song al., 2021; Karras al., 2024b). Let denote the conditional distribution of interest as q ly). We attempt to learn diffusion model to generate samples Xa E X C R² given the conditioning variable y e y, where y can be either continuous (e.g.. a text embedding) or discrete (e.g., a class label). Starting from a clean data Xo, the forward process of DDPM produces samples {x+) 17 at progressively higher noise levels by gradually injecting Gaussian noises through T-step transitions
CG:
Eo.t = Ca.t wv1 - 0g V log (og
CFG:
Ég.t = 60.t + w (Ea, Ea(xe, 1)),
(4)
AutoG: Eg,E = 60,e + w (Eo.: EB.w (x,, t,,
where the hyper-parameter w E R+ controls the guidance strength, and the shorthand notations Co.t εo y) is used for clarity. We emphasize that Eq. (4) is applied for = 1,2, ... T, not t = 0. Since in our notation, eo (a, y) the noise prediction at the final time step used to update from x1 = a to xu, and (0,00,)) is never used in the reverse denoising process and meaningless.
In CG, an auxiliary classifier Po (y (x,) with learnable pa- rameters @ is trained alongside cØ,: to provide the guidance signal during sampling (Dhariwal & Nichol, 2021). In con- trast, CFG removes the need for an external classifier by utilizing the diffusion model itself (Ho & Salimans, 2022). The elegance of CFG lies in its training process, where the conditioning variable y is randomly dropped. This enables single noise prediction network 6ø,r to operate either conditionally as cy cx v), or unconditionally as Eg (x,, 1) = c l,y = Ø) by masking the conditioning variable y = Ø during sampling. Compared to CFG, Au- toG further enhances guidance by using a degraded version beed of the diffusion model, such as a checkpoint from an earlier stage of training (Karras et al., 2024a).
((x((x11)) = (xx Vaxx--1 (1 aI)
((xarly) = (x) II (x))((x))1)
where {ar)T= is decreasing series in [0, 1] controlling the noise variance. The reverse process of DDPM starts from noisy sample xr, and gradually denoises it by T-step transitions po px 1 yielding a clean sample Xo:
p0(x--1Xxe.) = NNx / of1).
T
Po (x = pa (xTly)) II po px Ixx, Aml
3. Guidance Theory Pitfall
where 9 represents learnable parameters, and (x ly) is usually fixed to a standard Gaussian distribution regardless of y, i.e. po (x) = Nx(x|(0,1) I). The variance term of
Common Interpretation of Guidance. The original mo- tivation underlying guidance (which later we will correct) is that we attempt to sample from a constructed po po (xoly)
2

---

REG: Rectified Gradient Guidance for Conditional Diffusion Models
given a trained diffusion po (Ho & Salimans, 2022):
noise prediction network 60,r, which can then be used to sample from po following the denoising process Eqs. (2)-(3). However, a closer examination reveals that this derivation would yield equations for Ég,t associated with only = 0. On the contrary, Eq. (4), which has been employed by CG, CFG, and AutoG in practice, applies to all time steps = 1,2, T, but not 0.
Po (xol)) x Px(Xxly) Ro(xo:)):
(5)
where the reward y) : X' x y - R encourages
sampling Xy more frequently from where Rx y) is large. The rewards associated with CG, CFG, and AutoG are:
CG:
Ro(xo: y)
(po(y) (x))
This discrepancy highlights a significant gap between the theoretical derivations and the practical implementations of current guidance techniques. Another way to illustrate this gap is to observe that Eq. (5) implicitly assumes no change to Px( ly) for t = 1, , In contrast, the actual practice Eq. (4) modifies EØ.e. thereby altering these distributions through the score function formula in Eq. (7).
Po (Xp y)
CFG:
Yx =
Po (Xo)
(6)
AutoG: y) =
Po (Xoly)
.T.
po (xoly)
Intuitively, the CG reward encourages sampling from where Poly xo) is large, which advantageous for conditional generation tasks (e.g., emphasizing class-specific charac- teristies aligning better with text embedding). For the CFG reward, Bayes' theorem allows us to rewrite it as Ro (xo,y) x (po(y) [,]]))x))] implying that the CFG reward differs from the CG reward merely in its use of an implicit Bayes posterior classifier derived from the diffusion model.
Invalid Marginal Scaling. Since the theoretical interpre- tation and practical implementations only differ in the time steps at which Eq. (4) is applied, it seems appealing to bridge this gap by re-framing the original goal. Specifically, instead of enforeing a single scaled constraint exclusively on the marginal at the denoising endpoint, we redefine the objective to impose scaled constraints on all marginals (Ho & Salimans, 2022):? 2
Eqs. (5)- are the commonly believed guidance interpreta- tions. To ensure that the reverse denoising process indeed samples from Po (xoly)), two key relationships are required. The first is the score function formula (Song et al., 2021; Ho et al., 2020):
Po (x:ly) x po(xxly) Re (xx,
(8)
where t t= 1,2, ... ,,, and the rewards are:
CG: yx = [0o,x, (y(x)))
Xx, log (x = E0(X,, y,)
/1 āi
Re(x,,y)- = (x.xx(x
CFG:
Xx, (xg = ca (x),
Pe,X, (xt)
(9)
V1 0z
PO.X, (xelly)
AutoG:
(x(x,,))=
Pe (xx (y)
In the discrete-time DDPM formulation, strictly speaking, Eq. (7) is valid only for time steps t 2, ,T, but not for ( = In contrast, the continuous-time stochastic differential equation (SDE) formulation of diffusion mod- els (Song et al., 2021; Lu et al., 2022) naturally extends the validity of the equation to all & E []. including L = O, which we adopt in this paper. See Appendix A for details.
Here we explicitly note that the distributions are associated with the random variable X, in Rg (x. It is important to note that R₂ (x,, defined in Eq. (9) with f = 0 recovers the previously defined in Eq. (6), and that ps,x, (y|Xi = a) and Pe. (y = a) are not necessarily identical, even for the same (a. y) value. To simplify notation, we will omit X, in subsequent expressions, as it should be clear from the context. Taking the derivative of the logarithm of Eq. (8) and applying the score function formula in Eq. (7) yields:
Next, by taking the derivative of the logarithm of Eq. (5), we obtain the second relationship: Vx. log po (x (xoly) (xo y) Vx, log Ro y). It is stated in the literature (Ho& & Salimans, 2022) that substituting Eq. (7) into this gradient equation and using the rewards shown in Eq. (6) allows CG, CFG, and AutoG to construct a new
V Xx log Po
Eo. = co. - /1 5e Vx. x. log (x),
(10)
Substituting the rewards from Eq. (9) into Eq. (10) leads to Eq. (4). In other words, the reformulated goal of scaling all marginal distributions in Eq. (8) aligns with the guidance practice described in Eq. (4).
'GG and CFG rewards are explicitly stated in (Ho & Salimans, 2022), while AutoG reward is unspecified in (Karras al., 2024a) and we reformulate it within this framework. Another subtly is that strictly speaking. the original goal of CG and CFG stated in (Ho & Salimans, 2022) correspond to Eqs, (8) and (9) in our paper (i.e., scaling all marginal distributions). We slightly abuse the concept and start from scaling only the terminal marginal Eq. (5) for presentation clarity.
"To align with the fact that Eq. (4) is also applied at = T, here must introduce Rr and enforce scaling at = T. However, this suggests the denoising starts from distribution other than the fixed standard Gaussian. See 5 4 for more discussion.
3

---

REG: Rectified Gradient Guidance for Conditional Diffusion Models
However, this reformulated goal itself is invalid, because we observe that once R2 is given, R is implicitly defined up to normalization constant due to the denoising process:
induced expected reward Ex y) at time step as:
Ex(x), - po po x,, y) R) (xx: dxo:
(14)
R Rx-1(xx-1.y) & E [x Mo. Rx, (11) E [N(x of]]]
where t = 0.1,..., ,T Note that the definition gives Eo(xo.y7 = y) at t = O. For later simplicity, we introduce XT+1 = 2, and thus E (xr+(xx+1,)) = I (x (xx y) dxo based on Eq. (14). Since this expression does not depend any specific x, or t, we also denote it as E (xT+1.x))) = Fly) and will use them interchangeably in subsequent discussions.
Unless explicitly stated otherwise, all expectations in this paper are taken with respect to po (X:ly). The expression of Ho,t is provided in Eq. (3), and Fo, defined as follows:
1 1 at R( 3o.1 M0, +
(12)
We now summarize our main result in Theorem 4.1, with the proof deferred to Appendix C. Theorem 4.1 introduces scaled joint distribution po (xu:rly) that we aim to con- struct, justifying the existence and uniqueness of the tran- sition kernels corresponding to it, and showing the form of the marginals Po (xxl))) and the updated noise prediction network E. It implies that unlike the marginal scaling shown in Eqs. (5) and (8), the joint scaled goal po (Xo:r ly) is valid since a new DDPM can be constructed to realize it. Theorem 4.1. To satisfy the scaled goal given in Eq. (13), we must have an unique set of transition kernels:
Vou
See Appendix B for the proof. This observation leads two significant corollaries. Firstly, the scaled objective presented in Eq. (8) is invalid as it imposes excessive constraints that may not be satisfied. Secondly, the original formulation in Eq. (5) is also invalid. Because it implicitly assumes that all Rg (where/ / = T, T,T-- 11.... 1) should be identity functions, which consequently makes Rg an identity function as well. Essentially, these two corollaries state that is not feasible to construct new DDPM corresponding to either the scaled objective in Eq. (5) or Eq. (8).
Er(x,,
Pa (xxx|xx+1,33 = E y) pa (x), (15)
Finally, if we preserve pe(x)ly) unchanged from (pTTly) ly), ie., Ry is an identity function, then all subsequent rewards Rg (where = T 1,T - 2,.... 1) will also be identity functions based on Eq. (11). This implies that strictly ad- hering to the guidance theory requires a modification of the distribution the denoising start. However, the chain start is typically fixed to a standard Gaussian distribution, irrespective of whether guidance is applied. We will revisit and elaborate on this subtlety in $ 4 after reconstructing the guidance theory on a correct foundation.
where t 0.11.... and xT 2, which also determines:
po(x E, (x,, po(xely).
(16)
Ely)
It implies the noise prediction network should be:
t/ =C0 - vI ā. Vx, log E, V)..
(17)
Important Remarks. Surprisingly, Theorem 4.1 states that strictly adhering to the guidance theory must modify the denoising start distribution from poxxly) = A((x 0, to pe (xrlyy) = Er (x,,)) y)/E(y) (N (x I). Whether this adjustment should be adopted in practice remains an open question, as the new distribution (xT is generally unknown and analytically intractable for sam- pling. We leave this question for future work and con- clude this topic by discussing a notable special case. Specifically, when Er(xT, y) varies slowly with respect to xy, the ratio Ex(xr,)) E(y z since E(y) = Exy~N(tr 0.1) (xT.x.. y)]. this case, (xrly) can still be well approximated as a standard Gaussian.
4. Guidance Theory from Joint Scaling
Scaling the Joint Distribution. Examining the interpre- tation of guidance in the previous section, we identify the critical flaw lies in that it scales the marginal distributions, whereas the joins distribution should serve as the cornerstone of our analysis. Specifically, to build the correct guidance theory, we begin with a joint distribution scaled objective:
(o ly) x Po (xx:rrly) Ro (Xx,
(13)
Eq. (13) similar to Eq. (5) in that the reward value depends only on the final generated sample xo and the conditioning variable y. However, it differs from Eq. (5) in that the reward influences the entire denoising chain Xo.7. Further- more, Eq. (13) can derive Eq. (5) by marginalizing out XI.T. Consequently, both of them have the same impact to the generation, since the generation is determined solely by the marginal po po(xoly), y. Before moving forward, we define the
Although Eq. (17) elegantly defines the updated noise pre- diction مي it is infeasible to calculate. This is because the
"To our knowledge, we are the first to theoretically confirm that guidance impacts the denoising start distribution. A prior study (Wallace al., 2023), sharing our spirit, attempt optimize the noise distribution as a form of guidance.
4

---

REG: Rectified Gradient Guidance for Conditional Diffusion Models
term Ex(x,, y) at time requires foresight into the future it necessitates the complete execution of the denoising process to the terminal point & 0. Alternatively, replac- ing Vx, log E, y), with Vx. log R& (x,,y) y) in Eq. (17) yields Eq. (10), which is precisely what current guidance practices employ. Hence, we posit that the original guid- ance practice best interpreted as an approximation of the optimal noise prediction network, operating under joint scaling with the constraint of no foresight into the future. Importantly, our interpretation reconciles previous confusions (Chidambaram et al., 2024; Bradley & Nakki- ran, 2024) regarding why guidance implementations do not result in sampling from pe. as Eq. (17) represents the op- timal solution for sampling from po, while the practical implementation an approximation shown in Eq. (10).
propose novel rectification Eq. (10) by incorporating gradient term, dubbed rectified gradient guidance (REG):
REG f,1 -Eg,
v1 - 0rt Xx, lo (x),
1 vI 0(1 60,z)
(21)
0x
REG correction term
where @ represents the clement-wise product, and 17 Co. computes the sum of all elements in 60,t- To motivate our above guidance equation, we first notice that E (x y) in Eq. (17) becomes Ro (Xo, when a deterministic sampler is adopted in the denoising process. If we further apply the chain rule, we obtain:
The effectiveness of guidance hinges well-designed rewards R2 ensuring that 6g.t R Ed.t or equivalently xx, log y)) = Vx, log y). To quantify the approximation error, we present Theorem 4.2 and 4.3, with proofs deferred to Appendix D. Notabbly these theorems and their proofs remain valid within the continuous-time SDE formulation of diffusion models. When & is treated con- tinuous, Theorem 4.2 states that the mean squared approxi- mation error decreases as the denoising process progresses (t + 0 and A + 0), as outlined in Eq. (18). Moreover, the expected error is approximately linked to the second mo- ment based on Eq. (19). Additionally, Theorem 4.3 states that the approximation still exhibits a bias even under a looser expectation perspective.
E0.E E0,t v1 āi Vx, log Ro (20.
O&0
(22)
60,t V1 ār-
V log Ro y).
Ox,
We have performed several approximations to get Eq. (21) from Eq. (22). Firstly, we replace V log Ro (x) with Vx. log Re 3). It is straightforward to show that their mean squared error is upper bounded by 2w T.]T||A]] ||l + 2wIL following the proof for Theorem 4.2 in Appendix D. No- tably, this bound has one fewer term than the original guid- ance approximation error shown in Eq. (18). Secondly, we have used the analytical expression Xn vā.(
v (0,,,) given by the DDPM formulation to simplify the jacobian matrix 0xo/Ox, during which the ratio 1/ aa has been omitted since can be absorbed into the guidance strength we It is important to note that different diffusion formulations (Karras et al., 2022) may produce slightly dif- ferent analytical expressions relating x, to Xo. leading to variations in how the Jacobian matrix 0x,/Ox, is simpli- fied compared to the DDPM case, which will be covered in our experiments in & 5.2. Finally, the last line of Eq. (22) involves a D-by-D Jacobian matrix 0xp/ /xx, multiplied by D-dimensional vector Vx, log Ro(xo. y), which is com- putational prohibitive. We have simplified this operation by approximating it with element-wise vector multiplication in Eq. (21), which performs well empirically. This approxi- mation can alternatively be interpreted as assuming that the Jacobian matrix is (approximately) diagonal.
Theorem 4.2. Assume a deterministic sampler is used in the reverse denoising process, and the original noise network Cg2r is L-Lippshitz continuous with values bounded by B. For CFG and AutoG, the following bound holds;
V1 1 aoll Vx,
xx log Ex Ex(x,,y y)- Vxx log Re (xtx
Bl dA
(18)
+ 2I + LwLt,
dx:
where A == =x0- X,, and Xn represents the estimate of Xo based X2 at time step t. Moreover, under these assump- tions, we approximately have:
Bille,, E0.ell] S &CBBllx, X O2,
(19)
where C1 and C2 are constants of x, Theorem 4.3. The bias of approximation is:
5. Experimental Results
Re Rx(xx,)) y) (20)
5.1. 1D and 2D Synthetic Examples
Blea,- - -E0,0 = EEEe,, log Ex(
In this section, we conduct qualitative experiments on 1D and 2D synthetic examples. Based on the definition in Eq. (14), the exact calculation of x, log E, (x,y) y) involves gradient propagation through the denoising chain, which is generally computationally intensive but can be affordably computed in 1D or 2D.
REG as an Alternative. Upon recognizing that the origi- nal guidance technique in Eq. (10) is an approximation to the true guidance equation in Eq. (17), we attempt to explore whether alternative approximations exist. To this end, we
5

---

REG: Rectified Gradient Guidance for Conditional Diffusion Models
0.0
Time Step Time Stop
20
[oolden w/ REGI
further evidence to support this observation. To rigorously substantiate our claim, present the win ratios in Table 1, indicating the cases where the error with REG and with- out REG is smaller, respectively. Finally, Figure 2 (g)-(h) display the magnitude of our REG correction term. We ob- serve that the REG correction term exhibits rough vertical and horizontal patterns for the X-axis and Y-axis, respec- tively, as well as a "low-resolution shape associated with each class. This behavior is expected because, when away from the target shape, the REG X-correction term, which relates to derivative with respect to the X-coordinate, is only weakly dependent on the Y-coordinate. Consequently, it exhibits a vertical pattern, as shown in column (g). In contrast, near the target shape, data points occur at varying Y-coordinates for fixed X-coordinate, causing the REG X- correction term to vary accordingly.
0.5
2.0
10
-10
-1.5
20
Golden - -Wo REG
1.0
2.00
Golden'l &
10
-2.6
Clog fi wlo REG
FlogRi a/ REG
1100
0.0
-1.0
0.5
0.5
2.0
Figure 1. Left: Guidance values are plotted along the X-axis in the range [-1.0, 2.0] at time step L = 13. Right: Heatmaps depict the absolute differences between each gradient guidance value and the optimal guidance V log Fi,, plotted on uniform grids in [-1.0,2.0] at each time step. These two figures justify that our proposed REG aligns better with the optimal guidance V log E compared to the vanilla CFG, i.c. V log Rs without REG.
In the ID example inspired by (Kynkäänniemi et al., 2024), we attempt to learn a conditional data distribution q<rly- =
Table I. The win ratio x% : 5% is reported for different time steps and classes, where x% and 5% denote the cases where V log Rg with REG and without REG achieve smaller error, respectively.
0) = 0.5 x N (x 0.25²) + 0.5 x (x 0.25²) ) using a diffusion model with 20 steps and an MLP as the noise pre- diction network. The unconditional data distribution g(x) is set 0.5 x [axl - 1,.05²) + 0.5 x (ax 0.5²). As in (Kynkäänniemi et al., 2024), the number of classes and other conditiond distributions are considered irrele- vant in this context. The reward Ro(2, y) = (olp)))/ (z) chosen in Eq. (13). Following Eq. (14), the optimal V log Fr 4 is the derivative of a sealar to scalar, and can be computed via numerical integration each time step t. Fur- thermore, evaluate the original V log RS as per Eq. (10) and the V log R2 corrected with REG as per Eq. (21). The results are visualized in Figure 1. On the right, Vogg R. with REG aligns more closely with the optimal solution V log E, across all time steps and grid points, compared to the version without REG. The left panel of Figure 1 high- lights a specific case at 13. Experimental setup details and additional results are provided in Appendix E.
Time Step
Class 1
Class 2
20
60.2%
39.8%
65.2% 34.8%
= 16
56.7% : 43.3%
68.2% : 31.8%
t = 12
67.8% 32.2%
74.4% 25.6%
= 8
67.7% : 32.3%
73.1% 26.9%
t=
65.5% 34.5%
72.0% 28.0%
5.2. Quantitative Results: Image Generation
In this section, we perform quantitative experiments on class-conditional ImageNet generation and text-to-image generation. We consider state-of-the-art guidance tech- niques, including vanilla CFG (Ho & Salimans, 2022), cosine CFG (Gao et al., 2023), linear CFG, interval CFG (Kynkäänniemi et al., 2024). AutoG (Karras et al., 2024a), and demonstrate that our REG a method-agnostic approach capable of enhaneing the performance of all these techniques when used in conjunction with them. As sum- marized in Table 2. we deliberately select models spanning diverse architectures, various samplers (e.g., DDPM, Euler, and 2nd Heun), and different prediction parameterizations (e.g., noise prediction and xn prediction), to rigorously ver- ify the effectiveness of our proposed REG method.
In the 2D example, we design two-class conditional image generation task inspired by (Pärnamaa, 2023). The train- ing dataset consists of 8,000 samples per class, represented as pairs (Xo. y). where Xo is a point in a 2D plane, and y is either 0 or 1. We train diffusion model with 25 time steps using simple MLP as the noise prediction network. In this example, both V log F. and V log R2 represent the derivative of scalar with respect to a 2D vector and can be visualized as gradient arrows in 2D plane. Figure 2 presents the results. Columns (a)-(c) in Figure 2 demon- strate that CFG with REG produces better results compared to without REG. Additionally, for both classes, column (e) lighter in color and features coarser arrows compared to column (f), making it visually closer to the optimal column (d). The red-highlighted region for the first class provides
Class-Condition ImageNet Generation. We evaluate various resolutions, including 64 64, 256 x 256, and 512 x 512, using DiT (Peebles & Xie, 2023) and EDM2 (Karras et al., 2024b) as baseline models. Since EDM2 employs a different analytical relationship between x; and x. com- pared to the DDPM formulation, the REG correction term must be re-devived. See Appendix E.2 for details. Fréchet Inception Distance (FID) and Inception score (IS) are the
"For brevity and where confusion arises, will refer to Vx log Ex(xe,)) y) as V log F,, and similarly for V log Re-
6

---

REG: Rectified Gradient Guidance for Conditional Diffusion Models
2.0
35
Clace
1.0
Cloce
0.0
-0.
Target
CFG REG (6) CFG REG
(6) Eo (4) log Au REG
(1) wio
(g)
(h) REG Y-Correcton
Figure 2. Results of guidance synthetic 2D two-class conditional generation task using a simple diffusion model with 25 time steps. (a)-(c) illustrate the target shape to learned, the shape generated using CFG with our proposed REG, and the shape generated without REG, respectively. (d)-(f) depict V log E, and V log Rd at 9, which are gradients of a scalar with respect 2D vector, visualized as arrows a 2D plane. (g)-(h) show the magnitude the REG correction term (i.e., the second line of Eq. (21)) at 9.
Table 2. A summary of the models used in our experiment.
Table 3. Class-conditional ImageNet generation results are re- ported. The red 4 and green arrows indicate better FID and IS metrics. respectively, achieved by using REG with a specific established guidance method compared not using REG.
Model
# Param
Sampler
Prediction
DIT-XL/2
675 M
250-step DDPM
e-prediction
EDM2-S
280 M
2nd Heun
xo-prediction
EDM2-XXL
1.5 B
2nd Heun
xo xo-prediction c e-prediction prediction
Resolution
Benchmark
FID
IS 1
SD-v1-4
860
PNDM
EDM2-S
1,580
SD-XL
2.6 B
Euler Discrete
64x64
AutoG
1.044
69.01
REG (ours)
1.035
69.01
DIT-XI/22
9.62
121,50
two key metrics used to evaluate the generated image quality and diversity, respectively. We sweep the guidance scale across a broad range to ensure coverage of the turning points on the FID and IS curves. The best FID achieved by each method is reported Table 3 along with the corresponding IS. As shown Table 3, cosine and linear CFG perform marginally better than vanilla CFG. On the other hand, inter- val CFG significantly outperforms vanilla CFG, but also note that determining the optimal interval range involves fine-grained and computationally expensive search.
Vanilla CFG
2.21
248.36
REG (ours)
2.04 +
276.26
Cosine CFG
2.30
300.73
256x256
REG (ours)
1.76 4
287.48
Linear CFG
2.23
268.69
REG (ours)
2.18 4
284.20
Interval CFG
1.95
250.44
REG (ours)
1.86 4
259.57
EDM2-S
2.56
Vanilla CFG
2.29
268.56
REG (ours)
2.02 4
275.30
Cosine CFG
2.16
282.46
Table shows that incorporating our proposed REG, the FID usually decreases and IS increases across different models, resolutions, and guidance methods. Note that for EDM2-S and EDM2-XXI, interval CFG (Kynkäänniemi al., 2024) applies guidance to around 20% of all the time steps, so the impact of REG is less pronounced in this case. Alternatively, when applying interval CFG to DiT (Kynkäänniemi et al., 2024), around 30% time steps use guidance, and REG shows meaningful improvement in this case reducing FID from 1.95 to 1.86. To better justify REG's efficacy, we plot the Pareto front of FID versus IS in Figure 3. It illustrates that using REG universally pushes the Pareto front downward and to the right, achieving more optimal FID and IS metrics. See Appendix E.2 for experimental details and extra results.
REG (ours)
1.99 4
291.77
Linear CFG
2.21
282.89
REG (ours)
1.99 4
291.04 1
Interval CFG
1.67
287.45
REG (ours)
1.67
288.43
512x512
EDM2-XXL
1.91
Vanilla CFG
1.83
265.76
REG (ours)
1.74 +
289.24
Cosine CFG
1,80
261.94
REG (ours)
1.69 1
268.84
Linear CFG
1.81
262.03
REG (ours)
1.69 4
268.30
Interval CFG
1.45
283.26
REG (ours)
1.45
288.72 1
experiment. Due to the exhaustive search required for the interval range in interval CFG and the need for carefully constructed "bad" version in AutoG, these two methods are not practical within our computational budget and have been
Text-to-Imane Image Generation. We use pretrained stable dif- fusion model SD-v1-4 (Rombach et al., 2022), and SD- XL (Podell et al., 2023) available on Huggingface for this

---

REG: Rectified Gradient Guidance for Conditional Diffusion Models
1.5
AutoG w/o REG AutoG REG
4.0
Gosine CFG wio REG
erated images) using FID and CLIP score the evaluation metrics. We sweep a broad range of guidance scales, similar to the class-conditional ImageNet generation experiment, and present the results in Table 4.
Cosine GFG w/ REG
1.4
0 1.3
3.0
1.2
2.5
1.1
Table 4 indicates that despite SD-XL having larger param- eter count, it appears to perform worse than SD-v1-4 on this specific COCO benchmark, a phenomenon observed in previous literature (Wang et al., 2024a;b). We also ob- serve that cosine and linear CFG underperform compared to vanilla CFG in this setting, consistent with findings re- ported in prior work (Wang et al., 2024b). Nevertheless, Table 4 demonstrates that our REG can improve the FID and CLIP scores for all settings. Moreover, the Pareto fronts of FID versus CLIPP score are shown in Figure 4, further justifying the effectiveness of REG. Qualitative results of generated images under different text prompts are deferred to Appendix E.2.
EDM2-S 64
2.0
EDM2-S
70
72
74
76
78
280
280
300
320
340 360
1S
IS
2.3
Interval CFG wio REG
4.0
Vanilla
Interval
Vanilla CFG w/ REG
2.2
3.5
2.1 2.1
3.0
2.0
2.5
1.9
DIT-XL/2 256
2.0
DIT-XU2
256
250
260
270
280
290 300
260
280
300
320
340
IS
IS
Figure 3. The Parcto front of FID versus IS is presented by varying the guidance weight w over a broad range for different methods. The turning points are also included. Curves positioned further toward the bottom-right indicate superior performance. See Ap- pendix E.2 for extra Parcto front results.
Runtime and Memory Overhead. Since REG requires the computation of an additional Jacobian matrix compared to the standard CFG approach, it naturally incurs higher memory usage and runtime. We also note that similar inference-time gradient calculations have also been explored in universal guidance (Bansal et al., 2024), albeit in differ- ent context. Table 5 presents a summary of the runtime and memory overhead introduced by REG on single NVIDIA A40 GPU under identical experimental settings, compared to vanilla CFG approach. In almost all settings, we have affordable less than 2 runtime and memory overhead.
Cosine CFG w/o REG
Vanila CFG wio REG . Linear CFG wio REG Vanilia CFG REG
Cosine CFG REG
Linear CFG w/ REG
25
24
30
23
FID
22
28
21
26
20
24
SD-XL
30.7
30.8
30.9
31.0 31.1
31.2
31.4
31.5 31.6 31.7 31.8 31.9
CUP Score (%)
Score (%)
Figure The Pareto front of FID versus CLIP score is shown by varying the guidance weight across a broad range for different methods. Curves closer to the bottom-right are better.
Table 5. Comparison of CFG and REG runtime performance and peak GPU memory usage across different models, resolutions, and batch sizes. Memory reported with batch size to isolate per- ver-image cost.
Table 4. Text-to-Image generation results on the COCO-2017 5K dataset are reported. The red I. and green arrows indicate hetter FID and CLIP score, respectively, achieved by using REG with specific established guidance method compared to not using REG.
Model
Resolution / BB
CFG REG Runtime (sec)
EDM2-S
64 /4/5
25,96 / 42.99 (1.66x)
DiT-XL/2
256
59.79 94.23 (1.58x)
EDM2-S
512 / 8
46.14/ / 62.87 (1.36x)
Model
Benchmark
FID +
CLIP (%) 1
EDM2-XXI.
512/
49.21 / 92.60 (1.88x)
SD-v1-4
512/4
32.63 / 39.54 54 (1.21x)
Vanilla CFG
20.27
30.68
SD-XL
1024
47.48 74.52 (1.57x)
+ REG (ours)
19.63 +
30.75
SD-v1-4 512x512
Cosine CFG
23.19
30.80
Model
Resolution BS
CFG/ REG Memory (GB)
+ REG (ours)
21.35 4
30.96
EDM2-S
64/
0.87 / 1.49 (1.71x)
Linear CFG
20.55
30.85
DiT-XL/2
256 1
4.15/5.01 (1.21x)
+ REG (ours)
20.27 L
30,87
EDM2-S
512/1
1.19/ 1.81 (1.52x)
Vanilla CFG
23.73
31.38
EDM2-XXL
512 1
(1.59x)
+ REG (ours)
23.46 L
31.51 1
SD-v1-4
512
2.73 /4.39 (1.61x)
SD-XL 1024x1024
Cosine CFG
32.14
31.58
SD-XL
1024/1
6.91 / 19.49 (2.82x)
+ REG (ours)
25.62 4
31.66
Linear CFG
24.43
31.55
+ REG (ours)
23.67 4
31.58
6. Related Work
Guidance techniques are indispensable in modern diffusion models, significantly improving conditional generation qual- ity. Classifier guidance (CG), introduced by Dhariwal
omitted. We evaluate different CFG approaches with and without REG on the COCO-2017 dataset (with 5,000 gen-
8

---

REG: Rectified Gradient Guidance for Conditional Diffusion Models
Nichol (2021), improves the quality of conditional samples by using the gradient of an auxiliary classifier a guidance signal, enabling trade-off' between the FID and IS metric. Classifier-free guidance (CFG) removes the need for an ex- ternal classifier by training a noise prediction network to handle both conditional and unconditional generation via random conditioning dropout (Ho Salimans, 2022). CFG has shown significant empirical success and inspired numer- ous extensions (Kynkäänniemi al., 2024; Karras et al., 2024a; Bansal et al., 2024; Zheng & Lan, 2024; Chung et al., 2024; Ahn et al., 2024). Building on CFG, auto sutto-guiaacee (AutoG) employs a "bad version" of the noise prediction network, such as a checkpoint from a lighter model, to produce the guidance signal (Karras et al., 2024a). Au- toG achieves state-of-the-arr FID and IS scores compared to other guidance methods. However, identifying an effective "bad version" is non-trivial and may require an exhaustive search. Universal guidance (Bansal et al., 2024) extends the framework by utilizing arbitrary guidance signals without retraining. While our work is in a completely different con- text- - focusing on building a theoretically sound guidance theory- REG similar to universal guidance in involving 0xx in the implementation.
7. Conclusions and Limitations
In this paper, we rebuild the guidance theory on the correct theoretical foundation based on joint scaling and reconcile the practice and theory gap. We demonstrate that guidance methods are approximation to the optimal solution under no future foresight constraint. Leveraging this framework, we introduce rectified gradient guidance (REG), a versatile enhancement that consistently improves the performance of existing guidance techniques. Comprehensive experiments validate the effectiveness of REG. Our work resolves long- standing misconceptions about guidance methods and paves the way for its future advancements.
Our proposed REG is grounded in theoretical insights and mainly serves as an experimental validation of our theory's ss correctness. Its practical use depends on application needs, as it improves conditional generation performance with a minor computational overhead due to gradient calculations.
A Acknowledgements
Zhengqi Gao leads the theoretical development, algorithm design, and overall project coordination. Kaiwen Zha con- tributes to the theoretical analysis and experimental setup. Tianyuan Zhang and Zihui Xue contribute to the experi- mental design and evaluation. Duane S. Boning supervises the project and provides guidance throughout. The authors gratefully acknowledge the constructive feedback provided by the anonymous reviewers, which significantly improves the quality of this paper.
Theoretical analyses of unconditional and conditional diffu- sion models are abundant, yet the motivation behind guid- ance has been addressed only superficially in the litera- ture (Dhariwal & Nichol, 2021; Ho & Salimans, 2022), with its theory primarily developed through Gaussian mix- ture case studies (Wu et al., 2024; Bradley & Nakkiran, 2024; Chidambaram al., 2024). Guidance was commonly described being motivated by sampling from a scaled distribution Po that places greater emphasis on regions align- ing with the conditioning variable. Leveraging the score function formula, it transforms sampling from the scaled distribution into an updated noise prediction network Eg.t. which can be seamlessly integrated into the reverse denois- ing process. However, recent works (Bradley & Nakkiran, 2024; Chidambaram et al., 2024; Xia et al., 2024) challenge this interpretation by showing that using the modified εa,t in the reverse denoising process does not yield sampling from Pa. Our work attempts to reconcile this theory and practice discrepancy by rebuilding the correct guidance theory from scaling the joint distribution. We complement the findings of Bradley & Nakkiran (2024); Chidambaram et al. (2024); Xia et al. (2024) by providing the correct form of εg2, to ensure sampling from pg-
Impact Statement
Our research focuses on building the correct guidance theory for conditional generation in diffusion models, ultimately contributing to improved image generation quality. We ac- knowledge that large-scale diffusion models pose significant societal risks, such as generating fake images and videos, exacerbating misinformation, amplifying biases, and under- mining trust in visual media. While our method enhances the generative capabilities of diffusion models, may inad- vertently amplify these associated risks. To mitigate these concerns, emphasize the importance of responsible devel- opment and deployment of diffusion models under stringent oversight. This includes, but is not limited to, integrating bias-reduction strategies and implementing advanced mech- anisms for detecting and preventing misuse.
Another related and concurrent work (Pavasovic et al., 2025) demonstrates that, although CFG may reduce sample diver- sity in low-dimensional settings, it proves beneficial in high- dimensional regimes due to a 'blessing of dimensionality". Their analysis reveals two distinet phases: an early phase where CFG aids in accurate class selection, followed by later phase in which its impact becomes negligible.
References
Ahn, D., Kang. J., Lee, S., Min, J., Kim, M., Jang, W., Cho, H., Paul, S., Kim, S., Cha, E., et al. A noise is worth diffusion guidance. arXiv preprint 2024.
9

---

REG: Rectified Gradient Guidance for Conditional Diffusion Models
Bansal, A., Chu, H.-M., Schwarzschild, A., Sengupta, S., Goldblum, M., Geiping, and Goldstein, T. Universal guidance for diffusion models. In International Confer- ence on Learning Representation, 2024.
Karras, T., Aittala, M., Lehtinen, J., Hellsten, J., Aila, T. and Laine, S. Analyzing and improving the training dynamics of diffusion mmdels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024b.
Bradley, A. and Nakkiran, P. Classifier free guidance is predictor-corrector arXiv preprint arXiv:2408.0900 2024.
Kingma, D. P. Auto-encoding variational bayes. arXiv preprint arXiv: arXiv:1312.6114, 2013.
Chidambaram, M., Gatmiry, K., Chen, S., Lee, H., and Lu, J. What does guidance do? a fine-grained analysis in a simple setting. In Advances in Neural Information Processing Systems,, 2024.
Kong, Z.,, Ping, W., Huang. J., Zhao, K., and Catanzaro, B. Diffwave: A versatile diffusion model for audio synthesis. In International Conference on Learning Representation, 2021.
Chung. H., Kim. J., Park, G. Y., Nam, H., and Ye, J. C. Cfg++: constold classifier free guidance for diffusion models. arXiv preprint arXiv:2406.08070 2024.
Kynkäänniemi, T., Aittala, M., Karras, T., Laine, S., Aila, T., and Lehtinen, J. Applying guidance in a limited interval improves sample and distribution quality in diffusion models. In Advances in Neural Information Processing Systems., 2024.
De Bortoli, V., Thornton, J., Heng. J., and Doucet, A. Diffu- sion schrödinger bridge with applications to score-based generative modeling. In Advances in Neural Information Processing Systems, 2021.
Lipman, Y., Chen, R. T., Ben-Hamu, H., Nickel, M., and Le, M. Flow matching for generative modeling. In Interna- tional Conference on Learning Representation, 2023.
Liu, L., Ren, Y., Lin, Z., and Zhao, Z. Pseudo numerical methods for diffusion models on manifolds. In Interna- tional Conference Learning Representation, 2022.
Dhariwal, P. and Nichol, A. Diffusion models beat gans on image synthesis. In Advances in Neural Information Processing Systems,, 2021.
Lu, C.,, Zhou, Y., Bao, F., Chen, J., Li, C., and Zhu, Dpm- solver: fast ode solver for diffusion probabilistic model sampling in around 10 steps. In Advances in Neural Information Processing Systems, 2022.
Gao, S., Zhou, P., Cheng, M.-M., and Yan, S. Masked diffusion transformer is strong image synthesizer. In Proceedings of the IEEE/CVF International Conference Computer Vision, 2023.
Nichol, A. Q. and Dhariwal, Improved denoising dif- fusion probabilistic models. In Proceedings of the 38th International Conference on Machine Learning, 2021.
Goodfellow, I. Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. Generative adversarial networks. Communications of the ACM, 2020.
Pavasovic, K. L., Verbeek, J., Biroli, G., and Mezard, M. Un- derstanding classifier free guidance: High-dimensional theory and non-linear generalizations. arXiv preprint arXiv:2502.078A4 2025.
Ho, J. and Salimans, T. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12559 2022.
Ho, J., Jain, A., and Abbeel, P. Denoising diffusion prob- abilistic models. In Advances in Neural Information Processing Systems,, 2020.
Peebles, W. and Xie. S. Scalable diffusion models with transformers. In Proceedings of the IEEF/CVE Interna- tional Conference on Computer Vision, 2023.
Ho, J., Salimans, T., Gritsenko, A., Chan, W., Norouzi, M., and Fleet, J. J. Video diffusion models. In Advances in Neural Information Processing Systems, 2022.
Podell, D., English, Z., Lacey, K., Blattmann, A., Dockhorn, T., Müller, J., Penna, J., and Rombach, R. Sdxl: Im- proving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952 2023.
Karras, T., Aittala, M., Aila, T., and Laine, S. Elucidating the design space of diffusion-based generative models. In Advances neural information processing systems, 2022.
Pärnamaa, T. Tiny diffusion: A minimal pytorch implemen- tation of diffusion models. https:: ///tts://giiee /tte://b/ttttt thub. com/ hanelp/tiny-difiiie 01-11.
2023. Accessed: 2025-
Karras, T., Aittala, M., Kynkäänniemi, T., Lehtinen, J., Aila, T., and Laine, S. Guiding a diffusion model with a bad version of itself. In Advances in Neural Information Processing Systems,, 2024a.
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., and Chen, M. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204 06/25, 2022.
10

---

REG: Rectified Gradient Guidance for Conditional Diffusion Models
Rombach, R., Blattmann, A., Lorenz, D., Esser, and Ommer, B. High-resolution image synthesis with la- tent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022.
Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Er- mon, S., and Poole, B. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representation, 2021.
Wallace, B., Gokul, A., Ermon, S., and Naik, N. End- to-end diffusion latent optimization improves classifier guidance. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023.
Wang, F.-Y,, Yang, L., Huang, Z., Wang, M., and Li, H. Rec- tified diffusion: Straightness is not your need in rectified flow. arXiv preprint arXiv:2410.07303, 2024a.
Wang, X., Dufour, N., Andreou, N., Cani, M.-P. Abre- vaya, V. F., Picard, D., and Kalogeiton, V. Analysis of classifier-fre free guidance weight schedulers. Transactions Machine Learning Research, 2024b.
Wu, Y., Chen, M., Li, Z., Wang, M., and Wei, Theoretical insights for diffusion guidance: A case study for gaussian mixture models. arXiv preprint arXiv:2403.01633 2024.
Xia, M., Xue, N., Shen, Y., Yi, R., Gong, T., and Liu, Y.-J. Rectified diffusion guidance for conditional generation. arXiv preprint arXiv2440.117377, 2024.
Zheng, C. and Lan, Y. Characteristic guidance: Non-linear correction for diffusion model at large guidance scale. In Proceedings of the 41st International Conference on Machine Learning, 2024.

---

REG: Rectified Gradient Guidance for Conditional Diffusion Models
A. Remarks on Score Function Formula
We will illustrate the score function formula for the unconditional generation case, corresponding to the second line of Eq. (7), noting that the first line of Eq. (7) associated with the conditional generation is similar.
The continuous-time SDE formulation of diffusion models (Song al., 2021; Lu et al., 2022) describes the forward diffusion
process as a stochastic differential equation (SDE) and establishes the existence of reverse-time SDE that is mathematically equivalent to the forward-time SDE. The only unknown term in the reverse-time SDE is the score function Vx, log qe (x+), which is learned by the neural network eg (x,,,, t) through proper parameterization involving noise schedules during training. Notably, the score matching training objective uniformly samples over the entire range [o, T] with a weighting scheme. This design ensures that the score function formula Eq. (7) holds approximately across the entire range [0, 7 by definition.
In contrast, the discrete-time DDPM formulation can, strictly speaking, only derive that Eq. (7) holds for 1, 2.... .T, but not for t = 0. A straightforward explanation is as follows: note that eo (a, = 1) is the noise prediction at the final time step, used to update from X1 training. Consider an exaggerated case where co(.; t= enables perfect generation. In this case, the left-hand side of Eq. (7), Vxo log pe (xo) equals Vxo log ((x)) and remains proper and finite, while the right- side becomes infinite, illustrating the inconsistency of Eq. (7) for =.
Xo. and c f 0) is never utilized in the reverse denoising process and unused during
= 0) = oo, while c.(,,t) t) for t = 1,2, .... .T behaves correctly and
However, in practice, the score function formula Eq. (7) can be considered valid at t = 0 the DDPM formulation, provided sufficiently dense time grids (i.e., a large enough 7') are used. This is because Eq. (7) holds at L 1, and for dense time grids, both sides of Eq. (7) at t closely approximate their counterparts at = 0. Consequently, Eq. (7) effectively holds at f = O. This perspective is also the discretized view of the SDE formulation. Our aim here is to highlight the correct way to understand the score function formula.
B. Invalid Marginal Scaling
We will prove that R&-1 is implicitly determined by Re, i.c., Eq. (11), in this section. First, we calculate the derivative of logarithm of Eq. (8), yielding:
Vx, lo (x,ly) Xx, log pa (x, + xx, log y)
(23)
Substituting the score function relationship shown in Eq. (7) into it, we obtain:
EΦ E0,1 vI āe Xx, log yx),
(24)
where we use the short notation εa, = Eg. L y) and Eg ex(x,, y) as in the main text. The modified transition kernel Po (x lx,,)) corresponding to this modified Eg,t can be derived by analogy to Eq. (2) and Eq. (3):
po y) =N(xx=1 10 of1),
-
Mage + 1-a0 Xx, log (x;,
(25)
P0.
Xx
EB
Vot
VI Ci
Vat
where Eq. (24) has been used to simplify Po.t- Now, based on the defintion of the reverse denoising process, we have:
Do px ly) = / pa (xx 128, y) Do (y) dx.,
+ (x(x--1ly) Rr- (X:-1,)) x / [N(x: 1 of (x:: Re y) dxi.
po(x /
+ R (x-1(xx-1,))
( Po 0i (xx (y) Re(xx,, y) dxi,
(26)
J [N(x (x (y) Rd (x,, dx:
* -
[N of po (x) dx.
E Exo(x) [N 110 1 (x+,
* -(-1(X(-1,))
Epo(x.l) 1 II
Eq. (26) implies that when R₁ is given, R is implicitly determined up to a normalization constant. Note here we assume the co-variance matrix is fixed to I Nevertheless, the same derivation can be applied to learnable co-variance matrix.
12

---

REG: Rectified Gradient Guidance for Conditional Diffusion Models
C. Scaling the Joint Distribution: Proof of Theorem 4.1
Based on the discrete-DDPM formulation, we know the joint distribution construcuted as a Markov chain:
Po (xo:Tlly) = p(xrly) II t=1
Po (x,,))),,))
(27)
When (Xo:T y) is given, the decomposition to the initial p(xrly) and the T transitions po (xxe-1/xt:>> unique. This because Py (xorll) given, by doing marginalization, we obtain unique p x;ly) and pe (xxly)). Then, based on the conditional probability definition: po(x;-1 xx. y) 1, / (o (x(l))), the transition kemnel is also unique.
Thus, it suffices to demonstrate that the transition kernels presented in Eq. (15) satisfy the joint distribution specified in Eq. (13) and that each kernel represents a valid distribution. Once these conditions are established, Eq. (15) is guaranteed to be the unique solution. Now, we can derive:
(x(x E (x1+1,y) po (xx1xx)+1,)))
Po (x ly) (o ly) II
II t=0
No Po (xx11)++1,1))) t=
II
1=0
(28)
y) (x+1(x)+1
II (x (x+1,,) Ro(xo, y) Ely)
po (xo:rly) x (x
where we have used the notational convention XT+1 Ø introduced earlier, yo(xu, Ro(xo. y), and 1 = Ely). The above equation states po (Xo:rly) xx Rx (o(xx.Y)poxxorr (pooxxxrly)) with 1/B(y) being the normal- ization constant. Next, we proceed to demonstrate that Eq. (15) represents a valid distribution:
/ po( x)+1,,)dxx
/
y)
Ec+1 Ex41(x0+18)) po (x(x)) X++1, dxx:
Er Ex y po (x (X++1 y d)x
E (x
po y) Ro po (xe 1x++1 y dxe, doo
(29)
E (x+1(X(41,1))
1x (xo, y) ddx, dxo
Ex+1(X++1,))
/ |o(xo Ro dxo = E (x(xxx)
Ex1((x+111)
= Ex+1(X++1,y)
Next, we use induction to prove Eq. (16). To begin with, substituting t = T into Eq. (15) and noticing our introduced
convention XT+1 = Ø, we obtain Pa y) = (r(xx,y)/E(y) Po (o(x)ly), which aligns with Eq. (16) at f = T. Now suppose Eq. (16) holds for f, then for t- 1, we have:
px(x = / Pa (x: Kes y Do dxx: = /
Eg Ex-1(Xx-11)) Er Exe, y) Po (0 yx( Ely)
po (y) dx:
(30)
- Er- (x po po(xe-1,, xely) dx,
E-- 16-1(22 (8t-1198 (x
Fl()
Frly)
which completes the induction. Finally, Eq. (17) is straightforward to prove by calculating the derivative of logarithm of Eq. (16) and using the score function relationship shown in Eq. (7):
(x = Ex(x,, y)
E(y) p(x:ly)
log Po (x = loog (x,, y) log E(y) + logp(x;ly)
Vx, log Po (x+l)) = Vx, log Eg (X+, + Xx, logp(x;
(31)
(x1, t.
=Vxi log F, (x Eo (x( t,y)
VI Ci
vI ā
Eo (x,, f,y) = .o(xx,,,y) - v1 C4 Vx. log E, (x,,)).
13

---

REG: Rectified Gradient Guidance for Conditional Diffusion Models
D. A Approximation Error: Proof of Theorem 4.2 and Theorem 4.3
Proof of Theorem 4.2. We first notice that when using a deterministic sampler in the reverse denoising process, Ex Ex(xe,)) y) becomes Ro ($o. y). where Xo is the estimate of X at time t. Thus, we have:
ael Vx, log y) Vx, log Re y)ll vI ae V log (Ro: y) Vx, log Ri (x,: ylll
I v ā Xx log Ro (xo. y) Vxo log Ro (x y)ll
(32)
+ 1 a V vn log Ro (Ro,y) Vx, log Ro (xx
1 c V x, log Ro (x, - Vx, log R2 (x,, y)ll
Here we take CFG as an example, while similar proof can be adopted for AutoG and any CFG-variant guidance methodolo- gies. For clarity, using the score function formula Eq. (7) with the reward R₂ in CG shown in Eq. (9), we can derive:
dr Va log (a y) = w (ce(a, y) - Eg <g(a, 1)) Now let us deal with the right hand side of Eq. (32) one by one:
(33)
ddo
RHS Term 1 = v1 - Vv &o
log (ko
Uso log x))xol))l
(Chain rule)
dx:
d&o
< v1 0zl
I|| V log Ro Rx y)ll
(Extract the same term)
dx
dko
S w|l
I|| (xt, 0,y) cs(x0.0))
(Use Eq. (33))
dx:
dxo
< 2w B|
I||,
(Eg.e is bounded by B)
dx:
(34)
RHS Term 2 = w |eea (x, y, - <x 0) - cn (x,, (.y) + (x,, 0)l
(Use Eq. (33))
< w |lea (x. 0,y) - cx(xx, 0,y) + w || 0) ($t.
(Triangular inequality)
< L |2م xell,
(co.1 is Lipschitz continuous)
RHS Term 3 = (X1, 0,y) -Eo(x,, 0) - Ex(xx, + (x)x (xe,
(Use Eq. (33))
5 w ||Ea Xe: 0,y) - Ex 4, y,, + - 1) Ea EX,, 0)ll
(Triangular inequality)
< Lw.t.
(Co.s is Lipschitz continuous)
Combining them together, we obtain:
dRo dx;
V1 C+e Vxx log Fx,,,, y) - Vx, log R,(x,,))) S Bw:B|
I||
+ 2w. 180 - xx|1 + 2w...
(*)
(35)
If we denote A = Xo Xx,, then we obtain the bound shown in Eq. (18) in the main text. When the one-step prediction
Eo Jaee (xt VT aico (x,, y)) works well (e.g., the end of denoising, is close to 0), we can further simplify the
upper bound (*):
(x) BwB
DEg (x1, 1, (xe,
2wL
[[(1 [ v1- āi
||+
11(1
V
vI (xee(xx + 2wLt
Vav
Ox:
Var
BwB
Dep dEo(Xi, t,
2wL
(1 Vā + VI aell
|1) +
[[1 - vā.)) |x||| + vI a (X,, t)ll + 2wLt
Vūt
Ox.
Vāa-
B
+ 2w: T,
(36)
(1 Vor + v1 - b.
[11 V ||x|| + v1 B B| + 2wLt
Vai
Vag
1wL, (1- Vāz) |x|l +
2w
4w BL
(1 vār) +
vI Ee + 2wLt.
Vāu
Vau
Vār
C2
By taking expectation of the above inequality with respect to pa (x+ly), we obtain:
E Epo(s.ly) E0g,el]- = (p. -V Vx, log Er (x), - Vx, log (x,,,,
< (x [Ci||x:|| +
(37)
= C1 Epo(xi 1Ix1 + C2.
14

---

REG: Rectified Gradient Guidance for Conditional Diffusion Models
Proof of Theorem 4.3. Using the definition of Eg,, and Eb given in Eqs. (10) and (17), we obtain:
Rd (xx,,)) E. yx
(x.(xxly) Eg,r] = /1 ae. (xo(x Vx, log
(38)
Now let us simplify the right-hand side. To begin with, we notice that chain rule states:
2 Oog (xs.))) () dx.
@po(xely)
yx) dxe + /polxll)) y))
Ô 0xx
/ (o(xxly) log Re(x,, y)
log
dx.
(39)
0xt
0x:
y)
Moreover, the right-hand side of the above equation actually integrates out the variable xt, leaving it dependent solely on y. Consequently, the derivative with respect to x5 becomes zero, and Eq. (39) simplifies to:
(x)(xxly)
Ru (xxx
(x 0,
log
dx, + (x Vx. log
8x:
E, (x::)) y)
Es (x,, y)
RO (xx. y)
Re (x dx.;
= (x)(x))))
Vx. x. log
po(xxly) Vx. log (x,ly) log E, (X,,
E, (x,, y
(40)
R₂ (xx,
Rs (X,,
(x)()
Vx. xx log
Epo(xclt) Vx, lo log
E, (Xx: y)
Ex(,,, y)
R₂ (Xx.
- - co(xx,, t.y)
Re (x,,
Exo(x.ly)
Xx, log
log E, (x:,
E, (xx: y)
v1 ās
Thus, combining this equation with Eq. (38), we obtain:
Rí (x,y))
(x) Ea,r] = lEa, log E, (xx,
(41)
E. Experimental Settings and Additional Numerical Results
E.1. 1D and 2D Synthetic Examples
ID Synthetic Example. The ID synthetic example is inspired by the work of Kynkäänniemi et al. (2024). We generate a [x] 1.5, 0.25²), and 8,000 samples from the unconditional data distribution ((x) = 0.5 x (xr - 1, 0.5²) + 0.5 x A (x 0.5²). ). For learning the conditional and unconditional distributions, we define separate noise prediction networks, (z t. y) and cœ t). It is possible to follow the convention in CFG (Ho & Salimans, 2022) by using a weight noise prediction network with label dropout during training, and we find significant difference compared our implementation in this specific case. We emphasize that this example only considers single class with label y 0, and, similar to (Kynkäänniemi et al., 2024), regard the total number of classes and the distribution of other classes irrelevant.
total of 8,000 samples from the conditional distribution q(xly = 0) 0.5 x (z]0.55 0.25²) + 0.5 x (Nx
-0.5
005
-0.5
40.5
-1.0
-1.0
0.5
-1.5
-1.5
115
-1.5
-1.0
-2.5
Goldan leg.
kog Et ano REB
fi
-2.5
-2.0
Goldon O' S
Fog RL wio RBG
-2.0
-2.5
Golden V. Rp
Vogg
log who REG
-2.0
Daldon' Dog
Viag
Dfe wib REG
-1.5
Go Goiden Thg Tlag wi REG
Vag
-2.5
-2.0
-1.0
0.5
2.0
-1.0
2.0
-1.0
0.5
2.0
-1.0
0.5
2.0
0.5
2.0
(n) =
20
(b)
16
(c)
12
(d) =
(0)
Figure 5. Guidance values are plotted along the X-axis in the range [-1.0, 2.0) at different time steps.
In our noise prediction networks, employ sinusoidal embeddings for time, class labels, and the coordinate input, each with a dimension of 128. These embeddings are concatenated and passed through an MLP with three hidden layers, each having 128 hidden units. We use 20 time steps with 3 linearly scheduled from 0.001 to 0.2 (i.c., is linear from a = 1 - 0.001 to 025 = 1 - 0.2 in the DDPM notation). Note that for experimental purposes, we deliberately limit the capacity of the diffusion model by reducing the number of time steps since diffusion models with sufficient capacity can nearly perfectly learn these examples without any guidance. The network is trained using AdamW W with a learning rate 0.001 for 200
15

---

REG: Rectified Gradient Guidance for Conditional Diffusion Models
epochs. To evaluate V.. x, log E, v) at specific time step 4, follow the denoising process generate 100 instances of ao given observing 2t at I, and then evaluate the mean of their reward, according Eq. (14).
We have plotted comparisons of the golden V log E,, V log R, with REG, and V log R, without REG at various time steps in Figure 5 to complement Figure in the main text. The fitting results are further shown in Figure 6.
-1.0
0.5 w/o guidance
2.0
-1.0
0.5 (b) CFG w/o REG
0.5 (c) CFG w/ REG
Figure 6. Histograms of 8000 samples drawn from trained simple diffusion model shown for three cases: (a) without any guidance, (b) using the vamilla CPG without REG, and (c) using the vanilla CFG with our proposed REG. The target conditional data distribution 0.5 x (0.5, 0.25?) + 0.5 (1.5, 0.25?). While all approaches correctly recover the mean locations, method achieves better alignment of the peak heights, more accurately reflecting the target's equal magnitude Gaussian components.
2D Synthetic Example. We are inspired by Pärnamaa (2023) and design the 2D synthetic example follows: We prepare the target shape (e.g. dinosaur) and extract the 2D coordinates from its SVG file. Then, we define a Gaussian mixture with equal weight coefficients as the data generation distribution, with the mean of Gaussian mixture component being every extracted grid coordinate, and the standard deviation being 0.05. This setup makes sure we can evaluate the log probability for a given Xg. this example, we define single noise prediction network with label dropout (dropout probability of 0.2) during training, following the convention of CFG (Ho Salimans, 2022). Similar to the 1D example, sinusoidal embeddings for time, class labels, and the coordinate input, each with dimension of 512. These embeddings are concatenated and passed through an MLP with three hidden layers, each containing 128 units. We use 25 time steps with
33 linearly scheduled from 0.001 to 0.2 (i.e., ae transitions linearly from a1 = 1 - 0.001 to = 1 - 0.2 in the DDPM notation). The diffusion model is trained using the AdamW optimizer with a learning rate of 0.0001 for 200 epochs.
Evaluating golden log E, performed via numerical integration using 500 samples. Empirically, we observe that unusually large values of V log E, occasionally appear. Increasing the number of samples in the numerical integration (e-g., from 100 500) helps mitigate this issue, it provides dimensional spaces. However, using 500 samples is already computationally expensive, it requires not only evaluating log E, but also computing its gradients through backpropagation. Thus, we exclude samples with abnormally large V log E, values when plotting Figure 2 as a straightforward remedy.
accurate approximation of the expectation in higher-
Since V log Eg in this case represents the derivative of a scalar with respect to a 2D vector, it can be visualized as gradient arrow in a 2D plane. However, as shown Table 1, while the proposed REG consistently provides a better approximation of the optimal solution, this improvement barely visible in columns (d)-(f) of Figure
E.2. Quantitative Results: Image Generation
We have summarized the models used our experiments in Table 2. In the main text, REG is derived using the e- prediction parametrization, where we simplify the Jacobian matrix by using X 1 (x: VI ārEa,)), leading to Eq. (21). For EDM2 (Karras et al., 2024b), which uses the Xy parametrization, the REG correction term trivially becomes 0(1 (x(x,, I, /xx, where Dp represents the Xo-prediction network as defined EDM2 (Karras et al., 2022; 2024b). Additionally, SD-v1-4 and SD-XL employ the PNDM sampler (Liu al., 2022) and the Euler Discrete sampler, respectively, which differ from DDPM. Nevertheless,
/ot
Eq. (21) in these cases well, and it performs effectively in practice.
We conduct our experiments using the open-source DiT (Peebles & Xie, 2023), EDM2 (Karras et al., 2024b), and Huggingface Diffusers codebases, modifying their source code to support various guidance techniques. The interval CFG (Kynkäänniemi et al., 2024) study provides the interval ranges for DiT and EDM2, which we adopt here for consistency. For the "bad" models used in AutoG, we rely on the publicly available checkpoints shared by Karras et al. (2024a). However, when the interval range or the "bad" model checkpoint is unavailable (e.g., text-to-image tasks), we cannot reproduce the
16

---

REG: Rectified Gradient Guidance for Conditional Diffusion Models
interval CFG and AutoG baselines. This limitation arises because interval CFG requires exhaustive nyper-parameter search identify the interval range, and AutoG relies carefully designed "bad" model. For linear CFG, the guidance weight to zero at the start of denoising and gradually increase the weight scale toward the end. For cosine CFG (Gao et al., 2023), we use a power of 2 for class-conditional ImageNet generation and power of 1 for text-to-image generation, as we find that a power of 2 produces extremely poor results in text-to-image generation. In class-conditional ImageNet generation, we evaluate FID and IS metrics using 50,000 generated images, following the protocols outlined in DiT (Peebles & Xie, 2023) and EDM2 (Karras et al., 2024b). text-to-image generation, we randomly select one caption per image from the COCO-2017 validation dataset, creating 5,000 pairs of images and captions. FID and CLIP scores are evaluated using TerchMetries.
Additional Pareto front results for class-conditional ImageNet generation are presented in Figure 7. Qualitative examples of generated images corresponding to various text prompts in the text-to-image generation task are illustrated in Figures 8-10.
4.5
- Linear CFG wi REG
5,0
--
Vanilla CFG
4,5
Cosine CFG wil REG CFG REG
Linear CFG wi REG
4.0
+ Linear CFG w/ REG
Varilla CFG
4.0
4.0
Linear CFG ai REG
3.5
3.5
3.5
3.0
3.0
3.0
2.5
2.5
3.0
2.5
EDM2-S 512
2.0
EDM2-XXL 512
2.0
EDM2-XXL 512
280
300
280
300
320
340
360
300
340
360
380
300
320
340
360
380
IS
5.0
1.5
2.6
Cosine wio Cosine REG
4.0
Linear CFG wio REG
Liner CFG wi REG
Vanila CFG wl REG
Vanili CFG wio REG
1.4
AutoG wlo REG
AutoG w/ REG
2.4
3.5
1.3
2.2
3.0
1.2
2.0
1.8
2.5
1.1
DIT-XL/2 256
2.0
EDM2-S 512
EDM2-XXL 512
1.0
EDM2-XXL 512
290
300
310
320
330
280
300
320
260
300
340
380
260
1S
Figure 7. Additional results for the Pareto front of FID versus IS are shown by varying the guidance weight w over a broad range for different methods. Curves positioned further toward the bottom-right indicate superior performance.
17

---

REG: Rectified Gradient Guidance for Conditional Diffusion Models
w/o REG
w/ REG
Vanilla CFG
Cosine CFG
Linear CFGs
Figure 8. Generated images by different methods are shown under high guidance strength given the prompt "A medieval knight riding a glowing unicomn through stormy battleficld".
18

---

REG: Rectified Gradient Guidance for Conditional Diffusion Models
w/o REG
w/ REG
Vanilla CFG
Cosine CFG
Linear CFGs
Figure 9. Generated images by different methods shown under high guidance strength given the prompt "A floating island with giant free whose roots hang down into the clouds".
19

---

REG: Rectified Gradient Guidance for Conditional Diffusion Models
w/o REG
w/ REG
Vanilla CFG
Cosine CFG
Linear CFGs
Figure 10. Generated images by different methods are shown under high guidance strength given the prompt "A dog wearing sunglasses and riding skateboard on sunny street".
20


Reviews:
[{"review_id": "DtwSioW6Jx", "reviewer": "Reviewer_BefU", "summary": "This submission focuses on demystifying the classifier-free guidance for diffusion models. CFG has proven to be essential for the success of diffusion models. However, recent literature noted that the guided score function does not correspond to the forward diffusion process. In this work, the authors identify the source of the discrepancy and then introduce guidance for the joint distribution. This results in guidance relying on the expected reward at $x_0$ at every timestep of sampling. Using such formulation directly would be computationally prohibitive, as it would require completing the denoising process to the terminal state, at every timestep. However, under mild assumptions, which can be met in practice, a convenient approximation is proposed, which the authors call “rectified gradient guidance”. REG comes at the cost of computing the diagonal of the Jacobian of the denoiser. The proposed method is supported by strong evidence in toy, controlled scenarios, as well as for state-of-the-art class- and text-conditional image generation diffusion models. In addition, the authors show the standard CFG can be seen as an approximation of REG, and characterize the approximation error.\n\n## update after rebuttal\nMy assessment remains positive after the rebuttal. The authors engaged in the discussion and provided additional details that were requested.", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you so much for acknowledging our contribution and the constructive feedback.\n\n---\n\n**Q1.** Reference [1]: Krunoslav Pavasovic et al., arXiv 2025.\n\n**A1.** Thanks for bringing [1] to our attention. We are aware of this work and will include it in our references. As the reviewer kindly noted, this paper was published on arXiv on February 11th, after the ICML submission deadline. [1] shows that while CFG may reduce diversity in low-dimensional settings, it becomes effective in high-dimensional regimes due to a \"blessing of dimensionality.\" The authors identify two phases: an early phase where CFG aids class selection, and a later phase where it has minimal impact. They propose non-linear CFG variants that deactivate in the second phase, improving quality and diversity without extra computational cost. \n\n---\n\n**Q2.** Include samples with the golden guidance in Figure 2.\n\n**A2.** Thanks for the great question. We respectfully clarify that Figure 2(a) already displays the golden/target samples that we aim to generate, and Figure 2(d) presents the golden guidance $\\nabla \\log E_t$. To the best of our understanding, these are all the golden cases available. We are more than happy to include any missing visualizations to address this concern.\n\n---\n\n**Q3.** Extra results on diffusion models using velocity-parametrization.\n\n**A3.** Thanks for the constructive feedback. We conduct extra experiments on SD-V2.1, a velocity-parametrized text-to-image diffusion model. Due to time and resource constraints, we perform a grid search over guidance weights and report the best FID $\\downarrow$ and CLIP $\\uparrow$ scores for each method in the table below. A more thorough evaluation (e.g., a full FID-CLIP curve) will be included in the final version if accepted.\n\n|        | (CLIP, FID) w/ REG          | (CLIP, FID) w/o REG       |\n|-|-|-|\n| Linear CFG    | (31.62, 27.83)           | (31.40, 28.94)         |\n| Cosine CFG    | (31.48, 23.32)           | (31.72, 24.54)         |\n\n\n---\n\n**Q4.** Diagonal Jacobian matrix assumption.\n\n**A4.** Thanks for the constructive remark. We agree that a diagonal Jacobian is more precise. We originally refer to a diagonally dominant Jacobian for a writing reason. From Eq. (22) to Eq. (21), three approximations are introduced, as detailed in Lines 220–224 on the right side of the paper, with the Jacobian assumption being the third. The first two approximations are essential—(2) is self-explanatory, and (1) is explained in our response to Q5. Given these, the transition from Eq. (22) to Eq. (21) is already approximate, regardless of whether the Jacobian is diagonal or diagonally dominant.\n\nWe will revise the text to refer to a diagonal Jacobian where appropriate.\n\n---\n\n**Q5.** Apply Vector-Jacobian-Product (VJP) in Eq. (22).\n\n**A5.** Thanks for the great question. In short, the application of VJP requires that the function $\\log R_0(\\hat{x}_0, y)$ be differentiable with respect to $\\hat{x}_0$. However, in current CFG-like frameworks, $R_0(\\cdot, y)$ is not arbitrary—it is specifically defined as shown in the second line of Eq. (6). This definition renders $\\log R_0(\\cdot, y)$ non-differentiable with respect to its first argument (or very consuming to evaluate). Consequently, VJP is not applicable in this case. \n\nNote that to address it, we use chain rule and approximate $\\nabla_{\\hat{x}_0} \\log R_0(\\hat{x}_0, y)$ with $\\nabla{x_t} \\log R_t(x_t, y)$, which can be further simplified via Eq. (7) and (9).\n\n---\n\n**Q6.** Runtime and memory cost.\n\n**A6.** Thanks for the great question. The tables below summarize runtime and peak memory usage of CFG and REG on a single NVIDIA A40 GPU. Runtime is reported using example batch sizes, while memory is measured with batch size 1 to isolate per-image cost. As expected, REG introduces minor overhead due to the extra gradient computation. We will add these tables in the updated paper. \n\n\n| Model | Resol. | Batch Size | CFG/REG Runtime (sec) | Increase (x) |  \n|-|-|-|-|-|  \n| EDM2-S | 64    | 8          | 25.96 / 42.99        | 1.66   |  \n| DiT-XL/2 | 256 | 8          | 59.79 / 94.23        | 1.58     |  \n| EDM2-S | 512   | 8          | 46.14 / 62.87        | 1.36      |  \n| EDM2-XXL | 512 | 8          | 49.21 / 92.60        | 1.88      |  \n| SD-V1.4 | 512  | 4          | 32.63 / 39.54        | 1.21      |  \n| SD-V2.1 | 768  | 4          | 36.55 / 59.76        | 1.64   |  \n| SD-XL | 1024   | 2          | 47.48 / 74.52        | 1.57    |  \n\n| Model | Resol.   | CFG / REG GPU Peak Mem (GB) | Increase (x) |  \n|-|-|-|-|  \n| EDM2-S | 64       | 0.87 / 1.49   | 1.71  |  \n| DiT-XL/2 | 256    | 4.15 / 5.01    | 1.21   |  \n| EDM2-S | 512      | 1.19 / 1.81      | 1.52   |  \n| EDM2-XXL | 512    | 4.59 / 7.31   | 1.59    |  \n| SD-V1.4 | 512     | 2.73 / 4.39      | 1.61   |  \n| SD-V2.1 | 768     | 2.72 / 6.51       | 2.39    |\n| SD-XL | 1024      | 6.91 / 19.49       | 2.82    |", "claims_and_evidence": "Both the claims regarding theoretical results, as well as, claims about the effectiveness of the proposed method are supported by appropriate evidence.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "I have reviewed the proofs. I focused on understanding what technique the authors used for each of the proofs. I have not thoroughly checked all derivations. I have no reason to believe they are incorrect.", "experimental_designs_or_analyses": "Yes. The synthetic experiments in section 5.1. show that the proposed relaxation has a smaller error than the alternative which is vanilla CFG. Image generation experiments in section 5.2. show that this translates to quantitative and qualitative evaluation for SOTA models. There are no issues.", "relation_to_broader_scientific_literature": "This submission continues on the path of demystifying, and trying to understand CFG. I agree with the authors that their work complements previous findings in this field. I am inclined to believe that this is the explanation we have been looking for.", "essential_references_not_discussed": "Nothing that was available at the time of submission. The authors may find [1] interesting if they didn’t know it already,\n\n[1] Pavasovic, Krunoslav Lehman, et al. \"Understanding Classifier-Free Guidance: High-Dimensional Theory and Non-Linear Generalizations.\" arXiv preprint arXiv:2502.07849 (2025).", "other_strengths_and_weaknesses": "This is a strong submission.", "other_comments_or_suggestions": "- It would be interesting to include samples with the golden guidance in Figure 2, given that it is already computed.\n- In Sec 5.2 none of the models uses v-prediction. Not that I expect a different behavior for such a model, but for an even stronger message it could be included.", "questions_for_authors": "1. L243 right column “assuming that the Jacobian matrix is diagonally dominant” - is diagonal dominance enough? I think that the assumption here is a diagonal Jacobian.\n2. On the approximation of eq 22 with eq 21: wouldn’t the efficient vector-jacobian-product be applicable here? This would eliminate the assumption mentioned in the question above.\n3. What is the computational overhead of the proposed method compared to vanilla CFG? Despite the use of approximation, I suspect that the additional computational and memory cost is significant for any reasonably sized model.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "G5oLXMYarY", "reviewer": "Reviewer_vJrA", "summary": "This paper addresses the discrepancy between the theoretical motivation and practical implementation of guidance techniques in conditional diffusion models. The authors propose a new method called Rectified Gradient Guidance (REG) to improve the performance of existing guidance methods.\n\nThe main findings of the paper include the identification of a significant gap between the theoretical derivations and practical implementations of current guidance techniques. The authors demonstrate that the commonly used marginal scaling approach is theoretically invalid and propose a joint distribution scaling objective as a valid alternative.\n\nThe key algorithmic contribution is the introduction of REG, which incorporates a novel correction term into existing guidance methods. This correction term is derived from a theoretical analysis of the optimal guidance solution and is designed to better approximate this optimal solution under practical constraints.\n\nThe main results show that REG provides a better approximation to the optimal solution than prior guidance techniques in 1D and 2D experiments. Extensive experiments on class-conditional ImageNet and text-to-image generation tasks demonstrate that incorporating REG consistently improves Fréchet Inception Distance (FID) and Inception/CLIP scores across various settings compared to its absence.\n\nThe conceptual contribution lies in establishing a unified theoretical framework for understanding guidance techniques in conditional diffusion models. The authors theoretically prove the invalidity of marginal scaling and demonstrate that established guidance implementations are approximations to the optimal solution with quantified error bounds.\n\nThe practical contribution is that REG is shown to be compatible with various guidance techniques and diffusion model architectures. The method can be easily integrated into existing diffusion pipelines and consistently enhances performance without requiring significant computational overhead.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you so much for acknowledging our contribution and the constructive feedback. \n\n---\n\n**Q1.** Runtime and memory cost.\n\n**A1.** Thanks for the great question. The tables below summarize runtime and peak memory usage of CFG and REG on a single NVIDIA A40 GPU. Runtime is reported using example batch sizes, while memory is measured with batch size 1 to isolate per-image cost. Since REG introduces one extra gradient computation on top of vanilla CFG, a moderate increase in runtime and memory usage is expected. Similar inference-time gradient calculations have also been explored in Universal Guidance (Arpit Bansal et al., 2024), albeit in a different context.\n\nWe emphasize that our main contribution lies in correcting the CFG theory, and REG serves as an empirical validation. Its practical deployment depends on the specific application and acceptable overhead. We will include these tables in the updated paper.\n\n\n| Model | Resol. | Batch Size | CFG/REG Runtime (sec) | Increase (x) |  \n|-|-|-|-|-|  \n| EDM2-S | 64    | 8          | 25.96 / 42.99        | 1.66     |  \n| DiT-XL/2 | 256 | 8          | 59.79 / 94.23        | 1.58       |  \n| EDM2-S | 512   | 8          | 46.14 / 62.87        | 1.36       |  \n| EDM2-XXL | 512 | 8          | 49.21 / 92.60        | 1.88      |  \n| SD-V1.4 | 512  | 4          | 32.63 / 39.54        | 1.21  |  \n| SD-V2.1 | 768  | 4          | 36.55 / 59.76        | 1.64   |  \n| SD-XL | 1024   | 2          | 47.48 / 74.52        | 1.57   |  \n\n| Model | Resol.   | CFG / REG GPU Peak Mem (GB) | Increase (x) |  \n|-|-|-|-|  \n| EDM2-S | 64       | 0.87 / 1.49      | 1.71     |  \n| DiT-XL/2 | 256    | 4.15 / 5.01      | 1.21    |  \n| EDM2-S | 512      | 1.19 / 1.81    | 1.52       |  \n| EDM2-XXL | 512    | 4.59 / 7.31     | 1.59    |  \n| SD-V1.4 | 512     | 2.73 / 4.39    | 1.61     |  \n| SD-V2.1 | 768     | 2.72 / 6.51       | 2.39     |  \n| SD-XL | 1024      | 6.91 / 19.49     | 2.82      |  \n\n\n---\n\n**Q2.** Experiments on diverse model architectures, applications, and datasets.\n\n**A2.** Thanks for the valuable feedback. To address architecture concern, we conduct extra experiments using SD-V2.1, a velocity-parametrized diffusion model; results can be found in A3 of Reveiwer BefU. Below is a summary of the models used, covering a wide range of settings. \n\nRegarding applications and datasets, we respectfully clarify that our choices are **consistent with current standards in the literature**. For examples, DiT (William Peebles and Saining Xie, 2023) and EDM2 (Tero Karras et al., 2024) primarily conduct experiments on ImageNet. In addition to ImageNet, Interval Guidance (Kynkäänniemi et al., 2024) and Autoguidance (Tero Karras et al., 2024) perform qualitative experiments on text-to-image tasks using SD models. Due to limited time and resource constraints, we will explore additional datasets in future work.\n\n\n| Model           | DiT-XL/2         | EDM2-S           | EDM2-XXL         | SD-v1-4         | SD-XL                | SD-V2.1           | \n|-|-|-|-|-|-|-|\n| # Params     | 675 M           | 280 M            | 1.5 B            | 860 M           | 2.6 B               | 865 M             | \n| Sampler          | 250-step DDPM   | 2nd Heun | 2nd Heun | PNDM    | Euler Discrete | PNDM      | \n| Parametrization  | epsilon | x0 | x0 | epsilon | epsilon | velocity | \n| Architecture     | Transformer      | U-Net            | U-Net            | U-Net           | U-Net               | U-Net             | \n\n---\n\n**Q3.** Comparison with recent guidance enhancements.\n\n**A3.** Thanks for the constructive suggestion. We respectfully note that the proposed REG method has already been compared with SOTA guidance techniques, such as Interval Guidance (Kynkäänniemi et al., 2024) and Autoguidance (Tero Karras et al., 2024), both of which are strong and recent baselines. As shown in Table I and Figure 7, REG is still able to improve upon these methods. We greatly appreciate suggestions from the reviewer on any specific missing guidance methods that we should compare to.\n\n---\n\n**Q4.** REG implementation details and open-source code.\n\n**A4.** Thanks for the constructive remark. We will open source our code to ensure full reproducibility, and add implementation details in the updated paper. \n\n---\n\n**Q5.** Other comments: (i) Add intuitive explanation for marginal scaling in Section 3. (ii) Add a summary table of results and architectures in numerical result section. (iii) REG with different sampling schemes and different parametrizations.\n\n**A5.** Thanks for the constructive remarks. \n\n(i) We will update our manuscript accordingly. \n\n(ii) We want to respectfully point out that Table 2 and 3 have already included all numerical results and Table 4 (in supplementary) have summarized model architectures. \n\n(iii) We respectfully note that our experiments already cover a wide range of settings. Please refer to the summary table in A2 for details.", "claims_and_evidence": "The claims made in the submission are supported by clear and convincing evidence. The authors provide theoretical analysis, mathematical derivations, and extensive experimental results to validate their claims.\n\n1. The discrepancy between theoretical motivation and practical implementation of guidance techniques is demonstrated through detailed analysis of the marginal scaling approach and its limitations (Section 3). The authors show mathematically why marginal scaling is invalid and how it conflicts with the constraints of diffusion models.\n\n2. The claim that established guidance implementations are approximations to the optimal solution is supported by Theorem 4.1, which establishes the optimal solution under joint scaling, and Theorems 4.2 and 4.3, which quantify the approximation error of current methods. The authors clearly show the gap between current practices and the optimal solution.\n\n3. The effectiveness of REG in 1D and 2D synthetic examples is demonstrated through visual comparisons (Figure 1 and Figure 2) and quantitative win ratios (Table 1). These results clearly show that REG provides a better approximation to the optimal solution than previous methods.\n\n4. The improvement in performance on class-conditional ImageNet and text-to-image generation tasks is supported by comprehensive quantitative results (Tables 2 and 3) across multiple model architectures and guidance techniques. The Pareto front analyses (Figures 3 and 4) further demonstrate the consistent improvement provided by REG.\n\n5. The compatibility of REG with various guidance techniques and diffusion model architectures is shown through experiments with different models (DiT, EDM2, SD-v1-4, SD-XL) and guidance methods (vanilla CFG, cosine CFG, linear CFG, interval CFG, AutoG). The implementation details confirm that REG can be easily integrated into existing pipelines.", "methods_and_evaluation_criteria": "The proposed Rectified Gradient Guidance (REG) method and the evaluation criteria utilized in this paper are both well-suited for addressing the problem of improving guidance techniques in conditional diffusion models.\n\nREG directly targets the identified theoretical-practical gap in existing guidance methods by introducing a theoretically justified correction term derived from optimal guidance solutions under joint distribution scaling. This approach is logical as it enhances established guidance techniques rather than replacing them entirely, ensuring compatibility with various existing methods and model architectures.\n\nThe evaluation criteria, including benchmark datasets like ImageNet and COCO, along with performance metrics such as FID, IS, and CLIP score, provide a comprehensive assessment of REG's effectiveness. The combination of synthetic and real-world datasets, together with both quantitative metrics and qualitative visual comparisons, ensures a thorough evaluation of the method's impact across different scenarios and applications.", "theoretical_claims": "I've carefully examined the theoretical claims and  proofs presented in this paper, focusing on the key theoretical contributions that form the foundation of the proposed REG method.\n\nThe theoretical framework begins by identifying a critical issue in existing guidance techniques: the discrepancy between the theoretical motivation based on marginal distribution scaling and the practical implementation. The authors demonstrate that marginal scaling is theoretically invalid due to the constraints of the diffusion model's denoising process. This is established through a detailed analysis of the reverse denoising process and the implications of attempting to scale marginal distributions at different time steps.\n\nThe paper then introduces the concept of joint distribution scaling as a valid alternative. Theorem 4.1 is central to this argument, establishing the form of the optimal noise prediction network under joint scaling. The proof of this theorem is rigorous and follows logically from the definition of the scaled joint distribution objective. The authors demonstrate the existence and uniqueness of the transition kernels corresponding to the scaled joint distribution, and derive the form of the updated noise prediction network. This theorem provides the theoretical justification for the REG method.\n\nBuilding on this foundation, Theorems 4.2 and 4.3 analyze the approximation error of existing guidance methods compared to the optimal solution. These proofs are technically sound and provide quantitative bounds on the approximation error. The analysis considers the practical constraints of guidance implementation and shows how the lack of future foresight affects the accuracy of guidance signals. The proofs involve careful application of mathematical analysis and probability theory, considering the properties of the diffusion process and the guidance rewards.\n\nThe derivation of the REG correction term is also well-supported theoretically. The authors start from the optimal guidance equation and make reasonable approximations to derive a practical correction term that can be implemented in existing diffusion frameworks. The chain rule application and Jacobian simplifications are justified, and the empirical validation in the experiments supports the effectiveness of these approximations.", "experimental_designs_or_analyses": "The experimental designs and analyses in this paper are generally sound and valid, providing comprehensive support for the proposed REG method across different scenarios and applications.\n\nFor the 1D and 2D synthetic examples, the experimental design effectively demonstrates REG's improvement in a controlled setting where ground truth can be computed. The comparisons are fair and isolate the effect of the guidance method, with win ratios and visual comparisons providing clear evidence of REG's effectiveness.\n\nIn class-conditional ImageNet generation, the evaluation is comprehensive, testing across multiple resolutions and model architectures. The use of standard metrics like FID and IS is appropriate, and the Pareto front analysis effectively shows how REG improves the efficiency of guidance techniques. The consistent improvement across different models and guidance methods strengthens the validity of the claims.\n\nFor text-to-image generation on COCO-2017, the experimental design is appropriate, evaluating REG with different model architectures and relevant metrics. The qualitative examples complement the quantitative results, providing additional evidence of improved generation quality.\n\nWhile the experimental designs are robust, there are minor areas where additional analysis could enhance the work. More detailed analysis of computational overhead, especially for larger models, would be beneficial. Additional ablation studies could better isolate the contributions of different components of REG. Including more diverse and challenging prompts in text-to-image generation would further demonstrate REG's robustness. Comparing REG against other recently proposed guidance enhancements would also be valuable.", "relation_to_broader_scientific_literature": "REG can be seen as an evolution of classifier-free guidance (Ho & Salimans, 2022) with a theoretically motivated correction term. Unlike previous enhancement techniques like AutoG (Karras et al., 2024a), which requires identifying a \"bad\" model version, REG provides a general correction applicable to various guidance methods without additional model training or complex setup.", "essential_references_not_discussed": "None", "other_strengths_and_weaknesses": "# Strengths\n\n1. The paper demonstrates originality by identifying and addressing a fundamental theoretical-practical gap in guidance techniques for diffusion models. While classifier guidance and classifier-free guidance have become standard approaches, this work provides a novel theoretical framework that re-examines their foundations. The identification of the invalidity of marginal scaling and the proposal of joint distribution scaling represent creative advances that build upon but significantly extend previous work.\n\n2. The improvements demonstrated by REG across multiple benchmarks and model architectures indicate substantial practical significance. For conditional diffusion models, which are widely used in applications like image generation, text-to-image synthesis, and video generation, even small improvements in guidance techniques can lead to meaningful enhancements in output quality and diversity. The versatility of REG, as shown in the experiments, suggests it could become a standard enhancement in future diffusion model implementations.\n\n3. The paper is well-written and structured logically. The theoretical sections build upon each other in a coherent manner, making complex concepts accessible. The experimental results are presented clearly with appropriate visualizations and quantitative analyses. The appendices provide additional details that enhance reproducibility and understanding.\n\n# Weaknesses\n\n1. While the paper mentions that REG introduces minor computational overhead, a more detailed analysis of the additional computational requirements would be beneficial, especially for larger models and in production settings. This could help practitioners better understand the trade-offs when implementing REG.\n\n2. The paper demonstrates REG's effectiveness across several model architectures and guidance methods, but a more extensive analysis of its performance across a broader range of architectures and applications would strengthen the claims of generalizability. Additionally, testing on more diverse and challenging datasets beyond ImageNet and COCO could provide further insight into its robustness.\n\n3. The paper could benefit from a more comprehensive comparison against other recently proposed guidance enhancements. This would help establish REG's position relative to other state-of-the-art methods and highlight its unique advantages.\n\n4. While REG is presented as a versatile enhancement, the implementation details, particularly regarding the correction term, might require careful tuning and understanding of the underlying diffusion framework. Providing more implementation guidance or open-source code could lower the barrier to adoption for practitioners.", "other_comments_or_suggestions": "1. In Section 3, when discussing the invalidity of marginal scaling, adding a brief intuitive explanation alongside the mathematical proofs might help readers better grasp the concept.\n\n2.  In the experimental sections, providing a summary table that compares the performance of REG across different guidance methods and model architectures could help readers quickly see the consistent improvements.\n\n\n\n3. Including more specific implementation details about the REG correction term, particularly regarding computational considerations, would be helpful for practitioners looking to implement the method.\n\n4.  While the paper mentions computational overhead, a more detailed discussion of potential limitations, such as increased memory requirements or compatibility with certain model architectures, would provide a more complete picture.", "questions_for_authors": "The paper mentions that REG introduces minor computational overhead but doesn't provide specific details. Could you quantify the additional computational requirements of REG compared to standard guidance methods, especially for larger models like SD-XL? This would help clarify practical trade-offs. If the overhead is substantial, it might affect the practical significance despite performance improvements.\n\n\nThe experiments demonstrate REG's effectiveness across several model architectures, but how does REG perform with score-based generative models that use different parameterizations or sampling schemes? Understanding its performance on architectures beyond those tested would clarify its generalizability.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ofe9NbQcOa", "reviewer": "Reviewer_BraT", "summary": "This paper introduces a Rectified Gradient Guidance to improve conditional generation in diffusion models. Also the paper provides the theoretical foundation of the proposed guidance method. Experiments show the proposed method enhances the quality of generated images.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you so much for acknowledging our contribution and the constructive feedback. \n\n---\n\n**Q1.** Runtime and memory cost.\n\n**A1.** Thanks for the great question. The tables below summarize runtime and peak memory usage of CFG and REG on a single NVIDIA A40 GPU. Runtime is reported using example batch sizes, while memory is measured with batch size 1 to isolate per-image cost. Since REG introduces one extra gradient computation on top of vanilla CFG, a moderate increase in runtime and memory usage is expected. Similar inference-time gradient calculations have also been explored in Universal Guidance (Arpit Bansal et al., 2024), albeit in a different context.\n\nWe emphasize that our main contribution lies in correcting CFG theory, and REG serves as an empirical validation. Its practical deployment depends on the specific application and acceptable overhead. We will include these tables in the updated paper.\n\n| Model | Resol. | Batch Size | CFG/REG Runtime (sec) | Increase (x) |  \n|-|-|-|-|-|  \n| EDM2-S | 64    | 8  | 25.96 / 42.99    | 1.66    |  \n| DiT-XL/2 | 256 | 8   | 59.79 / 94.23    | 1.58    |  \n| EDM2-S | 512   | 8  | 46.14 / 62.87    | 1.36   |  \n| EDM2-XXL | 512 | 8   | 49.21 / 92.60   | 1.88    |  \n| SD-V1.4 | 512  | 4   | 32.63 / 39.54  | 1.21  |  \n| SD-V2.1 | 768  | 4   | 36.55 / 59.76   | 1.64  |  \n| SD-XL | 1024   | 2 | 47.48 / 74.52    | 1.57   |  \n\n| Model | Resol.   | CFG / REG GPU Peak Mem (GB) | Increase (x) |  \n|-|-|-|-|  \n| EDM2-S | 64       | 0.87 / 1.49        | 1.71       |  \n| DiT-XL/2 | 256    | 4.15 / 5.01         | 1.21       |  \n| EDM2-S | 512      | 1.19 / 1.81      | 1.52       |  \n| EDM2-XXL | 512    | 4.59 / 7.31      | 1.59   |  \n| SD-V1.4 | 512     | 2.73 / 4.39        | 1.61     |  \n| SD-V2.1 | 768     | 2.72 / 6.51     | 2.39     |  \n| SD-XL | 1024      | 6.91 / 19.49        | 2.82    |  \n\n---\n\n**Q2.** Extra visualization results. \n\n**A2.** Thanks for the constructive remark. We respectfully point out that Figures 1, 2, 5, 8, 9, and 10 already provide qualitative visualizations for synthetic 1D/2D cases and real benchmarks. To address this concern, extra visualizations (e.g., standard 2D toy dataset, text-to-image and class-conditioned generation) are performed and added in this [anonymous link](https://anonymous.4open.science/r/icml-1655-rebuttal).\n\nPlease note that due to OpenReview rebuttal format constraints, we are unable to include figures directly in this response. However, anonymous links are permitted under ICML guidelines, and we will include these visualizations in the revised manuscript.\n\n--- \n\n**Q3.** Standard public 1D and 2D datasets and 2D generation accuracy.\n\n**A3.** Thanks for the helpful feedback. Our 1D setup follows standards from prior works, such as Interval Guidance (Kynkäänniemi et al., 2024) and Autoguidance (Karras et al., 2024). For 2D, we use custom shapes with fine-grained structures, more challenging than standard 2D toy datasets like “two moons” or “Swiss rolls.” Extra results on standard 2D datasets are performed and can be accessed via this [anonymous link](https://anonymous.4open.science/r/icml-1655-rebuttal) (anonymous links permitted by ICML guidelines).  \n\nAbout generation accuracy, we clarify that Figure 2(b) appears reasonably well when compared to the target in Figure 2(a). We acknowledge that the results may not be perfect, as we use a relatively small diffusion model in order to compute the golden gradient $\\nabla \\log E_t$ efficiently. We also clarify that the metric in Table 1 indicates how often REG achieves lower error compared to no REG. A value x\\%>50% suggests consistent improvement.\n\n--- \n\n**Q4.** Cosine and linear CFG perform worse than vanilla in SD models and DiT-XL/2. \n\n**A4.** Thanks for the great question. We first want to clarify that our experiments are designed to verify whether REG can enhance a given guidance method. This argument has been validated by the results shown in Table 2 and 3. Our aim is not to compare “cosine + REG” vs. “vanilla CFG,” since such a comparison conflates the base method’s performance with the effect of REG. Hence, we believe the essential question here is that cosine and linear CFG perform worse than vanilla CFG in SD models and DiT-XL/2 (they do perform better in EDM2). \n\nThis trend is consistent with results shown in related works, such as the analysis of CFG weight schedulers in Xi Wang et al., TMLR 2024 --- Both our Figure 4 (right) and their Figure 7(c) show that vanilla CFG achieves the lowest FID, linear CFG shifts slightly upward, and cosine CFG shifts it further upward in CLIP-FID space.\n\nTwo factors likely contribute: (i) all these methods are heuristic, and their effectiveness can vary across model architectures and datasets. (ii) linear and cosine CFG require tuning two hyperparameters, while vanilla CFG uses only one—making linear and cosine CFG more expensive and less robust to optimize.\n\nWe will include this clarification in the updated paper.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "Yes, the theorems in section 4.", "experimental_designs_or_analyses": "Yes", "relation_to_broader_scientific_literature": "The proposed Rectified Gradient Guidance is related to the guidance techniques in diffusion models, like classifier guidance and classifier-free guidance.\n\nThe paper replaces the scaled marginal distribution target with a valid scaled joint distribution objective, aligning with the theoretical motivations of guidance methods. This theoretical advancement builds on prior works that have explored the statistical theory and mathematical foundations of conditional diffusion models", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "Strengths:\n\n1. The paper is well-written and easy to follow.\n\n2. The paper provides the theoretical insight about the limitations in current guidance method and proposes a solution to improve the performance of existing guidance methods.\n\n3. The authods conduct extensive experiments to justify the effectiveness of the proposed method.\n\nWeaknesses:\n\n1. The paper lacks reporting on inference time requirements. The authors should clarify whether the proposed REG introduces computational overhead compared to baseline methods.\n\n2. The visualization results presented in the paper are insufficient to fully demonstrate the method's effectiveness. Additional qualitative examples across diverse scenarios would strengthen the paper.\n\n3. For the 1D and 2D experiments, the evaluation would be more convincing if conducted on standard public datasets rather than custom data. The reported accuracy for 2D generation also appears quite low.", "other_comments_or_suggestions": "None", "questions_for_authors": "1. In Table 3, why do most of the guidance methods exhibit worse performance than vanilla CFG, particularly in the context of the SD model?\n\n2. Same problem also appear in the DiT-XL model.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "We4hknpYul", "reviewer": "Reviewer_U91Q", "summary": "This paper studies the foundations of classifier-free guidance (CFG). It finds that the common interpretation of scaling marginal distribution $p(x_t \\vert y)$, i.e., use guidance to change sampling trajectories,  is not theoretically grounded as it is impossible to construct a DDPM process that corresponds to the scaled distribution. The paper then proposes to scale the joint distribution $p(x_{0:T} \\vert y)$, which could be proved to produce a valid DDPM process. With this new framework, CFG can be treated as a special approximation\n\nThe paper then proposes a new form of approximation to the theoretically-grounded formulation, i.e., REG, and demonstrates its effectiveness on 1D and 2D synthetic data as well as high-dimension image synthesis tasks, i.e., ImageNet generation.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you so much for acknowledging our contribution and the constructive feedback. Below we address each concern raised.\n\n--- \n\n**Q1.**  Where marginal distribution scaling is mentioned in CFG paper?\n\n**A1.**  Thank you for the great question. As a quick recap, Section II of our paper explains the theoretical pitfall of guidance. We begin by examining the case of scaling only the terminal marginal distribution (Eq. (5)) and then generalize to scaling all marginal distributions (Eq. (8)). The latter is a stricter goal that subsumes the former. We show that neither is theoretically well-justified.\n\nStrictly speaking, Ho & Salimans (2022) state that classifier guidance (CG) leads to sampling from scaled versions of all marginal distributions—see the second equation under Algorithm 1 on page 4. This aligns with our Eq. (8)–(9), where their reward term $p_\\theta(c | z_\\lambda)^w$ corresponds to our notation $R_t(x_t, y) = p_\\phi(y | x_t)^w$ in Eq. (9). They also express the CFG reward using $p^i(c | z_\\lambda)$ in the second-to-last paragraph before Section 4. Thus, the CG and CFG reward forms are indeed explictly stated there.\n\nFor presentation clarity, we choose to start with the simpler case of scaling only the terminal marginal (Eq. (5)), but we acknowledge that Eq. (8) is the original formulation in Ho & Salimans (2022).  We will revise the text and footnote accordingly to make our claim more accurate.\n\n--- \n\n**Q2.** The qualitative visualizations in Fig. 8-10.\n\n**A2.** Thanks for the great question. To elaborate: in Figure 8, the knight’s gesture while holding the sword appears unnatural, and the horse’s legs are blurry in the baseline images. In Figure 9, there are black artifacts in the sky (not birds), and the lower part of the tree suffers from low contrast in the baseline images. In Figure 10, the dog’s paws are unnatural, and the sunglasses are deformed in the baseline images.\n\nWe agree with the reviewer that these differences might be subtle. For additional context, it might be helpful to consult qualitative results reported in other works to put our qualitative results in context --- see, for example, Figure 6-7 of [1], Figure 4 of [2], Figure 1 of [3], and [4].\n\nTo better address the concern, we have followed the suggestions and added multiple samples with the same method for a given prompt. These can be found in File 1 and File 2 at this [anonymous link](https://anonymous.4open.science/r/icml-1655-rebuttal). \n\nPlease note that due to OpenReview rebuttal format constraints, we are unable to include figures directly in this response. However, anonymous links are permitted under ICML guidelines, and we will include these visualizations in the revised manuscript.\n\n--- \n\n**Q3.** Apply REG to flow-matching based diffusion models.\n\n**A3.** Thank you for the insightful question. We agree that evaluating REG in the context of flow-matching models would further enhance the impact of our work.\n\nDue to time and resource constraints—particularly the large model size (SDv3 has over 2 billion parameters) and its lengthy inference time—we were unable to run experiments on SD v3. However, we conduct extra experiments on SD v2.1, a velocity-parameterized diffusion model. This setup was mutually requested by other reviewers and, we believe, is also relevant to the current question. Namely, while velocity-parameterized models are not identical to flow-matching approaches, they are closely related. In fact, the distinction between them can often be seen as a subtle difference in the noise schedule [5]. We perform a grid search over guidance weights for different methods with SD v2.1 and report the best FID $\\downarrow$ and CLIP $\\uparrow$ scores for each method in the table below.\n\nWe will perform thorough evaluations on REG for flow-matching based generative models and report them in the final version if the paper is accepted.\n\n|        | (CLIP, FID) w/ REG          | (CLIP, FID) w/o REG       |\n|-|-|-|\n| Linear CFG    | (31.62, 27.83)           | (31.40, 28.94)         |\n| Cosine CFG    | (31.48, 23.32)           | (31.72, 24.54)         |\n\n---\n\n**References**\n\n[1] Tuomas Kynkäänniemi et al., 'Applying Guidance in a Limited Interval Improves Sample and Distribution Quality in Diffusion Models,' Neurips, 2024.\n\n[2] Tero Karras et al., 'Guiding a Diffusion Model with a Bad Version of Itself,' Neurips, 2024.\n\n[3] Tianwei Yin et al., 'One-step Diffusion with Distribution Matching Distillation,' CVPR, 2024.\n\n[4] Xi Wang et al., 'Analysis of Classifier-Free Guidance Weight Schedulers,' TMLR, 2024.\n\n[5] Ruiqi Gao et al., 'Diffusion Meets Flow Matching: Two Sides of the Same Coin', https://diffusionflow.github.io/.", "claims_and_evidence": "The paper is well-written and the claims are well-supported. \n\nI have one question about Sec. 3's \"Common Interpretation of Guidance\": I do not think that the original CG or CFG only scales the marginal distribution of $p(x_0 \\vert y)$. However, the authors state\n> (L161 left) CG and CFG rewards are explicitly stated in (Ho & Salimans, 2022)\n\nCan authors point me to the specific part where **marginal distribution** scaling is mentioned in CFG paper?", "methods_and_evaluation_criteria": "The evaluations are solid and thorough.", "theoretical_claims": "I read through all the proofs and found they are detailed and easy to follow.", "experimental_designs_or_analyses": "The experiments are extensive and convincing.", "relation_to_broader_scientific_literature": "The paper falls in the recent community effort to fundamentally understand the mechanism of CFG.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "The paper is solid in theory and thorough in experiments. I do not find a major weakness in the paper besides my questions spread in other sections.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "People have applied CFG to flow-matching-based approaches by modifying the velocity functions [a] and the velocity has close connections to score functions. Thus I am wondering whether REG could be easily applied to a model based on flow matching.\n\n1. If the answer is affirmative, I am wondering whether the authors have the resources to apply REG to models based on flow matching, e.g., Stable Diffusion 3. This would further enhance the impact of the paper.\n2. If the answer is negative, what would be the difficulty?\n\n[a] Ma et al., SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers. ECCV 2024.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper introduces Rectified Gradient Guidance (REG), a method to correct theoretical flaws in existing conditional diffusion model guidance. It shows that previous methods incorrectly scale marginal distributions and proposes scaling the joint distribution instead, establishing a valid theoretical foundation. REG approximates this corrected optimal solution to make it practical and improve conditional generation quality. Extensive experiments validate REG's superiority, showing consistent performance improvements in synthetic and real-world image generation tasks over existing techniques.

All reviewers feel positive about this paper, mainly due to the theoretical contribution and practical approximation of the proposed method. There are some concerns regarding some experiments details, which have been well addressed in the rebuttal. Please make sure to incorporate additional results and clarifications into the final version to make the paper stronger.
Decision: Accept (poster)