Overshoot and Shrinkage in Classifier-Free Guidance: From Theory to Practice
Authors: Krunoslav Lehman Pavasovic
Jakob Verbeek
Giulio Biroli
Marc Mezard
Venue: ICLR 2026 Poster (2026)
Keywords: Diffusion
flow-matching
classifier-free guidance
Primary area: generative models

Abstract: Classifier-Free Guidance (CFG) is widely used in diffusion and flow-based generative models for high-quality conditional generation, yet its theoretical properties remain incompletely understood. By connecting CFG to the high-dimensional framework of diffusion regimes, we show that in sufficiently high dimensions it reproduces the correct target distribution—a “blessing-of-dimensionality” result. Leveraging this theoretical framework, we analyze how the well-known artifacts of mean overshoot and variance shrinkage emerge in lower dimensions, characterizing how they become more pronounced as dimensionality decreases. Building on these insights, we propose a simple nonlinear extension of CFG, proving that it mitigates both effects while preserving CFG’s practical benefits. Finally, we validate our approach through numerical simulations on Gaussian mixtures and real-world experiments on diffusion and flow-matching state-of-the-art class-conditional and text-to-image models, demonstrating continuous improvements in sample quality, diversity, and consistency.

Published as a conference paper at ICLR 2026
OVERSHOOT AND SHRINKAGE IN CLASSIFIER-FREE GUIDANCE: FROM THEORY TO PRACTICE
Krunoslav Lehman Pavasovic
FAIR at Meta, École Normale Supériure, Paris [krunolp] (meta. com
Jakob Verbeek* FAIR at Meta, Paris
Giulio Biroli éicole Normale Supériure, Paris
Mare Mézard Bocconi University, Milan
ABSTRACT
Classifier-Free Guidance (CFG) is widely used in diffusion and flow-based gener- ative models for high-quality conditional generation, yet its theoretical properties remain incompletely understood. By connecting CFG to the high-dimensional framework diffusion regimes, we show that in sufficiently high dimensions it reproduces the correct target distribution a "blessing of dimensionality" result. Leveraging this theoretical framework, we analyze how the well-known artifacts of mean overshoot and variance shrinkage emerge in lower dimensions, charac- terizing how they become more pronounced as dimensionality decreases. Build- ing these insights, propose simple nonlinear extension of CFG, proving that it mitigates both effects while preserving CFG's practical benefits. Finally, validate approach through numerical simulations Gaussian mixtures and real-world experiments on diffusion and flow-matching state-of-the-art class- conditional and text-to-image models, demonstrating continuous improvements sample quality, diversity, and consistency.
No CFG
Power CFG Stand. CFG
Figure I: Qualitative comparison of unguided sampling, standard Classifier-Free Guidance (CFG), and our proposed non-linear power-law CFG (DiT/XL-2 on ImageNet- 1K 256 x 256). Standard CFG increases fidelity at a substantial expense to diversity and semantic meaning compared to unguided CFG. Our power-law guidance improves fidelity at cost to semantics or diversity. Each column sample starts from the same seed.
INTRODUCTION
Diffusion (Sohl-Dickstein et al., 2015; Song & Ermon, 2020; Ho al., 2020) and flow- methods (Lipman et al., 2022; Albergo et al., 2023; Liu et al., 2022) have emerged as the de facto state-of-the-art for generating high-dimensional signals. Diffusion relies on Uhstebn-Uhlenbeck Langevin dynamics, where noise is progressively added to the data until it becomes completely random. New samples are generated by reversing this process through a time-reversed Langevin
"Joint last author.

---

Published as a conference paper at ICLR 2026
Stand. CFG, d=200
Stand. CFG, d=5
Nonlin. CFG, d=5
0.4-
1=00
1.00
$=000
Density
0.3
d= 1.00
d= 1.00
0.85
0.83
0- 0.92
0.91
w 0
1.5
0.2
20
0.1
0.0
10.0
12.5
15.0
17.5
20.0
0
2
2
Figure 2: CFG accurately generates the target distribution in high dimensions, causes mean overshoot and variance shrinkage in low dimensions, which are mitigated by nonlinear CFG. We simulate back- ward process using a two-Gaussian mixture with target mean m = (1.1....) ) and variance o² 1, and project the generated samples onto the target mean, q(t = 0) = I - m//ma. Left: in high dimensions (d = 200), CFG accurately recovers the target distribution. Center: in low dimensions (d = 5), CFG exhibits mean overshoot and variance shrinkage. Right: Our proposed nonlinear extension of CFG reduces these arti- facts, partially restoring the target distribution.
equation. This backward evolution is steered by a force, the score, estimated from the data. In contrast, flow matching circumvents the diffusion construction by directly specifying the probability paths between noise and data. This is done by regressing onto a target vector field which in turn generates the desired probability paths. An important task for both paradigms is generating data conditioned
class label or textual description of the image content. This can be achieved through conditioning mechanisms in the model architecture, as well as guidance techniques (Dhariwal & Nichol, 2021; Ho & Salimans, 2022) that steer the generation process towards samples aligned with user intentions or desired properties.
The notion of guidance was first introduced in classifier guidance (Song et al., 2020; Dhariwal & Nichol, 2021), where a pre-trained classifier is leveraged to induce class conditioning in the sampling of unconditional models. Relying on a pre-trained classifier is, however, computationally expensive and requires classifiers robust noisy samples. Classifier-free guidance (CFG) (Ho Salimans, 2022) was developed as an alternative, and was quickly adopted as a standard technique (Nichol al., 2022; Betker et al., 2023; Saharia et al., 2022; Esser et al., 2024). CFG does not rely on an auxiliary classifier, instead, the model is trained to generate unconditional and conditional samples, and at inference extrapolates the denoising path towards the conditional one. Using CFG, however, the process is no longer guaranteed to sample the original conditional distribution.
CFG affects the generation process mainly in two ways; theoretically. this has been characterized in case of Gaussian mixtures one and finite dimensions, where CFG causes mean overshoot, causing samples more shifted towards the boundary of the class, and variance shrinkage, sulting in a sharper distribution than the target one (Chidambaram et al., 2024; Xia et al., 2024; Wu et al., 2024; Bradley & Nakkiran, 2024). These two effects linked closely with the effects that the practicioners have observed, with CFG steering the samples towards the "mode" of high- quality and input-consistent samples corresponding directly the well-observed increased satura- tion/contrast, while also reducing sample diversity in the process (Astolfi et al., 2024; Saharia al., 2022).
Theoretically, it is unknown whether the unwanted effects can be dampened and whether in fact CFG can ever generate the correct distribution. In practice, CFG has indeed shown beneficial regardless of these effects, would be useful if similar guidances existed that reduce these effects, while keeping the practical benefits of increased quality of the generated samples. In this work, we provide theoretical analysis of the properties of the distribution generated by CFG, and how they compare to the target one. We examine how these effects arise, what influences them, and whether they can dampened. Our results are backed by numerical simulations, and finally we test how do our findings extend to real world settings.
In summary, contributions are the following:
result showing that, in infinite and sufficiently high di- mensions, -GG-guided trajectories generate the correct distribution, one that is generated by un- guided conditional trajectories. This is established by relating CFG to the emergence of dynamical regimes (Biroli et al., 2024). In this setting, we show that CFG accelerates convergence of samples toward the target class.
(1) We provide
2

---

Published as a conference paper at ICLR 2026
Evolution of diffusion paths
Potential
Init. distr.
5
Target distr
Data
lI
Time t
-2
[g: speciation
-2.5 0.0
Forward time
9
Figure 3: Dynamical regimes in diffusion. Left: Illustration the speciation phenomenon using one- dimensional Gaussian mixture. Starting from pure Gaussian noise at large time t, the backward diffusion begins in Regime I, where the class has not been decided yet. After speciation time &, (dashed line), the class membership decided. Right: Evolution of the effective potential (conditional potential in Eq. (6) over time for high-dimensional Gaussian mixture showeasing the symmetry breaking phenomenon.
(2) We demonstrate that, as dimensionality decreases, the commonly-observed artifacts of mean overshoot and variance shrinkage appear (illustrated on a toy Gaussian example Fig. 2). We precisely characterize these effects and support our theoretical analysis with numerical simulations.
(3) Finally, we introduce a simple nonlinear generalization of CFG and prove theoretically that it mitigates both mean overshoot and variance shrinkage. We further show its superiority numerical simulations (as in Fig. 2, right), and validate its practical benefits state-of-the art diffusion and fow-matching models, achieving improved sample quality, consistency, and diversity.
Our results can intuitively be described as follows: as the dimension grows, the "attraction force" for each class grows exponentially. One can visualize well-separated class acting as magnet that pulls diffusion trajectories towards itself (the attraction force depending on the mean and variance magnitude). During the backward process this force becomes very strong when the system starts to approach one class. Unless the magnitude the CFG guidance scale also grows exponentially with the dimension (which is not the case practice), these natural attraction forces completely dominate the CFG term. The backward dynamics and the attraction forces are portrayed in Figure 3 (left and right respectively).
2 RELATED WORK
Having introduced CFG, Ho & Salimans (2022) highlighted the trade-off between image quality, measured by Fréchet inception distance (FID, Heusel et al. (2017)), and diversity, measured by inception score (Salimans al., 2016) when adjusting the guidance strength parameter w. Since then, significant body of research has examined CFG from various perspectives.
Theoretical works on CFG. Several works employed Gaussian mixture models (GMMs) ana- lyze diffusion and guidance (Shah et al., 2023; Liang et al., 2024; Cui et al., 2023; Bai et al., 2024; Song et al., 2020). In contrast, Du et al. (2023) explored alternative conditioning, while Bradley & Nakkiran (2024) characterized CFG as a medictor-correcto (Song et al., 2020). Most relevant to this work, Chidambaram al. (2024) demonstrated CFG's mean overshoot and variance shrinkage in one-dimensional settings, while Wu et al. (2024) extended the findings to multi-dimensional set- tings using GMMs. We expand on these by developing a high-dimensional statistical analysis and precisely characterizing how these effects diminish as dimensionality increases, ultimately demon- strating that the CFG-generated distribution in fact aligns with the target one for d + oo.
CFG variants and experimental analyses. Among experimental analyses of CFG, Karras et al. (2024a) propose guiding generation using a less-trained version the model. Kynkäänniemi et al. (2024) apply CFG during limited interval, and Wang et al. (2024) use weight schedulers for the classifier strength parameter. Several other CFG alternatives have been proposed, such as rectified guidance (Xia al., 2024), projected score guidance (Kadkhodaie et al., 2024). characteristic guid- ance (Zheng & Lan. 2023), second-orde CFG (Sun et al., 2023), CADS (Sadat et al., 2023), CFG++ (Chung al., 2024), REG (Xia al., 2024), APG (Sadat al., 2024) and Feedback Guidance
3

---

Published as a conference paper at ICLR 2026
(Koulischer al., 2025). In later sections, demonstrate proposed nonlinear CFG generalizes to these methods, consistently enhancing their performance.
Dynamical regimes, statistical physics and high-dimensional settings. Statistical physics meth- ods have shown particularly useful in analyzing high-dimensional generative models, egg., data from Curie-Weiss models (Biroli & Mézard, 2023), high-dimensional Gaussian mixtures (Biroli et al., 2024), and hierarchical models (Sclocchi et al., 2024). Furthermore, several recent works studied dynamical regimes diffusion models (Biroli & Mézard, 2023; Raya & Ambrogioni, 2024; Biroli Huang, 2024; & Chen, 2024; Aranguri et al., 2025), however none of them analyzed the effects brought by classifier-free guidance.
al., 2024; Sclocchi et al., 2024;
BACKGROUND AND HIGH-LEVEL DISCUSSION
We begin by providing an overview of the standard framework for generative diffusion, serving as the foundation for our analysis. Our exposition focuses on diffusion, though our findings directly extend to flow-matching with Gaussian paths (see, e.g., Lipman et al. (2024), Sec. 4.10.2).
3.1 GENERAL SETUP
Let {ā+ E Rd represent n independent data points sampled from the true underlying data dis- tribution (a) The forward diffusion process, starting from data points {ā, {- modeled by an Mastein-Uhlenbeck process, described by the following stochastic differential equation (SDE):
dt (tt) dt + v2 dB((t)
where dBB(t) denotes the standard Brownian motion in Rdd At any given time t, the state Z(t) is distributed according to Gaussian with mean de and variance A, 1 c-2r The forward process is terminated at time ty > 1, when #(ty) is effectively pure Gaussian noise, distributed as (0, Ia), with Ia being the identity matrix in Rd.
The backward diffusion process operates in reverse time T = t, - described with:
dī(r) = dr + 25( dr v2d
where 5(z,t) V log (z) denotes the score function. The backward diffusion process generates points 7 sampled from the distribution (T) for every time step т. At the end of the backward process, i.e., when r = t,. the process generates points drawn from the original distribution Po-
In this work, we focus on generating data that can be categorized into distinct classes. We begin by assuming that the underlying data distribution a d-dimensional probability distribution Po c), where represents a discrete class index and H d-dimensional vector. The aim is to generate data conditioned on c, the class label. The procedure that mathematically guaranteed to generate the exact conditional target distribution consists of using the true conditional score, S,(ī, = V log P. (ølc) in Eq. (2). CFG, however, does not do that; it instead further directs diffusion in manner proportional to the difference between conditional and unconditional scores:
S SCFG (Iz c) S.(F.) AS
A == w (S, S,))).
While CFG has shown practical benefits, such improved fidelity and classification confidence (Wu et al., 2024), several key questions remain open: (1) Can one establish a theoretical frame- work proving that CFG indeed generate the target distribution? (2) If so, this framework also account for the well-documented artifacts of mean overshoot and variance shrinkage? (3) If these artifacts are indeed inherent to standard CFG, can one design alternative guidance schemes that provably mitigate them and, crucially, do such schemes also deliver practical improvements beyond the theoretical setting? In this paper, we draw on the statistical -hysicss framework of Biroli & Mézard (2023); Biroli et al. (2024) to provide affirmative answers to all of the above.
3.2 CONNECTING DYNAMICAL REGIMES OF DIFFUSION TO CLASSIFIER-FRES GUIDANCE
To analyze CFG, we build on the ttatitical-phyyse framework of Biroli & Mézard (2023); Biroli al. (2024), which identifies three dynamical regimes in diffusion models. Our exposition focuses
4

---

Published as a conference paper at ICLR 2026
on the first two, since we show that CFG has the same effect in the second and third regime. These regimes are distinguished by the presence or absence of symmetry breaking, characterized via the leading cigenvalue of the data covariance matrix.
dynamical regimes in diffusion. Using this framework, Biroli al. (2024) analyze the dynamical regimes of the backward process in Eq. (2) for two classes in the - oo limit. They identify the speciation time i, as the transition between Regime and Regime II. In Regime I, the backward trajectories have not yet committed to particular data class, while in Regime II, they have committed and begin generating the class-specific features necessary to limit)
Previous findings:
produce samples. The core structure of these regimes persists (in the well beyond Gaussian mixtures, extending to models such as data lying on manifolds (Ventura et al., 2024; Bae et al., 2024; George et al., 2025).
Our findings: connecting dynamical regimes to CFG. A central insight of our work is that the dynamical-regime framework provides a principled perspective for understanding CFG, from which three main results emerge:
Result I. In sufficiently large and infinite dimensions, CFG generates the correct targer distribution. Specifically: (i) Before speciation time ts CFG accelerates convergence toward the target class. (ii) Just before (, CFG-guided paths realign with the unguided trajectory that produces the correct distribution. (iii) After tg. CFG has no effect on the generation process.
Result II. In finite dimensions, mean overshoot and variance shrinkage arise. Using Gaussian mix- tures, characterize how the overshoot amplitude increases as dimensionality decreases and how CFG modifies the dynamics' potential, resulting in reduced variance of the generated distribution.
Result III. There exist many simple guidances that effectively mitigate these effects. Specifically, introduce simple nonlinear extension of CFG that reduces overshoot and variance shrinkage by multiplying the score difference A in (3) with |AS(FG > 0. This modification is theoretically justified and effective in simulations on Gaussian mixtures. As shown in Sec. 5, is also beneficial in real-world applications.
While this framework provides valuable insights, it has key limitation: it relies on access to the exact score. As a result, although explains how mitigate overshoot and variance shrinkage and supports nonlinear guidance schemes that improve practical performance, it does not clarify why standard CFG despite these artifacts often achieves strong empirical results. Addressing this question is beyond the scope of our current work, as it would require tools or analyses not available within our framework. Nonetheless, our theoretical analysis and nonlinear guidance show how to control these effects, providing meaningful theoretical insights and practical improvements.
MAIN RESULTS
The framework of Biroli & Mézard (2023); Biroli et al. (2024) characterizes dynamical regimes using a simple two-GMM. Its strength lies in broad applicability: such regimes appear across diverse generative models and data modalities (Ventura et al., 2024; George et al., 2025; Bae et al., 2024), and the framework has been validated on both real data and manifold-suppooredd distributions (Biroli & Mézard, 2023). We provide its generalizations to non-centen cenntredd/uuliicooppe C.I. C.2), heterogeneous variances, and nigh-dimensional manifold -utifold-structuredd data (Appx. C.3).
mixtures (Appx.
We focus here on the simplest case-two Gaussians of equal weight and isotropic variance-whin already captures the essential phenomena and serves as the foundation for our theoretical framework.
THEBRETICCL FRAMEWORK
We examine the case where (a is a superposition of two Gaussians with equal weight, means Im and isotropic variance o². To ensure the two Gaussians are well separated, we take the large d limit with fixed values of |//l/// and a¹. As mentioned above, we assume exact scores are available.
In this setting, the speciation transition between Reg. and II occurs on timescales i, = 1/2 log Biroli et al. (2024) showed that corresponds to the time at which diffusion paths commit to
'The choice |m| vdd is standard for modeling data with well-defined classes and has been adopted in several previous works (Ii & Chen, 2024; Wu al., 2024; Shah al., 2023; Biroli al., 2024).
5

---

Published as a conference paper at ICLR 2026
specific class, as determined by changes in the potential of the backward Langevin equation (see Fig. 3, Appx. A). This speciation time plays a central role in our first contribution: before ts, CFG guides trajectories toward the target, accelerating convergence. while just prior to 4, guided and unguided paths become aligned. After (s, trajectories naturally follow unguided paths, ensuring that CFG produces the correct target distribution in infinite and sufficiently high dimensions.
analysis also reveals two additional phenomena: (i) how finite- dimensional corrections give rise to mean overshoot and variance shrinkage, which we characterize exactly, and (ii) how nonlinear extension of CFG can mitigate these effects.
Building this foundation,
In practice our framework generalizes effectively to text-to-image models where one can view T2I learning as learning a mixture of zero-variance Gaussians corresponding to unique prompt-image pairs. In the presence of noise or non-unique mappings, this representation evolves into a standard Mixture of Gaussians (MoG). Given that MoGs are universal approximators, this formulation offers a strong theoretical basis for not only condicondi,,,nal, but also text-to-image training, which we later show by evaluating on various state-of-the-an models.
In the following, we present the theoretical arguments underlying these three results in concise, conceptual manner, with the full derivations and proofs deferred to Appx. B through Appx. E.
4.1
KEY FINDING I: CFG GENERATES THE CORRECT DISTRIBUTION INFINITE AND LARGE DIMENSIONS
To establish our first result, we analyze the distribution of 7 at time f, which can be written as ( - me ')2
(x' + me ')2
Pt( x exp
+ exp
21'
21'
wi - -1)e- 2t this setting, the CFG score (Eq. (3)) simplifies to
SCFG ( c) = - + 5
cmte
w me-
Pe c CC- tanh
(T,
F. me
(4)
P.
with e +1 and w > Our analysis develops the first result through three successive steps, all linked to the speciation time i, = log d, the point at which trajectories commit to specific class.
Step I: CFG guides trajectories before speciation. As we show in Appx. B.1, before 4, CFG provides an extra push toward the target class, accelerating convergence. Importantly, seen in the CFG score formula (4), CFG only affects the m directions; all directions orthogonal to m re- main unaffected. Formally, projecting the backward dynamics onto vector e 1 m shows that the resulting dynamics are independent of w. confirming only the m. direction is influenced.
Therefore, we project onto m and define q(t) I m/ ml with |m] evolution guided toward class c = satisfies
Vd. Then the backward
dq = 2 - (ty-ts- ((1 + w) w tanh (ge (6--)-))))))))
dn(T),
(5)
where T ty and dn(r) denotes 22 times Brownian motion (specific theoretical arguments
provided in Appx. B). This be rewritten in terms of an effective potential /CHO (q, T):
VCG =
2 o2 2 (1-b6) tW
(t-t,) + In cosh ge (1-b)
(6)
Conditional potential
CHG--induced potential
Examining the effective potential in (6), we observe that the CFG-induced term provides addi- tional push along the m direction toward the target class c == 1. This effect is especially significant for trajectories that deviate from typical paths, correcting those that would otherwise move toward the wrong class (see Fig. 7 in Section B). These observations allow us to conclude that, prior to the speciation fg. CFG amplifies the push toward the desired class, establishing Step
Step II: CFG paths align before exiting Regime I. At late times in Regime I, q becomes of order vi (Biroli & Mézard, 2023), and the CFG-added term in Eq. (4) produces only exponentially small

---

Published as a conference paper at ICLR 2026
corrections to the dynamics. Although different values of may have led to different positions for q earlier in the backward process, these differences are quickly forgotten. The full theoretical argument is spelled out in Sec. B.2, i.e. we provide the proof of the dynamics naturally readjusting, and the trajectories converging toward the same distribution they would follow without CFG.
Step II is therefore obtained by analyzing the corresponding SDE starting from this late stage of Regime I: we show its solution no longer depends the earlier CFG-induced deviations and coin- cides statistically with the backward evolution of single Gaussian corresponding to the target class c +1, implying the path alignment (see Fig. 8 in Section B for numerical simulations).
Step III: After speciation time t., CFG paths follow the unguided path. At the end of Regime I. as we show in Sec. B.3, the projection q diverges, and one has to analyze the rescaled variable 7 m/d. By this stage, the CFG-guided paths have realigned with the unguided conditional paths.
Specifically, we use results from Biroli al. (2024) to show that in Regime II the extra CFG term in Eq. (4) effectively vanishes, since - tanh (7 - (Te, + 0 as d - co. As a result, the dynamics as well as the trajectories follow the unguided backward evolution, yielding Step III.
Consistent conclusions in large, finite d. Within Regime I, for large, yet finite dimension, the CFG-added-term in the score in Eq. (4) remains of the same order as the conditional score of the unguided path so CFG has the same effect as in the infinite limit. The full proof is provided in Appx. D; when exiting Regime I and during Regime the extra CFG term is exponentially small in d, so the results from all three steps carry over for large but finite d (we discuss below the strength of the corrections for finite d).
In summary, during Regime I, CFG accelerates convergence toward the target class. Just before speciation, the paths realign with the unguided trajectory, subsequently following them throughout Regime II. This behavior is illustrated in Section B, Fig. 8. This result shows that, contrary to previous beliefs (Chidambaram et al., 2024; Wu et al., 2024), CFG can indeed generate the correct distribution, and serves as key guiding principle for the remainder of our work.
4.2 KEY FINDING II: CFG EFFECTS IN FINITE DIMENSIONAL SETTINGS
So far, we have shown that for any value of w the target distribution is correctly reproduced in the infinite dimensional limit. We now show that the mean overshoot and variance shrinkage arise as the dimensionality of the system decreases. Full arguments can be found in Appx. D.
Mean overshoot and variance shrinkage in finite In finite dimensions, the paths do not realign when exiting Regime I. The additional push introduced by CFG within Regime I has an effect on Regime II, resulting in an overshoot of the target distribution of relative amplitude of order 1/vd. The CFG-added also results in a larger second derivative of the potential /CFG t). Thus, the CFG Langevin equation is associated to more confining potential, ultimately shrinking the variance of the CFG-generated distribution. These are in line with previous empirical (Ho & Salimans, 2022) and theoretical findings (Chidambaram et al., 2024; Wu et al., 2024). In Appendix Figs. 8 and 12, we further analyze how CFG behavior changes with increasing dimensionality and number of classes both for numerical simulations and real-world scenarios. This concludes our second finding.
4.3 KEY FINDING III NONLINEAR POWER-LAW CFG MITIGATES FINITE-DIM. EFFECTS
Building on the finite-dim. deviations identified in Key Finding II, we propose Power-Law CFG: simple nonlin. extension of standard CFG. Our scheme raises the cond. score difference (3) along m a power 0, allowing guidance scale with the local strength of the conditional signal:
(Z, (, + (,
(,)))
Intuitively, this modification has two complementary effects: (i) When the score difference 6S, (ā, C) (z)) (vector norm) is small and potentially unreliable, guidance is naturally dampened. (ii) When the signal is strong, guidance is amplified, enhancing the push toward the correct class.
Importantly, as a direct consequence of our large-d theoretical analysis, nonlinear plwer-law CFG retains the exact high-dim. guarantees of standard CFG: in the limit of large dimension, it recovers
7

---

Published as a conference paper at ICLR 2026
the correct conditional distribution without distortion, ensuring that the benefits of nonlinear scaling are confined to the mnite-dmeensionll regime. This is discussed in Appx. H.
Reduced overshoot and shrinkage. Our analysis in App. E identifiies the key term (gg) == de=(ts-t) /Tu(q) as the driver for mean overshoot and variance shrinkage. This term is intrinsically linked to the Hessian (curvature) of the potential, and the effect of nonlinear guidance is shown to directly modify B(q), with < B(g) < almost everywhere. Mean Overshoot: Overshoot proportional to B(q) and the parameter a controls this through B(g)=: > 0 suppresses the overshoot, while a < 0 increases it. Variance Shrinkage is positively correlated with (g)): B(g) determines the ratio of curvature increments between nonlinear and linear CFG through B(q)". Specifically, a > 0 reduces the curvature increase (decreasing shrinkage), while a < 0 amplifies it (increasing shrinkage). Simulations (see, e.g., Fig. 2 (right) or Fig. 35) confirm this.
Application to flow matching. The power-law formula can be directly applied flow-matching as well; in order be fully consistent with our theoretical proposition, we expressed the Power-Law formula as dr (s) = (==)as² rather than (s(s) = 8a, However, even using Φr(s) = 8 (which is a valid guidance within our framework) yields very similar results and behavior this is described in detail in appendix G.2.1.
Magnitude vs. Directional Scaling. Our Power-Law formulation uses the Fuclidean norm (T)) (,) which captures both score direction and magnitude, unlike, e.g.- cosine met- ries that would isolate alignment. While the Euclidean norm is resolution-senan -sessiivee,, luckily, the guidance parameter w acts as a renormalization factor, which ultimately removes the need for an additional explicit resolution-based adjustments. As detailed in App. G.8, our analyses showed that this combined signal yields superior performance compared purely directional metrics, though alternative ausine-based methods remain promising avenue for future work.
4.4 DISCUSSION: OTHER NONLINEAR EXTENSIONS
Beyond the pawer-law modification, our analysis indicates a broad class of theoretically valid, po- tentially beneficial nonlinear guidances. Our framework and the "blessing of dimensionality" en- suring CFG recovers the correct target distribution in high dimensions, are not limited to the specific construction above: many nonlinear extensions share the same guarantees.
Formally, our results can be extended to nonlinear guidances of the form
5C SCIG-NL (Z, = S((,)) [S.(7,c) - (.) de ( (((t)))((z)))
(8)
under the condition lim.- (s) 0. ensuring guidance vanishes smoothly when cond. and uncond. scores coincide. This condition essentially relies on the part of our proof showing that scores equalize low noise levels (as often observed in practice), preventing pathological guid- ances that break this property). This framework recovers number of existing methods as special cases: constant de (s) w yields standard CFG; (s) = w (,),,,)) (1) gives limited-interval CFG (Kynkääänneemi al., 2024); and time-varying (s(s) W/ recovers weight schedulers (Wang al., 2024; Gao et al., 2023). Other recent proposals (Chung et al., 2024; Xia et al., 2024; Ventura et al., 2024) can also expressed as specific, simple choices of Φt. Importantly, all of these remain linear in the score difference dS,, whereas our nonlinear power-law scheme S.d demonstrates that altering the score difference in nonlinear manner is both theoretically natural and practically beneficial.
This perspective opens larger design space of alternative nonlinear functions di, which could as well be directly optimized, yielding theoretically sound guidance mechanisms, potentially outper- forming existing approaches. Building on this, both controlled GMM settings and large-scale generative models, validating theoretical predictions, assessing robustness, and comparing directly to linear CFG and alternative CFG schemes.
evaluate the nonlinear power-law guidance in
5 NUMERICAL SIMULATIONS AND REAL-WORLD EXPERIMENTS
We now turn to experiments, testing nonlinear CFG derived from our large-d theoretical guidelines. Through num. simulations and real-world experiments, we evaluate if the theoretically motivated power-law CFG delivers tangible benefits, e.g., in mitigating overshoot and preserving variance.
8

---

Published as a conference paper at ICLR 2026
Simulations: Gaussian Mixtures
Real-world, Cond: EDM2-5
Real-world, T2/M:
1.00
1.00-
0.75
0.99
0.99
0.75
0.75
0.75
0.25
0.25
0.50
0.25
0
0.2
0.
0.25
0.00
0.0
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.8
0.8
1.0
Sampling time
Sampling time
Sampling time
Figure 4: The flexibility of nonlinear power-law CFG, key ingredient in our theoretical argument, ap- pears consistently in simulations and real-world models. Left: Num. simulations on GMMs. Center: Class-conditional EDM2-S trained on ImageNet- K. Right: text-to-image model with MMDiT architecture. All exhibit the same hump-shaped behavior of [[S. c) - (z)) ++x consistent with our theoretical analysis.
Experimental details. We examine power-law CFG in GMM simulations and four generative models: DiT (Peebles & Xie, 2023) and EDM2 (Karras al., 2024b), trained and evaluated on ImageNet-1K (res. 256 and 512). We also consider two text-to-image (T2IM) models: first is trained on ImageNet-Ik and CC12M (Changpinyo
al., 2021), evaluated CC12M, using the diffusion DDPM training objective (Ho et al., 2020) with MMDiT architecture (Esser et al. (2024), similar to SD3). The second model, using MMDiT scaled to 1.6B parameters, trained with flow matching on YFCCI00M (Thomee et al., 2016), CC12M and proprictary dataset of 320M Shutterstock images, evaluated on COCO dataset (Lin et al., 2014). Section G contains another T2IM model trained with DDPM objective with the MDTv2 (Gao et al., 2023) architecture scaled to 800M parameters.
Comparing GMM simulations to real-world experiments. Fig. 4 shows both GMM simulations and large-scale gen. models exhibit the same characteristic hump-shaped behavior of the amplitude of the guidance term [ c)- St (z) [| suggesting our theoretical insights might be beneficial in practice. Crucially, the power-law param. a allows to alter the shape of these curves, offering the precise flexibility our analysis identifies as necessary for improving guidance. We further examine this in Fig. 35 in Appx. H. showing nonlin. CFG results in faster convergence, with paths of smaller Jensen-Shannon divergence to the target (across all time 7), while also reducing the overshoot.
Power-law CFG is robust. We per- form sensitivity analysis, showing that large values of a consistently yield im- proved performance, increasing robust- ness and stability when tuning for w.? This is shown in Fig. 5 for EDM2- S. in Section G for DiT/XL-2, two T2IM models, together with further ab- lation studies showing that non-lin. CFG consistently outperforms standard CFG when varying number of sampling steps.
3.5
8
2.6
3.0
0.2
2.4
22
2.5
0.6
0.9
2.0
2.5
5.0
10.0
Figure 5: Sensitivity analysis (EDM2-S, ImageNet-11
512 x 512). Left: Incrcasing parameter a consistently improves FID standard CFG (a 0.). Right: Increasing yields more stable FID values
larger range of w.
Power-law CFG improves image quality and diversity. We quantitatively evaluate our method using FID (Heusel et al., 2017) measuring image quality, and precision and recall (Sajjadi et al., 2018) measuring diversity. In Table 1, we compare power-law CFG to state-of-the-art guidance methods. As pawer-law
easily combined with other guidances, we test combining it with CADS (Sadat et al., 2023) and Limited-guidanee (Kynkäänniemi et al., 2024), the strongest com- petitors. Power-law CFG improves over standard CFG most cases and improves results of CADS and lim.-interval guidance. We provide qualitative results in Fig. 6, observing that power-law im- proves both quality and diversity, while being more robust to changing w.
Latent-space vs Pixel-space. The pawer-law method consistently showed robust improvements with the nonlinear parameter 0.9 set across all experiments in latent space. For pixel space (see Table 11 in Appx.), the optimal seemed to fluctuate slightly more and tuning as well as resulted stronger improvement in performance. While Power-Law demonstrates consistent
Although power-law CFG introduces another hyerparameter, we did not have to perform extensive hyperparameter search, and found large values, e.g., 0.9 to consistently perform well.

---

Published as a conference paper at ICLR 2026
Table 1: Power-law CFG often improves both fidelity and diversity metrics. We applied power-law to stan-
dard CFG, limited and CADS variants, as the two were strongest competitors. Applying power-law improved their performance further, achieving competitive results. Best results are bolded, second best underlined. (t)
indicates power-law CFG improves the guidance method compared to its version with stand, CFG, while (L)
means the metric deteriorated. T2IM represents lext-to-image models, CC class-conditional; FM short for flow- matching objective and diff. stands diffusion. Experimental details are provided in Section
HDMDS IM- 512)
D/UXL-2 IM-IK 256
DHI MMDT
(T2M (T2MM OC12m)
FM MMDN (22M COCO;
Susdled (Ho Salimas,, 2022)
2.29
0.751
0.582
2.20
0.525
0.594
8.58
0.661
0.569
5.30
0.629
0.594
Schedaler (Warg al. 3024)
2413
0.762
(1551
2.14
0.540
0.014
8:30
0.681
0.559
5.01
0.600
0.21
Linited dknkkaniizii et 2024)
1.87
0.760
0.598
1.97
0.801
0.632
8.58
0.690
0.553
5.00
0.607
0.502
Coetee (fin at
2:15
01720
4819
2.90
6.61
0.530
K.39
0.699
0.561
5.18
0.3)
0A16
CADS (Sudit 2023)
1,00
0.792
1.70
0.772
0.677
5.32
0,622
4.91
0633
0813
NEG 2004)
0.795
0.673
5.06
0.119
0619
(FCc+ (Chung 2014)
NYA
NIA
XIA
NIA
SIA
NIA
8.55
0.663
0.552
4.65
0,52
0.629
Power-los CPG (Dan)
Lo3 (1)
0.780 (1)
[L33] (T)
2,05 (1)
0.31 (1)
0.595 (1)
8.11.1) (1)
(1570(1)
0.553 (1)
(.8(
06011 (1)
(1)
Poweriba (?66- imirrd (Cren)
(17
(1)
0.00 (1)
(
(122111)
1042 (1)
8.92 (1)
0.55 (1)
(11)())
Power low CTG CADS (Dans)
1.52 (1)
(.770
0.622
1,6 (1)
(1)
0.632 (1)
799
(.002())
(1573
47)
0 (1)
2,02
Standlrrd CFO, wiak
Increasing of increases diverstyy
Irandaricc
Standland strangans.
Power low CFG, weok 2.
Proserdiae CFC strrng
Figure 6: Qualitative comparison of Standard and Power-Law CFG DiT/XL-2 trained on ImageNet- 1k (256x256). Left: while standard CFG results in diversity decrease or mode collapse (first image w = 5), power-law CFG (a 0.9) improves in diversity at no cost to fidelity, showing robustness to varying of w (note very large w = 10). Right: Increasing non-linear paramcter yields larger diversity, while preserving image quality. Experimental details with further examples (as well as text-to-image) provided in App. G.
benefits, its performance relative to other non-linear strategies remains unexplored. There may be more strategies (particularly for pixel-space) which are worth exploring.
6 CONCLUSION
By connecting CFG to the high-dimensional framework of diffusion regimes, we theoretically analyzed its behavior and showed that, in sufficiently high dimensions, CFG reproduces the correct target distribution-a the well-known artifacts of mean overshoot and variance shrinkage emerge as dimensionality decreases. Finally, we proposed simple nonlinear extension of CFG, proving that mitigates both effects while preserving CFG's practical benefits, consistently improving sample quality and diversity across state-of-the the-art text-to-image and class-conditional models.
beessing-f-ddimesionaal
result. We further demonstrated that
Limitations and future work. Our theory demonstrates that in high-dimensional settings, CFG generates the correct target distribution, extending previous results showing CFG alters it in finite- dimensions. In practice, CFG improves fidelity while reducing diversity: although our theory allows discovery of guidances that maintain strong fidelity while significantly boosting diversity, the reason why CFG-modified distribution is more effective in practice is not explained by our theory which relies on perfect score estimation. We hypothesize, therefore, that the practical benefits of (non- linear) CFG might be tied to the imperfect score estimators used in practice. Investigating how score approximation errors impact guidance effectiveness is an important area for future research.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
This work has received funding from the French government, managed by the National Research Agency (ANR), under the France 2030 program with the reference ANR-23-1ACL-0008 Further- more, this paper supported by PNRR-PE-AI FAIR project funded by the NexiGeneration EU program. We would like to thank Mathurin Videau, João Maria Janeiro, Kunhao Zheng, Theophane Valleys, Wes Bouaziz and Tony Bonnaire for fruitful discussions regarding the numerical experi- ments. We would further like to thank Levent Sagun, David Lopez-Paz, Brian Karrer, Ricky Chen, Arnaud Doucet, Yaron Lipman and Luke Zettlemoyer for feedback and support.
ETHICS STATEMENT
This study contributes to the growing body of research aimed deepening theoretical under- standing of diffusion models and their broader implications for generative modeling. By bridging the gap between theory and practice, we strive improve the performance and efficiency of these models, which have far-reaching applications in various fields.
However, as with any powerful technology, there are also potential risks associated with develop- ment and deployment of advanced generative models. The increasing sophistication of deepfakes raises concerns about misinformation, propaganda, and the erosion of trust in digital media. More- over, the misuse generative models for malicious purposes, such as creating fake identities or spreading disinformation, poses significant threats to society whole.
In light of these challenges, we hope that our paper, along with many others that aim to improve understanding of the models, will contribute to a deeper understanding of their strengths and limi- tations. We believe it is essential for developing effective strategies to mitigate the risks associated with generative models, and we hope that our work will step toward achieving this goal.
REPRODUCIBILITY STATEMENT
We have clearly stated the main assumptions underlying our work, along with their limitations and how they influence our conclusions. To support our theoretical contributions, Appendices B-E con- tain the full proofs of our claims, while Appendix G.3 lists all hyperparameter configurations re- quired to reproduce our experiments exactly. For real-world experiments, we specify the GPUs used as well as the Gflops of the models, offering transparency the computational resources required. Finally, our experiments class-conditional models rely on and reference publicly available check- points, enabling straightforward verification and further exploration by the community.

---

Published as a conference paper at ICLR 2026
REFERENCES
Beatrice Achilli, Luca Ambrogioni, Carlo Lucibello, Mare Mézard, and Enrico Ventura. Memo- rization and generalization in generative diffusion under the manifold hypothesis. arXiv preprint, 2502.09578, 2025.
Michael Albergo, Nicholas M Boffi, and Eric Vanden-Eiinden. Stochastic interpolants: unifying framework for flows and diffusions. arXiv preprint, 2303.08797, 2023.
Santiago Aranguri, Giulio Biroli, Marc Mezard, and Eric Vanden-Eijnde Optimizing noise sched- ules of generative models high dimensionss. arXiv preprint, 2501.00988 2025.
Pietro Astolfi, Marlene Careil, Melissa Hall, Oscar Mañas, Matthew Muckley, Jakob Verbeek, Adri- ana Romero Soriano, and Michal Drozdzal. ditional image generative models. arXiv preprini, 2406. 10429, 2024.
Pareto fronts con-
Stefano Bae, Enzo Marinari, and Federico Ricci- Terseenerssn A very effective and simple diffusion reconstruction for the diluted Ising model. arXiv preprint, 2407 072.6, 2024.
Weimin Bai, Yifei Wang. Wenzheng Chen, and He Sun. An expectation-maxxiiizaioon algorithm for training clean diffusion models from corrupted observations. arXiv preprint, 2407.01014, 2024.
Fan Bao, Chongxuan Li, Yue Cao, and Jun Zhu. All are worth words: a vit backbone for score-based based diffusion models. NeurlPS 2022 Workshop on Score-Based Methods, 2022.
James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang. Linjie Li, Long Ouyang, Juntang Zhuang. Joyce Lee, Yufei Guo, et al. Improving image generation with better captions. Technical report, OpenAI, 2023. URL https: / cdn. openai wwanmppaprs//eeaee dall-e-11lle 3.
Giulio Biroli and Mare Mézard. Generative diffusion in very large dimensions. Journal of Statistical Mechanics: Theory and Experiment, 2023(9):093402, 2023.
Giulio Biroli, Tony Bonnaire, Valentin De Bortoli, and Mare Mézard. Dynamical regimes of diffu- sion models. arXiv preprint, 2400.18499, 2024.
Arwen Bradley and Preetum Nakkiran. Classifier-free guidance is predictor-comrector, arXiv preprint, -408009000, 2024.
Soravit Changpinyo, Piyush Sharma, Nan Ding, and Radu Soricut. Conceptual 12m: Pushing web- scale image-text pre-training to recognize long-tail visual concepts. In CVPR, 2021.
Hansheng Chen, Kai Zhang, Hao Tan, Zexiang Xu, Fujun Luan, Leonidas Guibas, Gordon Wet- zstein, and Sai Bi. Gaussian mixture flow matching models. arXiv preprint, 2504.05304, 2025a.
Shoufa Chen, Chongjian Ge, Shilong Zhang, Peize Sun, and Ping Luo. Pixelflow: Pixel-space generative models with flow. arXiv preprint arXiv:2504.0790 2025b.
Muthu Chidambaram, Khashayar Gatmiry, Sitan Chen, Holden Lee, and Jianfeng Lu. What does guidance do? A fine-grained analysis in simple setting. arXiv preprint, 2409.13074, 2024.
Hyungjin Chung, Jeongsol Kim, Geon Yeong Park, Hyelin Nam, and Jong Chul Ye. CFG++: Manifold-constrained classifier free guidance for diffusion models. arXiv preprint, 2406.08070, 2024.
Hugo Cui, Florent Krzakala, Eric Vanden-Eijnden, and Lenka Zdeborová. Analysis of learning flow-based generative model from limited sample complexity. arXiv preprint, 2310.03575, 2023.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: large-scale hierarchical image database. In CVPR, 2009.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat GANs on image synthesis. In
Advances in neural information processing systems, 2021.
12

---

Published as a conference paper at ICLR 2026
Yilun Du, Conor Durkan, Robin Strudel, Joshua B Tenenbaum, Sander Dieleman, Rob Fergus, Jascha Sohl-Dickstein, Arnaud Doucet, and Will Sussman Grathwohl. Reduce, reuse, recycle: Compositional generation with energy- diffusion models and MCMC. In International con- ference on machine learning, 2023.
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederie Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In International conference machine learning, 2024.
Shanghua Gao, Pan Zhou, Ming-Ming Cheng, and Shuicheng Yan. Masked diffusion transformer strong image synthesizer. In ICCV, 2023.
Anand Jerry George, Rodrigo Veiga, and Nicolas Macris. Analysis of diffusion models for manifold data. arXiv preprint, 2502.04339, 2025.
Jiatao Gu, Shuangfei Zhai, Yizhe Zhang, Josh Susskind, and Navdeep Jaitly. Matryoshka diffusion models. arXiv preprini arXiv22310.15111 2023.
Melissa Hall, Oscar Mañas, Reyhane Askari, Mark Ibrahim, Candace Ross, Pietro Astolfi, Tariq Berrada Ifriqi, Marton Havasi, Yohann Benchetrit, Karen Ullrich, et al. EvalGIM: A li- brary for evaluating generative image models. arXiv preprint, 2412. 2024.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge to a local Nash equilibrium. In Advances in neural information processing systems, 2017.
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint, 2207.12599 2022.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilisticc models. In Advances in neural information processing systems, 2020. URL 11239.
Zahra Kadkhodaie, Stéphane Mallat, and Eero P Simoncelli. Feature-guided score diffusion for sampling conditional densities. arXiv preprint, 2410. 1646, 2024.
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. In Advances in neural information processing systems, 2022.
Tero Karras, Miia Aittala, Tuomas Kynkäänniemi, Jaakko Lehtinen, Timo Aila, and Samuli Laine. Guiding a diffusion model with a bad version of itself. arXiv preprint, 2406.02507, 2024a.
Tero Karras, Miik Aittala, Jaakko Lehtinen, Janne Hellsten, Timo Aila, and Samuli Laine. Analyz- ing and improving the training dynamics of diffusion models. In CVPR, 2024b.
Felix Koulischer, Florian Handke, Johannes Deleu, Thomas Demeester, and Luca Ambrogioni. Feedback guidance of diffusion models. arXiv preprint, 2506 5506000085, 2025.
Tuomas Kynkäänniemi, Miika Aittala, Tero Karras, Samuli Laine, Timo Aila, and Jaakko Lehtinen. Applying guidance in limited interval improves sample and distribution quality in diffusion models. arXiv preprint, 2404.07724, 2024.
Sangyun Lec, Beomsu Kim, and Jong Chul Ye. Minimizing trajectory curvature of ode-based gen- crative models. In International conference on machine learning, 2023.
Marvin Li and Sitan Chen. Critical windows: non-asymptotic theory for feature emergence in diffusion models. arXiv preprint, 2403.01633, 2024.
Yingyu Liang, Zhenmei Shi, Zhao Song, and Yufa Zhou. Unraveling the smoothness properties of diffusion models: Gaussian mixture perspective. arXiv preprint, 2405.16418, 2024.
Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft COCO: Common objects in context. In ECCV, 2014.
13

---

Published as a conference paper at ICLR 2026
Yaron Lipman, Ricky TQ Chen, Heli Ben- Maximilian Nickel, and Matt Le. Flow matching for generative modeling, arXiv preprint, 2210.02747, 2022.
Yaron Lipman, Marton Havasi, Peter Holderrieth, Neta Shaul, Matt Le, Brian Karrer, Ricky TQ Chen, David Lopez-Paz. Heli Ben-Hamu, and Itai Gat. Flow matching guide and code. arXiv preprins, 2412.06264, 2024.
Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint, 2209.03003, 2022.
Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. GLIDE: Towards photorealistic image generation and editing with text-guided diffusion models. In International conference on machine learning, 2022. URL.
https: www.iiie..o org/ abs /2122. 2112.11744 10741
William Peebles and Saining Xie. Scalable diffusion models with transformers. In ICCV, 2023.
Gabriel Raya and Luca Ambrogioni. Spontaneous symmetry breaking in generative diffusion mod- els. In Advances in neural information processing systems, 2024.
Seyedmorteza Sadat, Jakob Buhmann, Derek Bradley, Otmar Hilliges, and Romann M Weber. CADS: Unleashing the diversity of diffusion models through condition-annealed sampling. arXiv preprint, 2310. 2023.
Seyedmorteza Sadat, Otmar Hilliges, and Romann M Weber. Eliminating oversaturation and ar- tifacts of high guidance scales diffusion models. International Conference on Learning Representations (ICLR), 2024.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kam- yar Seyed Ghasemipour, Burcu Karagol Ayan, S. Sara Mahdavi, Rapha Gontijo Lopes, Tim Sal- imans, Jonathan Ho, David J Fleet, and Mohammad Norouzi. Photorealistic text-to-image dif- fusion models with deep language understanding. In Advances in neural information processing systems, 2022. URL https: arxiv. org/ abs/ 2205 11487.
Mehdi SM Sajjadi, Olivier Bachem, Mario Lucic, Olivier Bousquet, and Sylvain Gelly. Assessing generative models via precision and recall. In Advances neural information processing systems, 2018.
Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training GANs. In Advances in neural information processing systems, 2016.
Antonio Sclocchi, Alessandro Favero, and Matthieu Wyart. A phase transition in diffusion models reveals the hierarchical nature data. arXiv preprint, 2402. 2402.16991, 2024.
Kulin Shah, Sitan Chen, and Adam Klivans. Learning mixtures of Gaussians using the DDPM objective. Advances in neural information processing systems, 2023.
Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamics. arXiv preprint, 1503.03584 2015. URL https: / arxiv. org/ a /10... 03585.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. a19p/ http:://///iiioooo/////////////
arXiv preprint, 1907.05600. 2020. URL https: / http:///xxxxxx........
05600.
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint, 2011.13456, 2020.
Shikun Sun, Longhui Wei, Zhicai Wang. Zixuan Wang, Junliang Xing, Jia Jia, and Qi Tian. Inner classifier-fres guidance and its Taylor expansion for diffusion models. In International Confer- on Learning Representations (ICLR), 2023.
14

---

Published as a conference paper at ICLR 2026
Bart Thomee, David A Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, and Li-Jia Li. YFCC100M: the new data in multimedia research. Communications of the ACM, 59(2):64-73, 2016.
Enrico Ventura, Beatrice Achilli, Gianluigi Silvestri, Carlo Lucibello, and Luca Ambrogioni. Man- ifolds, random matrices and spectral gaps: The geometric phases of generative diffusion. arXiv preprint, 2410.05898, 2024.
Qing Wang, Sanjeev Kulkarni, and Sergio Verdú. Divergence estimation for multidimensional densities via k-nearest-neighhor distances. IEEE Trunsactions Information Theory, 55(5): 2392-2405, 2009.
Xi Wang, Nicolas Dufour, Nefeli Andreou, Marie-Paule Cani, Victoria Fernández. Abrevaya, David Picard, and Vicky Kalogeiton. Analysis of classifier-free guidance weight schedulers. arXiv preprint, 2404.13040 2024.
Yuchen Wu, Minshuo Chen, Zihao Li, Mengdi Wang, and Yuting Wei. Theoretical insights for diffusion guidance: A case study for Gaussian mixture models. arXiv preprint, 2403.01639, 2024.
Mengfei Xia, Nan Xue, Yujum Shen, Ran Yi, Tieliang Gong, and Yong- Jin Liu. Rectified diffusion guidance for conditional generation. arXiv preprint, 2410.18737, 2024.
Zhendong Yu and Haiping Huang. Nonequilbrium physics of generative diffusion models. arXiv preprint, 2405.1 1932, 2024.
Candi Zheng and Yuan Lan. Characteristic guidance: Non-linear correction for DDPM large guidance scale. arXiv preprint, 2312.07586, 2023.
Jean Zinn-Justin Quantum field theory and critical phenomena, volume 171. Oxford university press, 2021.
15

---

Published as a conference paper at ICLR 2026
SUPPLEMENTARY MATERIAL
The supplementrry material structured as follows:
In Section A, we give brief introduction to related work, focusing on Biroli et al. (2024).
In Section B, we give proofs for two equidistant, symmetric Gaussian mixtures.
In Section C, present arguments how extend the proofs to non-centered Gaussian mixtures (subsec. C.I) and multiple Gaussian mixtures (subsec. C.2).
In Section D. we present the theoretical and numerical findings for finite dimension (in- cluding low dimension d).
In Section E, we provide the arguments showing that the nonlinear Power-Law CFG im- proves mean overshoot and variance shrinkage.
In Section F, we present experimental details for Gaussian mixture numerical simulations.
In Section G, provide experimental details involving real-world experiments.
In Section H. propose another non-linear CFG alternative and provide num. experi- ments.
A INTRODUCTION TO RELATED WORK: CLASSIFIER-FREE GUIDANCE (CFG) AND SPECIFICATION TIME IN THE HIGH- HIGH-DIMENSIONAL LIMIT
We start by briefly introducing the calculation required for estimating the speciation time f, for case of two equally weighted Gaussians. This section is direct adaptation of the framework introduced by Biroli et al. (2024). The diffusion process, consisting of d independent Ornstein-Uhlenbee Langevin equations, reads as follows (using ((1) = -1 and g(1) = /2 in Eq. (1)):
dā((t) = -idt + dB((),
where dt)() equals the square root of two times the standard Brownian motion in Rd At time t = 0, the process starts from the probability distribution (a(ā), consisting of two Gaussian clusters that have means im and share the same variance σ². To guarantee that these Gaussians remain distinct in high-dimensional space, we assume that |vū 2 = du² where both and jí are of order 1,
As the process evolves, the emergence of speciation resembles symmetry breaking observed during thermodynamic phase transitions. common approach to analyzing this phenomenon to construct a perturbative expansion of the free energy as function of the field. Therefore, Biroli et al. (2024) derive an expression for log (z) using perturbative expansion in terms e -f, which valid for large time values. This method is justified since speciation occurs at large times.
One can rewrite the probability to be at H at time as
1 7 de 1)2
1
P.(
(a)
exp
2
A
1 12
exp
+ g(tt)
v A
2 A
where the function g()), defined as
/ [dāP(())ex exp
1 l6e 21
E C a
( log
cxp
A
A
can be viewed through a field-theoretic (or equivalently, a probabilistic) approach, where serves generative function for connected correlations among the variables (Zinn-Justin, 2021). By
16

---

Published as a conference paper at ICLR 2026
expanding this function at large times, show:
g = A
d 2e (a.)
Σ 2 A 1,=1
1 2
(aaa;) - (a:) (aj) +0 ((x0=))))
where we utilize the brackets (.) to denote the expectation value with respect to the effective distri- bution Po Po are 2 */(2.)) Therefore, the expansion can be used to show that large times:
e (a+)
1
MMJaJ ((xe)))))))))))
log P((T) C
A
24. k,f=1
where C' is an Ø-independent term and
Mj 6ig - E- (a:a)) {a,)
The curvature of log (3)) is closely linked to the spectral properties of the matrix M. In the large time regime, . approaches the identity matrix, and consequently, all its eigenvalues are positive. However, qualitative shift in shape occurs the maximum time fs, where the largest eigenvalue of M transitions through zero. This marks the onset of the speciation time, distinguished by change in curvature of the effective potential log (z)) In this case, it can be easily computed: m,mm, and its largest eigenvalue is (
the matrix M is given by Mij (1 -²e-2() d.j
-2e-2t 2t e ). We get therefore in the large d limit l4 = 2 log (dji²) which up to subleading corrections identifies the speciation timescale
1 log(d).
f,
B THEORETICAL PROOFS: TWO EQUIDISTANT, SYMMETRIC GAUSSIAN MIXTURES
ASYMPTOTIC STOCHASTIC PROCESS IN REGIME AND SYMMETRY BREAKING
In the limit of large dimensions, comprehensive analytical examination of the dynamics in Regime I, taking place on time-scales tę + O(1) (1/2) log d + O(1), can be provided, specifically at the beginning of the backward process. Assuming collapse (for further details, refer Biroli et al. (2024)), an investigation into diffusion dynamics shows that the empirical distribution P (ø) at time t can be approximated with high accuracy by (T)) This approximation represents the convolution of the initial distribution Po. comprising mixture of Gaussians centered at tm, and a diffusion -
kernel proportional to -Z-aae ')² /2 Consequently, the explicit expression for this approximation
((F-1))/(200)) (<2++m))/(200)] and
Po(
(10)
2 @V22T²2
(s+mme - /(2r))))))
Pt(T)
d
2 V2TTT
where P, = d2e 21 + A, goes to at large times. The log of this prohability
/²2 log cosh E. m -t
log P((
2r;
r,
17

---

Published as a conference paper at ICLR 2026
and hence the score reads
x² T; T,
c
my tanh
(11)
As there are two classes: + and -m, the score conditioned to one class equals the score associated given Gaussian. Therefore, for the classes have:
+ m (z, +) -x² m.e
Tt and
(12)
x² m,e
m S -)
Tt
B.I RESULT WHAT IS THE ROLE OF CLASSIFIER GUIDANCE?
Let us first analyze the "transverse" directions J 1 m. For these directions, for all w, the score the same and equals SC (z, - T.
ब.: Let us project the backward Eq. (2) on unit vector o 1 m.
We write p = E ह, and the backward equation now reads dp = p(1 2/Tr, - + v2dB which the backward equation for single Gaussian variable. When T + ły the projection p F is thus distributed as (N, o²), for all values of w.
Therefore, as all the components except the one in the mi direction are not affected, we can consider only the component along m:
Siora (ī,
F m/
ee 177 I
C tanh
(Tr Te
T mm Z-me-t
mile 'c
ym
De
By denoting
(t), where
|78] = dd, we can obtain the backward equation:
dx (xf 25 Torr dr + (n.
where t = t, i.e., the backward time. Therefore, we can obtain for Regime and by projecting direction, we have that:
onto the m m
dq = dz'.
m
= (q (
w tanh (ge (()-t)-))) dr dn(T),
m
as, in Regime I, we have that T, > 1, and also vd = et.
Again, from this point onward by t(r) we denote the backward time for ease of notation. This is like having an effective potential:
dg (g.
dr + dn((r)
0q
where
VCR
59² +2 -(1 w)oge (t-1x) w cnsh (ge- (teil)) 1
1
222
2e 11 (1- (g) tw -age (t-ta) + In cosh (ge (((2))
Classifie''s potential
Extra potential Van
18

---

Published as a conference paper at ICLR 2026
Classifier's potential
Extra potential
Total CFG potential
7.5
Forward time
5.0
2.5
0.0
9
Figure 7: Effect CFG on the guiding potential a Gaussian mixture. The backward diffusion for the variable giving the projection of E the center n of the Gaussian where one wants to guide the backward diffusion. From left to right: Potential within the class, CFG-added-potenntee Vextra with w 2, and their sum in Eq. (6). CFG exhibits faster convergence the target 0), but results in narrower potential for small & (with / ranging from to 8, as indicated on the right panel).
Small 2
Large d 200
Small d=2
Large d=200
0.4
1=0.0 6- 0.99
1.01
15.0
2
0.0
-2.5
12.5
0.25 Sampling time
0.5
0.75
0.25 0.5
Sampling time
0.75
Figure 8: Left: CFG produces the exact target distribution in high dimensions. We simulate the backward process using a two Gaussian mixture. We project and plot the generated samples onto target mean + q(t = 0) 0 7//n7]. For small d = 2. CFG generates a distribution with larger magnitude mean (dashed line) and smaller variance than the target one (for w = 0.). This effect diminishes as the dimension increases: for 200 it is practically absent. Right: High-dimmessionaliy of the data allows CFG trajectories to align. We plot the evolution of the mean trajectories q(t): starting large forward times denoted with small
2. CFG trajectories not align with the unconditional trajectories at (data) causing the CFG overshoot. For large dimension 200, the high-dimensionality of the data allows trajectories to realign with the unguided one speciation time l., resulting in the correct target distribution.
I t=1 (noise).
Therefore, for class c +1 (cquivalently for с = -)), there is little effect for ge (t-t,) > 1,
then -qe-(t-a,) In cosh (re-((-e)) 0. Instead, for ge (1-4) O(1), have that qe (t-t,) In cosh (ge-(t)t)) 0(1). Therefore, we can conclude our first result:
Result I. In Regime before speciation time ls. CFG is effective in aiding class selection and speeds up the convergence towards the target class
The utility of CFG is therefore to "push" in the right direction in Regime where arguably the class- based score/potential is likely not accurate in the rare region (q > for c -1 and q < for +1). The behavior of the two potentials is displayed in Figure
B.2 RESULT I:: PATH ALIGNMENT
The role of CFG in Regime I is to push the trajectories more in the direction of the selected class. We recall that the SDE verified by q when pushed towards class e reads:
-q= (q q
-
(1+w)) (ty--t)))-))))
tanh (te-(()-t-))))))))))))) dd(())
dn((),
(13)
For large times but still during Regime i.e. I, - 1, negative). In this regime the CFG term can be neglected as it leads to exponentially small corrections
1/2 log d, q is very large (positive or
the SDE (of order 22e (ty---)))-)) with t, times, the SDE SDE,just reads:
>>> 1. In consequence, in Regime at large
dg --- 2c (c)-(--) dn(r),
19

---

Published as a conference paper at ICLR 2026
The effect of CFG is to lead to different values of q when entering this late regime of Regime I. We call these values ((T,) and denote т; the fixed time at which the CFG contribution can be neglected. The value ((T,) is quickly (exponentially) forgotten when 7 departs from 7₁ iec. the evolution readjust to the correct value without CFG. This can be shown by solving the SDE starting from a given TV:
q(T) = ((T))) (r- ( (c- (e + V1 101-e-120 (2((--TT))
where is Gaussian variable with mean zero and unit variance. When > but still in Regime I the solution of the SDE does not depend any longer on ((Ti) and coincides statistically with the one of the backward process of the single Gaussian corresponding to the class +1. This allows to conclude the second result:
Result II Just before speciation time t.. CFG-guided paths realign with the unguided path that generates the correct, unmodified target distribution.
B.3 RESULT III: WHEN DOES CLASSIFIER- GUIDANCE TAKE EFFECT?
We proceed to answer this question by examining the classifier-free guidance score, defined in Ho & Salimans (2022):
S Siera (7. (1 (W) (F),
(14)
where +1 and w O. By plugging in the cond. (12) and uncond. scores (11), obtain:
Stora (F,
re x² +
(1+
cm.e
m
tanh
F T-me-t
r,
ri
x²
m,e
T T-mme
cm.e
w
C
tanh
(15)
Tt
Pt
T
Now, in Regime II, when the trajectory has committed to given class, Z. m O(d) and sign
m) Therefore, tanh Z-me 0, and one finds from (15), that Store (F. ()) This implies that, within this regime, classifier-free guidance equals the conditional score. There- fore, Classifier free-gute greigguidance only affects Regime I, Store (H. (z) for t t, log(d).
This allows us to conclude the third result:
Result III. In Regime II, after speciation time f,, CFG has no effect on the generation process.
C GENERALIZATIONS OF THE PROOF
In this section, present arguments for extending proofs to more general cases. We start by discussing proof generalization for non-centered Gaussian mixtures (Section C. and then move on to mixture of four Gaussians (C.2). Finally, conclude with some remarks on how to further extend these results to more complex scenarios.
C.I GENERALIZATION TO NON-CENTERED GAUSSIAN MIXTURES
ASYMPTOTIC STOCHASTIC PROCESS IN REGIME AND SYMMETRY BREAKING
Here we provide an example on how to generalize the study Gaussian mixtures to the case where the two Gaussians centered in my and m2g We take m. My two arbitrary vectors in d dimen- sions, on the sphere | = d the case where they have different norms, both scaling proportionally to d, could be studied well with the same formalism.
The initial probability density is
20

---

Published as a conference paper at ICLR 2026
(8- (Z-mm))/(200)) + e (-((--a)2/(20)) and
Po(
(16)
(VTT2²
( (f + (a-maa--i ()
P((2)
2 ((22TT,)
where r, = + Dr goes at large times. The log of this probability
x² + log ai + a. +C,
log P((F) = 2r, where C constant, and hence the score reads
x² e mi
e T.m To.
+
m² e
3.m Te
())
(17)
Tt
B-mi B-
As there are two classes: mi and m2, the score conditioned to one class equals the score associated given Gaussian. Therefore, for the two classes we have:
-x² m m
mi : (),
and
(18)
m2 : S -) xx mbe-
WHAT IS THE ROLE CLASSIFIER-FREE GUIDANCE?
We shall use as basis the vectors m = (m1 + m2)/2, m_ = (mi m2)/2, and we shall denote by the vectors orthogonal to the place generated by m., 172.
For these "transverse" directions
(171, 12)). for all w, the score the same and equals 2-0 Let us project the backward equation on unit vector T in the transverse (FFG 0 = P.
space. We write p J, and the backward equation now reads dp p(1 2/TTr,- )d + /2dB which is the backward equation for single Gaussian variable. When T - t, the projection is thus distributed as (0, o²), for all values of w.
E-J
Therefore, as all the components except the ones in the m. and mi directions are not affected.
We now project the score on 74 and mñ. using m7 0, m 4 = de /2 and m m .m² d2 /2/
m+ (m4e-- .74
Sicro (m,c))m++
Pt
Siero (Z,e) mm (m_ -t 7) m
17
Ze-
x F--m__-
1 tanh
Tt
Tr
Inserting these scores into the backward diffusion equation, one finds that:
Z.m+/ // evolves a Gaussian variable, At time T tyy the distribution of this variable is o2).
The variable (1) = T satisfies the same equation as the variable g()) which we analyzed in the "enterred' case where mi - m
Therefore, we can conclude that in this case, CFG has the same effect: it is effective in aiding class selection, speeding up the convergence toward the correct target class
21

---

Published as a conference paper at ICLR 2026
WHEN DOES CLASSIFIER- FREE GUIDANCE TAKE EFFECT?
We can proceed to answer this question by examining the classifier-free free guidance score:
Siera (1+ - -
(19)
where e (1 and 0. The CFG score guiding class is thus:
rt x² + (1+
mie w e-t mf é F-mi T/ + m² E
mie
z TT?
St Storo (i,c)= =
(20)
r; Pt.
Tr + F. "r.
Now, in Regime II, when the trajectory has committed given class say class 1s, F T.mi- E m2 is positive and of order O(d). Therefore Sigro (ī, c) = St (τ, c). This implies that, within this regime, classifier-free guidance equals the conditional score. Therefore, Classifier free -reedgdidncce only affects Regime I, as Sioro (z. c) = (T) for > t, = 2 log(d). This allows us to conclude that Regime I1, CFG is innocuous.
Therefore all the results obtained for the centered mi m nū also hold for the more general case when the two Gaussians are centered in my and m22-
C.2 EXTENSION TO THE MIXTURE OF FOUR GAUSSIANS
Here we present the computation for mixture of four Gaussians, in order analyze the behavior of the system for an increasing number of classes and emphasize the extendability of our framework. As before, assuming no collapse, we can approximate the empirical distribution PT time by (z( with high accuracy. In this case, the approximation represents the convolution of the initial distribution Po. being mixture of Gaussians centered + s.t. (41 f and diffusion
kernel proportional (I-ae /)²/2 The explicit expression for the distribution is:
(B ((20t) te (E-(5(((t)))))((((0
Po( (VZ20²2) d
te - (E+(At (i+((- (2))² + (2 (200))
and
(2-(1 (2))-)))--)) +e (F (
4 (V2nT,))
e -2) /22r))))) - -B+((( ')² /(rr))
where I, = 02 + A, goes to at large times. This can be rewritten as:
1
df (3 +17 -2f 22)2(2r))))
e Ar- - 21/r, cosh E E. + f12) e- r2
Pi(
(v2TT1
cosh E. ( (12) 1 T₁
)
11 /
The log of this probability is:
-
Z. - + (22) -t
132 -1 T
log P.(
log -i. cosh
th Uze "2 cosh
2T,
T
22

---

Published as a conference paper at ICLR 2026
And the score reads:
(T(
e (1h + - Bie " /T+ sinh i (1h + (12) 1--1 T,
+ (Th - -e) "/T, sinh # (fī - (12) c-
Ti
+
Tt
e e-rra 21/T's cosh T (m + (2) + efriage 21/T's cosh T (ī1 (22) T.
As F approaches one of the means - - 122 the second summand reduces to (x1 + (T2) tanh x (mi + : resulting in an expression akin to the one for mixture of 2 Gaussians in (11).
C.3 CONCLUSION AND FURTHER EXTENSIONS
The results above can be generalized to any finite number of Gaussians, centered around m, where m. is vector of norm (4 v.. CFG will only have effect on space spanned by vectors m, and only regime I. One can also consider non-isotropic Gaussians. eigenvalues not scaling with d, the backward process displays the two distinct regimes I and II, which examined in detail for the mixture of two Gaussians. This result can be obtained by analyzing the forward process. The key point is that all times of order one the noised Gaussian mixture still consists in non-over non-overlapping Gaussian (regime II). On times of order one close to the speciation time 1/2 log d the Gaussians overlap and the center are of the same order of the noise (regime I). Because of the existence of these two regimes, the general arguments presented at the beginning of the paper hold and CFG does reproduce the correct distribution in the large d limit.
long as the covariance has
The results derived for the Gaussian mixture model provide foundation for broader application. We also note two key points regarding the robustness and scope of this analysis:
The core features of regimes I and persist even in more complex, high-dimensional settings, extending beyond the specific case of Gaussian mixtures. Several studies Ventura et al. (2024); Achilli et al. (2025); Bae et al. (2024) have demonstrated that these regimes apply more broadly to models where data is distributed on manifolds, among other configurations. This robustness is a direct consequence of the infinite- dimensional limit, phenomenon also observed in other domains like supervised learning.
Robustness and Generalization Beyond Gaussian Mixtures:
Scope and Future Work: The spirit of our approach is twofold: (i) To analyze CFG the tractable yet rich setting of infinite dimensions. (ii) To start with the simplest nontrivial case-a a mixture of two Gaussians to characterize the fundamental mechanisms that can guide future work.
As noted, natural and straightforward extension is to consider mixtures involving any finite num- ber of Gaussians. Further generalizations as data supported on hidden manifolds-con be pursued by following the methodologies established in studies like Bae et al. (2024) and George et al. (2025).
C.4 RELATION TO OTHER MODELS
In this section we briefly note how our theoretical results connect to a broader range of methods related to diffusion and flow matching, specifically how nonlinear CFG applies 1-step flow models (Chen et al., 2025a) and methods that learn optimal source distributions (Lee et al., 2023).
Regarding one-step models. Since the main distinction between regimes lies in whether class membership is decided, 1-step flow model effectively jumps from the initial point (part of Regime I) directly into Regime II (in fact, directly into the third regime described in Chen et al. (2025a)). However, this does not pose a problem as theoretically the difference between the conditional and unconditional score equals zero (under the correct score assumption and in sufficiently high di- mension); thus, regardless whether CFG is applied or not these models obtain the correct target distribution. In practice, we believe that the models could still benefit from non-linear guidances.
For large values of ( + T/T, utilized the log-sum-exp trick to calculate the value of the fraction.
23

---

Published as a conference paper at ICLR 2026
Regarding methods that learn optimal source distribution for flows Methods such Fas- 1ODE (Lee et al., 2023) that successfully map similar inputs to the same class indeed shorten, or eliminate Regime I-for example, if the initial distribution is Gaussian mixture (GM) where each mixture has successfully been mapped to a distinct class, e.g., by using GM-Flow matching (Lee al., 2023), Regime would he eliminated altogether. One should note that in these cases it is likely that CFG would not prove as beneficial. In fact, this is exactly the finding of the authors in Lee al. (2023), in which an alternative guidance method is proposed as CFG was not found to perform as well. However, our nonlinear guidance retains an advantage: the flexibility of > 0 vs. < O. In our experiments (starting from standard Gaussian noise with significant class overlap in the initial distribution), the former proved beneficial by amplifying guidance when score differences are large and suppressing CFG when small, would be interesting future work test whether indeed would be beneficial in settings with prominent separations in the initial distributions, as might occur in FastODE- mappings or GM-Flows (Chen et al., 2025a).
D FINITE DIMENSION
In this section, we give exact analyses describing the effect of CFG in finite- (possibly low-) di- mensional settings, outlined in Section 4.2 in the main manuscript. We start the backward equation time , large enough that the distribution of a isotropic Gaussian with variance one. The backward equation for zzt) with the CFG score reads:
dx dr
=x
1
2
+
2m
e ((y
T(ts - 7)
I[ts
} + mm(()
(((--r)
sm-me-lss
+ 2wm
1 tanh
(21)
Piz-r
where r = 0 at the beginning the backward process and T -=1p(> 1) at the end.
This can be projected the evolution of the single parameter q(T) F. m/ Vd. We obtain
dq dr
2vd ((y-r)
=9 1
+ [ 77
l'ly
) +
+ 2wv d -r)
(QViee-lrrz))
(gVde ((5-T)
} ((T).
1 1- tanh
(22)
T Peg-r
Tep-r
Considering the right-hand hand side as a force due to moving external potential - V (q. t), the effect of CFG is add an extra term which has two main effects: (1) adds a positive term the force and, in consequence, it pushes q faster away from zero, (2) it increases the value of the Hessian at any point with respect to its
-
counterpart, thus making the potential more confining.
The initial condition q(T 0) ~ (0, and
T(4 - T) 2 -2(t, -(t,--) + e -(t
(23)
CASE: w 0
The solution of the backward equation is:
((T) (0)e 2-250 ray mn mr dr" + $ 2 -ede-(tr-r) r(1 71) [ + mm(r)) e (r-r)- my dr"
dr'.
(24)
Its probability distribution must coincide with the one of the solutions of the forward equation, which reads:
((1) Vdd. + v1 20 -
24

---

Published as a conference paper at ICLR 2026
where 2,. 2 ~ ~V(00, 1) and t ty T. Let us now focus the mean of g. When consider
dv (1)-
(r-r')-257 ra dr" dr',
r(ts -!)
using that
d dr
1 T(1 -")
2 T(l - 7²)
exp -2 [- T(1 +") dr"
exp
-2
dr"
one finds that the mean of q for the evolution with 0, starting from any value ((0) at any time ly,ia is
-2 (2// S I(t, - r() dr') (5))
= =00)e r-2 rej dr'
+ Vie-lts-p
1cxxp
(25)
Using
1
1
2 + (o² 22--21
T(l - -1) dr' - log
1 (o² 12--211
we find that
((T) = a(0) e² e-2 + (a2 1)e-21 21
e 1 (a² - 1)e- 21,
1 (o² 1)e + Vd (ty-
(26)
One can check that, when (() is obtained by the equilibrium process with 0, namely g(0) /de-tt, then at all times ((T) viee-(t--
CASE: INTERRUPTED GUIDANCE
Now let us consider a protocol of interrupted guidance. We start the backward process at it >> with a CFG coefficient
O. Then at time backward time Ti (forward time t = ty T1) we switch to w = 0. At time t the mean of obtained from the backward process with w > 0 is larger than the value v dt which would be obtained with the 0 dynamics (the reason is that the extra force in (22) positive). Let us write this mean as
(( = vd e - + (q(,,,
Let us measure the backward time starting from = t1. We thus write t f1 7. We can use formula (26) with I > 11, T 7 and (()) - w(),,w). This gives for the mean value of q:
((F.w) Vdc- (11-) (q( e + (a² - 1)e
(02 1)e 21
which, translated in terms of the forward time t F. give::
((t) = vde + + 5g(11, eh 1 (0² - 1)e -21
(27)
1 (0² 1)e-
In particular at the end of the backward process, for 7 t1 we get
g²
g(t 0) (g(t,,)), t
1+ (o² - 1)e
If choose tt2 log d, and assuming that the dynamics at t has produced an average ((11) Vde ōg. we find that
0 1+ (o2 - -1)dd
q(t = 0) = vā 1+69
This shows that the guidance interrupted at t, gives a good result only in the limit 0²/d < 1. Figs. 9 and 10 illustrate the effect of the choice of 4.
25

---

Published as a conference paper at ICLR 2026
6
d'16,7'=4
w 0 and
8
4
2
0
1
2
4
Figure 9: Mean value of obtained from the backward diffusion in Gaussian mixture model with
= 16, o² = (speciation time 1, = 1.38). The CFG run with w = from = to = 41 then one switches the class guidance
O. The top curve is when CFG kept all the time (ti = 0). The bottom curve is the case without CFG (w = 0). Three values of t are studied 0.69, 1.38, 3.19 (vertical lines). The dashed curves give the mean value of q for each of these three cases. They are in perfect agreement with the theoretical prediction (27).
5000
0=15.00
O ol
dhe
3000
Figure 10: Histograms of g(( 0) obtained from the backward diffusion in Gaussian mixture model with d = 16, 16, (the speciation time 1.38), run with 200, 000 trajectories. Left: CFG with w 8 applied all times. The final distribution has larger mean and a smaller variance than the desired class distribution (full line). Next three figures: The CFG is run with w = 8 from f t1, then one switches to standard CFG w O. From left right, t 0.69, 1.38, 3.. The mean values of in the four cases are respectively 5.56, 5.51, 5.29, 4.12 and the standard deviations 1.68, 1.74, 1.87, 1.98, with targets μ = 4, a In order to minimize the bias due CFG one must interrupt it before the speciation takes place in the background diffusion, hence at t > t..
CFG CONTRIBUTION TO THE MAGNETIZATION IN REGIME
Using Equation (14), can derive the equation for the average (q(T))
d(a(T)) = (g(T))> (1- l'tt -7)
2
2/d -(ty-r)
+ I(t; - 7)
dr
+ 2wvd e ((g-T)
(ty-T) ((d)e-(trt))
qVde
(1
1- 1ttanh tanh
(28)
T(1s (1)-T)
T(ly-1)
The extra w term is strictly positive. Therefore, we have:
(a(T)) (a(T)) VT.
Moreover, using that the right-hand side is less than equal to:
26

---

Published as a conference paper at ICLR 2026
(w (1
2 I(ty )
2(1 (1++)VVd e-(1)==) r(ty- -
which corresponds to the backward equation one would obtain if ||
(1 + w)d. We then find:
(a(T)) < (a(T)) < vde (1++a) ww).
We conclude that (T())) gets an extra contribution due to CFG of the order Vde-t.
CFG indeed shifts the mean value. The amount of shift is of order dt in Regime I. However, as we shall see next the CFG has almost no effect in Regime II, so we can use the result of the previous section argue that the total shift due to CFG the one of CFG Regime I followed by switch at w = 0 in Regime II, i.e., it is of order one.
CFG CONTRIBUTION TO THE SCORE IN REGIME I VS IN REGIME II
Another interesting inequality can be derived for the difference between the CFG and the standard, non-guided score, SCFG - Sc, evaluated on trajectories corresponding to CFG:
( 1 - tanh
((1
qVde ((ye-11-7) T(t, ( 7)
Scro Sc = w Tits 7)
(29)
We use the fact that for the same thermal noise, we have qn (T) 2 (T) because the CFG force always equal larger than the O one. Therefore for given (the same) thermal history have:
qu( -15--) -(e
T(ts - 7) $ tanh (w==(()) (dee)))dddeeeeeeee (tg-T)
tanh
(30)
l'ts -T) 7)
and we can obtain:
( (d==lr)Vdd--11-) 1 1
(de--(e--) Scre Sc S I(ts- - 7)
tanh (1-tanb)
(31)
I(t)-T)
This inequality tells us, as expected, that the extra CFG contribution to the score is very small at the beginning of the backward process. Its mean increases, and is of the order of one during the backward process in Regime I. However, after the speciation time (==o(T) is Gaussian variable with mean ((t-r) much larger than the square root of the variance. Therefore, replacing the fluctuating variable by its mean we obtain
Vde-(ty-1)
( de 2(1 2(t5 T(4 - 7)
SCEG - Sc S T(l T(l--T) T)
( tanh
1 (t-tanth
(32)
In Regime II, ty - of order one, and using the asymptotic form the hyperbolic tangent one finds that
(tt--( (_2de- SCEG - So S T(4y-7)
exp -2 de-
2(t 2 T(y T)
(33)
Therefore in Regime the extra contribution to the score is exponentially small in d and its effect completely negligible with respect to the one in Regime I.
27

---

Published as a conference paper at ICLR 2026
ANALYSIS OF THE CFG EFFECT THE VARIANCE
Let us derive the equation for (q (T)) (a(T)))-
Using ltô calculus, have (multiplying by (()) in the equation for dg(r)) dr
):
dg = -2 (1- dr
I(ts 7) -()))-))) + 2Q(T)
2
I(tr -T) e --(0--))
2vd
2wvd e ((1-5)
d( ((s-))))))
+ T(4 - -7)
((T)) - ((T) tanh
T(4
+ +Tq(T)
(34)
Taking the average and subtracting 2(4(T)) dr
dia(r))
we find the equation for (T²(T)) ()(T))
(
2 2 ((
1 (1- I(ts- -7)
2
dr
((y-))
(a(T)) tanh ( (qdde- (1)-
+ w
T(1s TT)
(Tlys -
(Vde- )w )wm)
(a(T) tanh
(35)
T(ty -
At the beginning of the backward process, one can expand (anh(z) and observe that the term in the parentheses is proportional to:
-
(36)
which is negative. Therefore, we can conclude that the classifier-fres weateeer-ree-ggideee ffre--giidace--ddeee term will result shrinkage the variance.
As for the mean, the main CFG effect on the variance is produced in Regime since the CFG score term is exponentially small in Regime II.
EFFECT OF NONLINEAR CLASSIFIER- GUIDANCE IN FINITE DIMENSIONS
E. NONLINEAR CFG ALONG
Recall the useful quantities:
Z m.
m
(37)
|||
From Eqs. equation 11 11-equation 12, the score difference is aligned with m:
||
-
(c- tanh 0) m.
(38)
r.
Denote A | (c tanh 0), so that (, 3,(T) = A m. With the power-law guidance (s)s) = ws" the nonlinear CFG score reads
(F, (,,E,, + w (5, (,) (( 3,))))"
(39)
28

---

Published as a conference paper at ICLR 2026
Hence the projection onto mi becomes
(z, --= J.m
+ A
(40)
1+a
| Ie
|
c
(c taananhh 0) le tanh
(41)
r.
Pe
Therefore, the SDE for q becomes with ||ml|| = d and selecting +1:
dq dr (1 r +
2d r e (t5-T) + 2w
d= 21+0 Fee -()=-))
lta (1 (1- Lanh((a)) 14o + +](())
(42)
Again, setting recovers the earlier linear-CFG equation (Eq. 22) exactly.
NONLINEAR vs LINEAR EXTRA-DRIFT
We use the same notation before. Define
(())
(Bn)
de ((y=))
(Note that for the "+m class one has u(q) > O, hence u/ul" u ta The nonlinear extra drift (coming from the power-law CFG, for the +m class) is
d= 21+a
(-e- (--(0--r)) 1+a u(q)t_a,
(g
while the linear-CHo CFG (a = 0) extra drift is
d - (1)--)) u(q))
A Fo(())
2w 2r
Therefore look at the following identity:
Pointwise ratio (nonlinear linear):
AF
de (t
7a
B(g) = de (t)-)
u(4)
B(g)",
u(z).
Fo
r
r
=B())
Hence whether the nonlinear guidance amplifies (ratio 1) or suppresses (ratio < 1) the linear-CFG push at a given y depends only on the sign of and whether the base B(g) is greater smaller than
If B(g)= > 1:
amplifies the push, < 0 suppresses it.
. If B(g) < 1: suppresses the push, < 0 amplifies it.
Hessian / curvature effect. Recall that the drift (deterministic part) is written F(g. 7) and the potential V satisfies F
0,,V The curvature (Hessian) is @e = The nonlinear term contributes an extra piece 0, F through the d-derivative of t Since u(q) we have
d [u]))+a] = (1+ a1) vl
dq
Using u'(g) = B soch? (8g) we obtain
d dg
[u(g) ta] = - + at) 3 soch? u(g)a.
29

---

Published as a conference paper at ICLR 2026
Therefore the additional contribution to F,F coming from the nonlinear term is (up to the multi- plicative prefactors shown above)
A () -2 1 (F
(t-))) 1ta (1 + o.) 3 (Bq) u(g)",
which simplifies to
(( -2w (e
1ta
sech u(q)".
Since V2V= = -O,F. the corresponding increment in curvature is
d
1ta
( +2w (1+ 1 a
3 sech u(g)",
or, more compactly (omitting positive constants),
(02V) x a(( ( ( (--(tr-)) Ita
sech² (Bq) u(g)
Comparison with the linear (a = 0) curvature increment. The ratio of nonlinear vs linear curvature increments is essentially
A (02V)
B
(82V)
so (for the same reasons for the drift) whether the nonlinear term increases or decreases the curvature relative to a = 0 depends on the sign of a and on whether B(g) >> 1 or B(q) < 1:
* If B(g) > 1: (gg) < 1: Q reduces the curvature increase, a amplifies it.
amplifies the curvature increase, a 0 reduces it.
SELECTING THE NONLINEAR CFG SCHEME
Consider the projected SDE for the component q q= --zmmmm mm/Vd under the nonlinear power-law CFG scheme equation 42. We claim that, given a fixed dimension d, mean vector m7, variance a², and a linear CFG scheme with weight w, one can always construct a simple nonlinear alternative with E {a', 0}. where a' > -1. We first develop the argument for a' > 0. The reasoning extends directly to -1 a' < 0, with the roles of the cases ((g) < and B(q) > reversed. The essential conclusion is that, in very low dimensions, effective of nonlinear CFG requires avoiding an overly strong push, even the correct direction, since the system is unstable and highly sensitive. For case a' 0 our claims are as follows:
1. The nonlinear term either suppresses the mean push along the conditioning direction (when the score difference is small, B(g) 1) preserves (when the score difference is large,
(g))² > 1).
2. The confinement of the effective potential is either reduced (when the score difference small, Bgg) < 1) or kept the same relative to the linear CFG case (when the score difference is large, B(g)² 1).
Recall that
| B(g)=
F. me
P. (1 tanh θ).
which quantifies the
score difference projected along m.
In small dimensions, the generative dynamics of diffusion or flow-matching are highly sensitive to large guidance weights when 0. In this regime, the system is not "self-correcingg" via high-dimensional averaging, so a large push along mi can cause overshooting or instability.
When B(q)= > 1, the score difference AS = S, c) S((i)) is already large, meaning that the preserves the linear push exactly, ensuring
conditional signal is strong. In this case, setting that the system is guided correctly without amplification. Choosing a > here would amplify the already strong signal, risking overshoot.
30

---

Published as a conference paper at ICLR 2026
When (g)) 1, the score difference is small, and the conditional signal is weak. Setting a' O dampens the nonlinear CFG term, preventing small and unreliable differences from producing disproportionate guidance. The nonlinear term is therefore suppressed where it is least reliable, and preserved where the signal is sufficiently strong but not too large.
Thus, by selecting a = aa for B(q) < and a = otherwise, the dynamics retain a stable, interpretable push along m, while ensuring that curvature is never increased beyond that of the linear CFG case. In low dimensions, this prevents both over-confinement of the potential and excessive push, providing a natural safeguard against instability.
Existence of regimes with B(q)= < 1.
The base factor introduced above was
( de (ty-r)
u(g),
( = (ag)
r Pey-r-
Two simple observations guarantee that the nonlinear scheme will sometimes suppress the linear- CFG effects (i.e. produce (4) 1) and therefore provide benefit relative standard CFG.
First, in the long-time limit / 1=
have e- 0 and r 1, hence
lim B(g) = 0 1 1-00
for every fixed Thus there always exist times during the backward process where B(g) < 1 and the nonlinear scheme suppresses the linear-CFG push and curvature increase.
Second, at the other endpoint t 0 (the earliest time in the backward process) one has r(0) = (o² 1)e-0-2 -02 o², so
2d
B(a)
u(
1=0 σ²
σ²
Consequently, in very small dimension, if the problem parameters satisfy the condition
2d
B(
< for all q,
then B(q) 1 already at 0 (and hence a neighbourhood of = 0). Even when 2d/o² 2 2 the inequality B(q) < 1 may still hold for most if (()) is typically small (i.e. when tanh(3g) 1). Combining these remarks conclude:
There always exist times (in particular sufficiently large for which B(q) < 1, so the nonlinear scheme will suppress linear- effects at those times.
For small d or sufficiently large noise variance o² (more precisely when 2d/o² 2 < 1), one also has Bgq)= < 1 near = 0 for all g, the nonlinear scheme suppresses CFG uniformly at early times as well.
In practice, therefore, the nonlinear power-law CFG will often reduce the unwanted CFG side-effects (mean overshoot and excessive confinement) across substantial portions of the trajectory; this the regime exploited in our experiments.
This means that the proposed simple nonlinear version of the CFG will always be more beneficial than standard CFG by reducing its unwanted effects. now proceed to argue about the frequency of the event B(q)= > 1.
In practice, we found that setting > 0 throughout generation performed well, without explicitly disabling nonlinear guidance when B(q) > For simplicity, we adopted this approach across all experiments. In Fig. 11, we show specifically that for wide range of data parameters, numer- ical simulations indicate that the efent B(q) 1 happens very rarely. Nonetheless, we anticipate that adaptively switching may provide further improvements in very low-dimensional settings, with minor gains in higher dimensions.
F EXPERIMENTAL DETAILS: GAUSSIAN MIXTURES
In this section, we present experimental details for the numerical simulations involving Gaussian mixtures, describing the procedures and the hyperparameter configurations.
31

---

Published as a conference paper at ICLR 2026
om0.0. ded
am0.0, de20
000.0, de
20
Denoky
FATT Mesurnntoo
2849/0000009
274380000000
2878/8000800
am0.25, cm3
am0.5, d-20
am0.25 d-50
22-
DenoRe
155
360000
5240/8008008
4761/8000800
0m0.5, de3
gm0.S, dl=2
0m0.5, d=50
22
1.5
5864N0000000
6010
6262/8000100
0m0.75..00 6m3
a-0.75, d=20
c-0.75, d=50
22
6642/0090000
787608008006
0134/8000100
0.0.9.de d-3
2- 9, d-20
0-0.9, d-50
22.
beadity
02-
01490000000
#012/8008006
9130/8000000
1's
Rigl
Biol
Big)
Figure 11: Histograms of the event B(q) for different values of (0.0, 0.25, 0.5, 0.75,,
and dimensions d = 3, 20, 50, with w fixed at 2. The red line marks the cutoff B(q) = 1. Across all settings, the event B(q) is seen to be rare, confirming that the condition under which would be set to zero occurs infrequently in practice.
Numerical simulations. In the case of mixture of two Gaussian clusters centered on t.m Rd with variance o², the score function reads as
x(1) r.
me-t
A(l) me
cm
Stera (z()).
c
tanh
I
P.
where I; A, + 02 with A, 21
We can then discretize the stochastic differential
equation associated to the backward process as
Z(t + 1) = at) + [z()) + 2S.cro (f(t), + /
where 77~ ~ (0 (), 1). with t, = 8 the time horizon and ty /L = 0.01. We use m = [1...., 1,,1]] o² 1, and each point is obtained by averaging over 100 initial conditions. The speciation time t, is calculated as ta 1/2 log d. Throughout the experiments, we plot the evolution of g(t) conditioning the guidance the positive class with 1.
32

---

Published as a conference paper at ICLR 2026
G EXPERIMENTAL DETAILS: REAL-WORLD ANALYSES
G.1 ASSETS
In Table we list the datasets and models used in our work along with their licensing.
Table 2: Assets used for our work.
Name
License/Link
COC0'14
https:/ https://tow... http://.coodaaaaeee....
ImageNet
https:/ /twwe www.sniiage-eeeeeeee .org
CC12M
https://gtthhn www..cooooocoooo..ccooooooooottt goog) wwwrerrrr--aaaaeeeeeeee
htt.aarrcc-ctttasssscccooppppppp -11m
YFCC100M
h2 htt/://wwwmlllll/ www.:aaulliiieellcoommooeee.eee
Florence-2
httppss//mugggtgaace..om crosoft http::/www..........ooooooooo//.
EDM2
httpa://gt h com http:///ww..aaaaa/////////////aa
MMDIT
https: htgps://ithhl pom/ luciiraaingg wwwiiiiaanaaaaa hww.inn//mmmtttmmmmmmmmmm htto://tteiinnnnmmmmoo/////////a
MDTv2
hatps:/ htub. con/ sail httpa///////1ao///aaa////////330
G.2 PERFORMING THE TIME REPARAMETERIZATION
In the second part of the paper, we evaluate the score of DiT models, in discrete time, as intro- duced by Peebles & Xie (2023). In this context, the forward process has linear variance schedule { where L is the time horizon given as a number of steps. Here, the variance evolves lin- early from 31 = 10-4 to B1000 2 x 10-2, 2 An unguided sample, timestep l', denoted 7 (4') can be expressed readily from its initial state, F(0) = a. as
t (t') - = t a a (t))
where ā (t') IT-_ (1 P.) and & is standard Gaussian noise. This equation corresponds to the discretization of the Cunteen-Uhleenbeck Eq. (9) under the following timestep /' reparameterization,
- log (a (()),
where time & is defined in the main manuscript. This gives the map between our theoretical timescale used in Gaussian mixtures, and the one used in real-world settings. We note that, as the neural network predicts the noise, in order to calculate the score, one needs normalize the output by the standard deviation (depending on the variance schedule). In this case, this corresponds to dividing the neural network output by (t') = V1 1-t). In numerical experiments, we divide the CFG-added-term by (t') + 1 to avoid numerical errors. This is theoretically justified due to the fact that, as discussed in main paper, the score difference SSt (T. c) Sy for large forward times decays exponentially (as e-t to zero.
For completeness, present the full comparison of numerical simulations to real-world using the time-reparameteriiztioon to plot the timesteps on the same time-scale. Our findings are por- trayed in Figure 12. As each framework uses a separate time reparameterization, the x-axis needs to be recalculated accordingly. For the EDM2 framework (Karras et al., 2022), this can be done as follows: given noise schedule (tt), the reparameterizzation can be calculated as t'(t) (1/2) log( 1 (t)), assuming that s(t) 1. For the case s(t), one needs to resort to equation Eq. (2).
G.2.1 APPLYING THE FORMULA TO VELOCITY FLOW- WWW-MATCIING.
For velocity-based flow models (Achilli al., 2025) it follows that: (,(xly) = (x(y)) + wxun(xly) ux(x)) and from their Lemma 1, ue (x(l)) + b, V log px(ly). This implies that lu (xly) - u, (x) = b9 |V log Pe 8) V log p: (x)
the case of straight paths, where (as used in experiments), by (2=1)a_ Therefore, Power-Law CFG
ar tand σe with (s) = w does indeed correspond to scaling the velocity difference according to the afore- mentioned equations. However, since the score difference &., V log (x(l))) V log (x:)
33

---

Published as a conference paper at ICLR 2026
decays exponentially, both approaches simply exponentiating d. and scaling the velocity differ- ence to determine the score- satisfy the nonlinear guidance condition limg-s soi(s) = O. Directly exponentiating '" corresponds to the choice (s) wbf s In our experiments, to stay consistent with the definition of Power-Law Law CFG where de (s) = ws we scaled the velocity difference ac- cording formulas above. We also examined directly exponentiating v"; although it did improve over standard CFG, it underperformed compared to Power-Law CFG.
G.3 HYPERPARAMETER CONFIGURATIONS
Here, we give exact hyperparameters used for reproducing all our experiments. The real-world experiments are performed using NVIDIA H100 Tensor Core 80GB HBM3. The EDM2-S model has model size of 280 Mparams and 102 Gflops, whereas the DiT-XL/2 model has model size of 675 Mparams and 525 Gflops. Parameter or is tuned in (0.3, 0.95) with an increment of 0.05 and parameter w is tuned in (1., 12.) with an increment of 0.05. To tune w, we first perform a small grid search of the increment of 1. and then do further extensive search of the best performing Wprelim in the range (Wprelim 2., Wpretim 2.) with the 0.05 increment. We begin with the hyerparameters used in our figures.
In Figure 1 we plot the generation of images starting from 7 initial seeds for the DiT/XL-2 model trained on ImageNet-IK (256 x 256) for (1) conditional model without using guidance, (2) standard CFG with w 4., and Power-Law CFG with
0.9, w = 8.
In Figure 2, the plots correspond to the histograms of the samples generated using the backward process with dimensions d E {200, 5, 5) and guidance parameter w E {0, 0.2, 15}, with o² 1, averaged over 10, 000 trajectories. The non-linear parameter has a 10.
In Figure 4, we use 20-dimensional Gaussian simulation with o² and vary the nonlinear param- eter as shown in the legend. For the real-world experiments we use the Class-conditional EDM2-S trained on ImageNet- IK 256x256 and an in-house cext-to-image model with MMDiT architecture.
In Figure 5, we perform sensitivity analysis for EDM2-S trained on ImageNet- IK (512 512), taking a from O. to 0.99 with 20 evenly spaced values, and w from 1. to 10. with 20 evenly spaced values as well. The right plot involves a values of 0.2, 0.4, 0.6, 0.8, 0.9 with w in the range of 1. to 12.5 with evenly spaced 20 points.
In Figure 6 we show generations of DiT/XL-22 trained on ImageNet- 1K (256 x 256). The red panel contains generations from weak and strong standard CFG (corresponding to w 2. and 5. respectively). The green panel corresponds to power-law CFG (a 0.9) with weak and strong guidance (corresponding to w = 2. and w = 10.). The blue panel corresponds to combinations of a and w (0, 2.5). (0.25, 4.), (0.5,6, and (0.9, 8.).
In Figure 3, plot the evolution of the 1D backward dynamics with means at 44 and unit variance. The potential plotted corresponds to equation (g, = _aa² - 2 log cosh (t-)().. For the derivation of this potential, see Appendix B.2 in Biroli et al. (2024).
In Figure 7, we examine the following functions:
Vcless (q, t; c) 79²
1
(t- -t- 2
Vexra (q, 1;C) -ce -(-Eaa + log cosh ( (te)))))))) log(2),
where the plots correspond to Volass Vear and (Velass + WVost) with w respectively. We select c = 1, and fix the speciation time to t, = .5. The additive constants are added for clarity only.
In Figure 8, the first two plots correspond to the histograms of the samples generated using the backward process with dimensions d 200) and guidance parameter w E {0, 0.2, 15). with x² = 1, averaged over 10, 000 trajectories. The last two plots correspond to the actual trajectories projected onto the target mean +M for values of E (0., 5., 10., 15., 20.).
*We expect that the images of bees on yellow flowers correspond class distribution. A similar trend appears for jellyfish, where stronger guidance produces images with more extensive blue backgrounds (Figure 6, left panel). This likely reflects stronger guidance pushing samples toward higher-likelihong regions-a connection also noted in the Autoguidance paper [(Karras et al., 2024a), Sec. 3), linking optimal score matching MI. estimation.
high-likelihen mode of the bee
34

---

Published as a conference paper at ICLR 2026
Gouss. Mitures
DTT. Inolinoggttttt 1k (512/512)
Score ovelut on
decco desom
500 des
MMDIT-Y MMDITUM
32-
2.0-
50
4.1-
1.5
cl
15
1.0
3.
3.0
0
Forward time
Forword tIMEC
Forward
Figure 12: Evolution of the score differences for numerical simulations and real-world exper- iments projected onto the same time-scale for direct comparison. First: Numerically simulating mixture of two, four, and eight Gaussians with equidistant means on a sphere (r = vd), with vary- ing dimension d, with w 4, x² = 1, averaged over 10,000 trajectories. As d increases, the score difference starts increase an earlier backward time т. Additionally, as the number of classes increases, the magnitude of the score difference grows, as well as the duration of large difference between the scores. Second: Three DiT/XL-2 models trained on ImageNet- IK using 2, 500, and 1000 classes (image size 512 x 512). We observe a similar pattem: d increases, the score dif- ference becomes larger at an earlier time. Furthermore, as the number of classes increases, the magnitude of the score difference increases, together with the duration for which the difference mains large. Third: evolution of the remaining models used in our experiments (EDM2-S, MMDiT and MDTv2). We observe similar behavior to theory and the DiT/XI--2 models.
Table 3: Hyperparameter configurations used throughout the experiments.
(n,
DIT/XL-2
EDM2-S
Dift. MMDIT CC12m
DifT. MDDVZ IMN-IK
FM MMDDTCCOM
FM MMDIT OCT2M
Sundard
[0., 1.5)
1.4)
(.55)
[0., (.2)
(..
[0., 2.1)
Non-lincar
(0.8. 4.95) (.855.1855
(0.85, 11.4)
70.6.7.0)
(0.8.
(0.7. 10.15)
(..6.
Non-lin. Limited
(0.9. 12.05)
(0.55, 8.25)
(0.85, 8.25)
(0:75, 10.05)
(0.65, 7.85)
Non-lin. + CADS
(0.7.4.75) 4.75)
(0.80, 11.75)
(0.75,8.15)
(0.80, 8.40)
(0.75, 10.75)
(0.55.
In Figure 9, we plot the backward diffusion in Gaussian mixture model with d = 16 4, 8. The CFG either run all times (top curve), stopped at times ( or not used at all (bottom curve).
In Figure 10, we perform linear CFG with w from & to t f,, after which we CFG off (w = 0) at times t 0.69, 1.38, 3.19.
In Figure 12, we use DiT/XL-2 model trained on 2, 500 and 1000 classes. For 2 classes, we have selected the same classes as Biroli et al. (2024), and for the 500 classes we selected the first 500 classes in ImageNet-II. The x-axis represents the Forward time t, where the parameterization is obtained as explained in Section G.2.
In Figures 33-34, we perform the same experimenet as Figure and use d 16 and o²
Finally, in the first column of Figure 35, we plot the estimated Lansen-Shannor Divergence between the target samples corresponding to randomly selected class and the diffusion particles throughout the backward trajectory. Note that this is performed in latent space. For obtaining the middle column, we first take all data samples from one class, embed them into the latent space and calculate the centroid corresponding to this class. Then, we normalize the centroid (making it unit norm) and plot the dot product of the particles throughout the backward diffusion process with the calculated centroid. The right column corresponds to the score difference. Across all experiments, we selected 4, sampled using DDPM (Ho et al., 2020) using 250 sampling steps, averaged over 25 samples. All other hyperparameter configurations are set to the default.
Table 3 displays the hyperparameters used to obtain the results given in Table 1. The evaluation code relied on EvaIGIM library by Hall al. (2024).
G.4 FURTHER RESULTS
Here, we detail the remaining experiments conducted. We provide the following:
Diversity and coverage metries corresponding experiments in Table (see Table
35

---

Published as a conference paper at ICLR 2026
Num. Simulation:: Gaussian Maxturies
Roal-word: Class, Cond. Models
Real-world: world: : T2M Models
Real
0.0
Late
d=
d=
0.5
ED12-5
DRTAL2
1.0
0.8
HDT-2-d1
MMDDi-dif
0.8
20
0.6
50
202
0.4
02
0.2
0.:
0.2
0.5 1.0
0.2
2.0
Figure 13: Evolution of the CFG score difference, from noise (t 1) to data (t = 0). Left (stand. CFG): Numerically simulating mixture two Gaussians: as increases, the score difference becomes substantial earlier (this happens during Regime I). Middle and Right (stand. CFG): Real-world experiments using advanced models show consistent behavior with theory: monotonically increasing score difference followed by decay after certain point.
Ablation studies showing that Power-Law CFG outperforms standard linear CFG when changing the number steps (see Tables 5-10)
Sensitivity analysis showing the FID benefit for increasing value of a (see Section G.4.1, Figures 14-16)
Further qualitative analyses of power-law CFG for either fixed w and varying a or varying w and varying a (see Sections G.8.1 and G.8.2)
Further generation examples of DiT/XL--2 and MMDiT diffusion model (see Sections G.8.3 and G.8.4)
Diversity and coverage metrics. Table present additional quantitative evaluations of our method, focusing diversity and coverage metrics (as described in Hall et al. (2024)), which com- plement the results shown Table Our analysis compares power-law CFG to standard CFG and state-of-the-art guidance methods, including combinations with CADS (Sadat al., 2023) and limited-guiie -rnited-guidance (Kynkäännieni et al., 2024), which proved to be the most competitive approaches. As demonstrated in the main manuscript, power-law CFG generally outperforms standard CFG (in- dicated by arrows in the table). Moreover, when combined with CADS and fimited-interee guid- ance, it yields improved results over existing methods many cases.
Ablation studies. In Tables 5 through 10, we conduct ablation studies on two class-conditional and four -i--to-imaee models, demonstrating that non-linear power-law CFG consistently surpasses standard CFG across varying sampling steps. The results show improved FID performance and en- hanced outcomes across multiple metrics when using the non-linear approach compared standard CFG.
Sensitivity analysis. In Section G.4.1, we present additional sensitivity analyses that build on Section 5 and Figure 5, demonstrating that high values of consistently enhance performance, im- proving robustness and stability during tuning. As noted in the main manuscript, while power-law CFG introduces an additional hyperparameter, a, extensive hyperparameter tuning was unnecessary, with large values like
0.9 consistently performing well. This is evidenced in Section G.4.1, Figures 14 to 16, which show that higher a values reliably improve FID scores. Class-cor condiiinniiional models (Figure 14) exhibit greater benefits than text-to-image models (Figures 15 and 16), though both show improved performance with Power-Lav Law CFG compared to standard CFG.
Further qualitative analyses. In Sections G.8.1 and G.8.2, we provide additional qualitative ex- amples for DIT-XL/2. Specifically, we conduct two studies: one varying the guidance parameter w with a fixed and another varying with a fixed w. When a is fixed, increasing w can lead to issues such as complete mode collapse (e.g., for the jellyfish class), oversaturation (e.g., for the bee class), or significant loss of diversity (e.g., for the dung beetle class), which are common artifacts of standard classifier-fre guidance. These effects are mitigated when using a non-linear pawer-law guidance approach. The second study explores the impact of increasing while keeping stant, demonstrating enhanced diversity as a strength increases. In Figure 13, we plot how the score difference evolves for other CC and T2IM models used in our analyses.
36

---

Published as a conference paper at ICLR 2026
Further generation examples. In Sections G.8.3 and G.8.4, we present additional generation examples for class-conditional (DiT/XL-2) and text-to-image (MMDiT) models, demonstrating how power-law CFG enhances image details, thereby improving image quality and fidelity for individual images, and increases diversity when examining a set of images for specific class.
Table 4: Comparison EDM2-S on ImageNet- KK 512x512 data, Diffusion trained text-to-image MMDiT CC1 2m data, and Flow-matching trained text-to-image MMDiT COCO data. Bolded are the best results and underlined are the second best.
Mivell
EDME- (CC, IM-IK
DIT/XI-2 IM-IK
(T2M.
(T2IM,
Density
Coverage
Demelty
Density
Cuverage
0.764
Satediler (Wang et 2024)
07700
Limited (Kynkannnemmi
2004)
0.777
0
Cosine (Gao
2023)
0.50
0.769
1,102
0.822
1.106
0.340
0.920
0,802
CADS (Sufat et el. 2025)
0354
0766
0.99
0.853
1.222
0.360
0.921
0.777
APG (Sadat et al, 2024)
0.815
0.760
1,033
0.367
1995
0.858
0.915
0.797
REG (Nio al., 2024)
0.850
0.771
1.122
0.833
1901
0.855
0
0.783
CPG++ (Chung al. 2024)
NIA
Power-law CPG (Uurs)
0,845
0.986
0.844
0,850
0.78417
Powe-low CRG Limited (Oars)
1.11(
0.920
(Oum)
(1)
(1)
Table 5: Ablation study: Changing the number of sampling steps for Class-conditional: DiT
ImageNet- IK 256x256
Version
Num. steps
FID (L)
IS (t)
Precision (1)
Recall (t)
sFID (4)
50
0
1.5
3.33
259.88
0.8163
0.5474
7.406
100
2.64
233.72
0.8027
0.5831
5.720
Stand. CFG
2.38
233.52
0,8032
0.5936
5.462
200
0
1.35
2.29
234.92
0.8031
0.5950
5.331
250
0
1.5
2.27
278.30
0,8291
0.5840
4.601
50
0.6
4.35
3.03
284.55
0,8215
0.5757
7.110
100
0.6
3.4
2.32
274.36
0.8199
0.6012
5,432
Non-lin. CFG
150
3.4
2.19
274.39
0,8202
0.6071
5.512
200
0.75
4.8
2.17
276.98
0,8204
0.5956
5.567
250
0.75
4.85
2.05
279.90
0,8310
0.5950
4.670
Table 6: Ablation study: Changing the number of sampling steps for Class-conditional: EDM2-S
ImageNet-IK 512x512
Version
Num. Steps
Q
w
FID (L)
w
pino (4)
8
0
1.95
4.78
0
2.3
103.33
16
0
1.50
2.52
2.3
57.47
Stand. CFG
32
0
1.40
2.29
2.3
54.78
64
0
1.50
2.25
2.15
54.39
0.05
2.30
4.74
-0.25
1.5
100.81
16
0.25
2.30
2.32
-0.05
2.15
56.92
Non-lin. CFG
32
0.85
11.40
1.93
0.35
2.5
52.77
64
0.85
11.30
1.89
0.35
2.1
52.56
37

---

Published as a conference paper at ICLR 2026
Table 7:
Ablation study: Changing the number of sampling steps for Diffusion text-to-image:
MMDiT CC1
Version
Num. Steps
FID
Clip score (1)
Coverage (t)
Density (f)
Precision
Recall (f)
0
1.75
8.98
581
0.8392
1.104
0.6623
0.5545
Stand. CFG
0
1.75
8.79
22.532
0.8450
1.124
0.6717
0.5590
50
0
1.55
8.58
22. 22.111
0.8401
1.109
0.6612
0,5692
22
1.117
0.6765
20
0.25
3.05
8.94
22.773
0.8424
1.114
0.6619
0,5495
22 22.590
Non-lin, CFG
50
0.60
7.0
8.11
22.415
0.8503
1.128
0.6703
0.5532
100
0.75
8.02
22.563
0.8472
1.115
0.6747
0,5723
Table 8: Ablation study: Changing the number of sampling steps for Diffusson text-to-image:
MDTv2 ImageNet-IK: 512x512
Version
Num. Steps
w
(J)
Clip score (1)
Coverage (f)
Density (t)
Precision (1)
Recall (t)
1.55
5.30
23,949
0,8218
1.167
0.7475
0.5133
30
1.55
4.09
23,998
0,8292
1.233
0.7492
0.5264
Stand. CFG
24.011
0.5294
50
1.2
3.68
24.306
0,8318
1.150
0.7510
0.5989
0.6
6.0
4.88
24.154
0,8251
1.236
0.7503
0.4916
6.0
4.03
24.033
0.8344
1,205
0.7583
0.5332
Non-lin CFG
40
0.7
7.0
3.73
23.367
0,8353
1.181
0.7557
0.5546
25.339
0.8361
1.170
0,7513
0.5609
Table 9: Ablation study: Changing the number of sampling steps for Flow-Matching text-to-image: MMDiT on COCO
Version
Num, Steps
w
FID (1)
Clip score (1)
Coverage (1)
Density (f)
Precision (t)
Recall (t)
2,85
26.373
0.7529
0.6121
0.5604
Stand. CFG
30
0
1,95
5.84
25.948
0.7581
0.8668
0.6051
0,5879
40
2.05
5.62
25.817
0.7651
0.8798
0.6091
0.5978
50
0
2.00
5.20
25.714
0.7726
0.9026
0.6299
0.5940
20
0.5
9.75
6.47
25.981
0.7241
0.7762
0.5719
0.5851
0.5
5.62
0.7577
0.8457
0.6105
0.5874
Non-lin, CFG
40
0.6
9.05
5.45
25.113
0.7633
0.8549
0.6149
0.6030
10.15
0.6191
Table 10: Ablation study:
Changing the number of sampling steps for Flow-Matching text-to-
image: MMDiT on CC12m
Version
Steps
(J)
Clip score (1)
Coverage (f)
Density (t)
Precision (1)
Recall (t)
20
2.75
10.75
25.224
0.8289
1.069
0.6396
0.5803
30
1.95
9.85
24.935
0.8318
1.068
0.6946
0.6000
Stand. CFG
0.8461
1.103
0.7064
0.5907
50
2.1
9.46
25.133
0,8520
1.145
0.7159
0.5946
0.2
3,25
10.68
25.585
0.8301
1.075
0.7101
0.5815
Non-lin.
10.0
9.81
25.002
0.8338
1.085
0.6968
0.5909
40
0.6
9.35
9.17
24.794
0,8352
1.087
0.6909
0.6030
8.05
24.723
0.8397
1.087
0.6911
0.6023
G.5 LOW-DIMENSIONAL EXPERIMENTS
In the following experiments, we have trained a U-ViT-like model (Bao et al., 2022) on lower- dimensional datasets to confirm that nonlinear CFG provides further improvementts to standard CFG in lower-dimensional settings. For CIFARIO the architecutre closely mimics that of U-ViT- S/2 and for ImageNe164x64 and Imagenet32x32 U-ViT-M/4. We have obtained ImageNet32x32 by using bilinear downsampling on ImageNet64x64.
Regarding sampling, for CIFARIO we used satteemmrryaaaaae sde' sampler with maximum of 1000 steps (we did not find further improvement by further increasing step size). For ImageNet32x32 and ImageNet64x64 we used 'pm solver' with a maximum number of sampling steps equal to 100 (we did not find improvements by further increasing it from 100).
38

---

Published as a conference paper at ICLR 2026
G.4.1
SENSITIVITY ANALYSIS
EDMZ ImageNet 1000 512x512
DIT Imagelee-11 512x512
12
32
1.0
FD
24
9.0
0.
52
0.4
0.5
5.8
1.0
-32
0.0
0.2
0.4
0.6
0.0
Figure 14: Class-conditional diffusion: image quality benefits from non-linear scheme, yielding lower FID for larger values of a.
MMDIT CC12m
MDTv2 ImageNet 1K 512x512
5,40
8.15
8.30
13
17.5
12
0.00
9.25
0.5
1000
4.5
3.0
15.0
3.4
30
6.0
5.0
0.0
0.4
0.6
0.8
α
Figure 15: Text-to-image diffusion models: image quality benefits from non-linear scheme, yield- ing lower FID for larger values of a.
MMDIT-FM CC12m
MMDIT-FM COCO
8.5
11.0
8.0
30.5
5.0
7.0
5.0
08
7.5
0.0
02
0.4
0.6
0.8
7.5
10.0
10.0
6.5
10.0
9.5
5.0
9.0
0.2
0.0
Figure 16: Text-to-image flow matching: image quality benefits from non-linear scheme, yielding
lower FID for larger values of a.
The results from the first experiment, showing the lower-dimensional improvements in FID can be seen in Tables 10-12, whereas the experiment of choosing very small guidance scale be seen in Table 13. The findings are as follows: as the dimension decreases and the unwanted effects of CFG get stronger, the benefit from applying non-linear CFG further increases, be seen from
39

---

Published as a conference paper at ICLR 2026
Table 11: Comparison of CFG variants on ImageNet-1K (pixel space) for Matryoshka Diffusion and
Pixelflow Flow Matching. Sccond row represents the performance with fixed a = 0.9, last row with tuned a.
Matryoshka Diffusion (Gu al., 2023)
Pixelflow Flow Matching (Chen 2025b)
Version
FID 1
Prec 1
Rec
FID
Prec
Rec
Standard CFG
3,51
0.81
0.54
2.43
0.83
0.56
Nonlinear CFG (a=00.9
3.39
0.82
0.55
0.83
Nonlinear CFG (a==0.6500.45
3.17
0.83
0.56
2.12
0.85
0.59
more pronounced improvemenet in FID. Secondly, for very small w. the performance of linear and non-linear CFG equals that of the conditional, unguided model.
G.6 NUMERICAL SIMULATIONS: MIXTURES OF GAUSSIANS WITH DIFFERENT MEANS, VARIANCES AND MIXTURE PROBABILITIES
In this section, we present experimental results comparing standard Classifier-Free Guidance (CFG) and Nonlinear CFG on Gaussian mixtures with 3 and 4 components.
Experiment 1: 3-Component Gaussian Mixture (d = 2) We first investigate two-dimensional (d 2) setting using a Gaussian mixture model composed components with assignment prob- abilities of 30%, 40%, and 30%.
The results align with two-mixture Gaussian analysis:
Standard CFG, especially with strong guidance parameters, can induce unwanted effects (as shown in Figure 17).
By selecting Nonlinear Guidance with a of 0, these negative effects can be dampened (as shown in Figure 18).
Experiment 2: 4-Component Gaussian Mixture (d 2 d 1000) The second experi- ment examines the improvements offered by Nonlinear Guidance over standard CFG, as well as the mitigating effect of high dimensionality on CFG's unwanted behaviors. We use 4-component Gaussian mixture for small-dimensional experiment (d 2) and a large-dimensional experiment (d 1000).
The findings are consistent with the Gaussuass component analysis (and can be seen Figures 19 and 20:
1. Nonlinear CFG effectively dampens the effects of mean overshoot and variance shrink- age induced by standard CFG.
2. As the dimensionality (d) increases, the mean overshoot and variance shrinkage effects associated with standard CFG diminish.
G.7 RESULTS FOR PIXEL-SPACE VS LATENT SPACE
Here, we provide the table for the experiment conducted in Section 5.3. We show that the benefit of power-law CFG also be observed when applying non-linear guidance in pixel space, and not just latent space.
G.8 ROBUSTNESS ANALYSIS: RESOLUTION, SCHEDULING, AND ALTERNATIVE METRICS
In this section, we discuss the robustness of the Power-Law formulation across different vector norms (resolution), noise schedules, and distance metrics.
Resolution and Vector Norms. In high-dimensional spaces, the vector norm of the score difference sensitive to the dimensionality of the data (e.g., a 512" pixel-space image vs. 322 latent). While the intrinsic manifold dimensionality may remain similar, the Euclidean norm scales with resolution. Theoretically, the overall shape of the score difference curve remains consistent (hump- shaped behavior tending to zero t - and - 1). but the absolute scale changes. In the context of Power-Law CFG, this resolution- based scaling is effectively managed by the renormalization provided by the hyperparameters; Specifically, choosing the optimal guidance scale renormalizes
40

---

Published as a conference paper at ICLR 2026
Table 12: Comparison of Standard CFG, Cosine-based variants, and Power-Law (Nonlinear) CFG
on EDM-2/S ImageNet-IK, IK (512 x 512).
Version
FID
FID DINO 4
Standard CFG
2.29
54.76
Cosine CFG
2.26
54.28
Cosine CFG w. a
2.21
54.09
Nonlinear CFG (Ours)
1.93
52.77
the score difference for the given resolution regime. We validated this by evaluating Power-Law CFG on pixel-space methods Matryoshka-oiffusiii (Gu al., 2023) and PixelFlow (Chen al., 2025b)), confirming that the method generalizes robustly to high-resolution pixel space without requiring explicit resolution-baase formulation changes.
Noise Schedule Scaling. The magnitude of the score difference is naturally influenced by the dif- fusion time (and consequently o,). While one could theoretically define schedule-specifee non- linear guidance
form (x, = 5 (ax) to decouple these effects, we find that the standard Power-Law formulation is sufficient to improve upon standard CFG. The exponential decay of the score difference at boundary times theoretically mitigates the need for complex, schedule-dependent scaling functions f(a,) in high dimensions, but suspect further empirical improvements might be observed by tuning for optimal f(a.-
Euclidean Difference vs. Cosine Distance. Finally, we investigated whether the performance gain of Power-Law CFG stems purely from directional alignment if the magnitude of the score difference is essential. We compared our method against "Cosine CFG," where the guidance is scaled based on the cosine distance (purely directional), and "Cosine CFG w. a," where the cosine distance is raised to a power or. We evaluated these variants using EDM-2/S on ImageNet (512 x 512 latent space). As shown in Table 12, while using Cosine distance improves over standard CFG, it does not match the performance of Power-Law CFG. This suggests that the "built-in" scaling provided by the Fuclidean norm-voer norm-which accounts for both the angular difference and the relative magnitude of the conditional and unconditional scores is critical component of effective non- linear guidance.
41

---

Published as a conference paper at ICLR 2026
Version
Num, steps
a
FID (4)
Precision (t)
Recall (t)
100
0
1.55
3.50
0.6569
0.5932
250
0
1.40
2.77
0.6938
0.6115
Lin. CFG
500
0
1.35
2.03
0.6894
0.6262
1000
0
1.45
1.80
0.7071
0.6186
100
0.9
3.90
2.94
0.6653
0.6111
250
0.9
4.35
2.16
0.6688
0.6298
Nonlin. CFG
500
0.9
4.35
1.62
0.6688
0.6298
1000
0.9
4.75
1.48
0.7228
0.6506
Table 13: CIFAR-10 Results
Version
Num. steps
FID (4)
Precision (t)
Recall (t)
25
1.55
5.81
0.3842
0.4632
50
0
1.40
4.01
0.4161
0.4750
Lin. CFG
75
1.60
3.81
0.4120
0.4724
100
0
1.65
3.35
0.4306
0.4967
25
0.9
3.40
4.97
0,4022
0.4564
50
0.9
3.35
3.90
0.4241
0.4672
Nonlin. CFG
75
0.9
3.40
3.33
0.4431
0.5050
100
0.9
3.85
2.88
0.4539
0.5263
Table 14: IMNET-32 Results
Version
Num. steps
a
w
FID (L)
Precision (1)
Recall (t)
25
0
1.60
5.32
0.3996
0.4845
1.80
4.48
0.4484
0.4962
Lin. CFG
75
0
1.70
4.37
0.4576
0.5137
100
1.75
4.34
0.4511
0.5267
25
0.9
3.80
4.76
0.4212
0.4718
50
0.9
4.00
4.02
0.4656
0,5183
Nonlin. CFG
75
0.9
4.20
3.75
0.4734
0.5512
100
0.9
4.15
3.71
0.4883
0.5563
Table 15: IMNET-64 Results
42

---

Published as a conference paper at ICLR 2026
Method
CIFAR-10 FID (4)
IMNET-32 FID (4)
IMNET-64 FID (4)
Uncond. Cond.
3.142
3,891
5,930
2.401
3.167
4.277
Lin. CFG w. W +
2,400
3.176
4.299
Nonlin. CFG w +
2.399
3.194
4.285
Table 16: FID Comparison Across Datasets
GMM Diffusion Sampling with Standard Classifir-tree Guidance
Uncondioonal 0.0)
Component 0 (ar 150.3, 3, a- 0.40
Comporent jum 152.5, d= 0.0)
Comsorent 2 juw 150.5, a-
Conferse
Conderane
Figure 17: Numerical simulation of Standard CFG in a mixture of Gaussians
GMM Diffusion Sampling with Norlinear Classifler-Free Guidance
Uncondilonal 0.00
Comoonert
Compenert Lsve
Componert 120
Figure 18: Numerical simulation of Nonlinear CFG in mixture of 3 Gaussians
Projection Component 100.0, 0.9))
Componeed
Compasent
Component
Companent
10C
63
25
360
125
375
263
16
20 Paacent
Figure 19: Numerical simulation of Standard CFG in mixture of 4 Gaussians
Projection Histograms Companent GMM (D-1009, - 00000-00..0 0.99
Compasent
Companent
Companent
Companest
82
-184
Figure 20: Numerical simulation of Nonlinear CFG in mixture of 3 Gaussians
43

---

Published as a conference paper at ICLR 2026
G.8.1
QUALITATIVE ANALYSIS: VARYING W, FIXED
Standarrd CPC
Nom-limar CFC,
(a) Class 107: jellyfish with
(b) Class 107: jellyfish with
(c) Class 309: bee with 0.
(d) Class 309: bee with 0.9
Standard CFO
Non-lnner CFO. a.
(e) Class 305: dung beetle with
(l) Class 305: dung beetle with
Figure 21: Generated images for different classes for varying values of w and Each panel shows the effect changing a from 0 to 0.9, demonstrating the impact on diversity and image quality.
44

---

Published as a conference paper at ICLR 2026
G.8.2 QUALITATIVE ANALYSIS: FIXED W, VARYING
2016
(a) Prompt bear in its natural habitat
(b) Prompt bouquer of roses
Purrert
Pronept LID of STTEIngL
(c) Prompt bowl citrus fruit
Prompt simple drawing a smiling
Figure 22: T2IM generated images for different prompts with w = 4. and varying value of a.
(a) Class 107: jellyfish
(b) Class 305: dung beetle
(c) Class 309: bee
(d) Class 959: carbonara
Figure 23: CC generated images for different classes with w and varying value of
45

---

Published as a conference paper at ICLR 2026
G.8.3
GENERATED IMAGES BY DIT/XL-2 (256x256)
Figure 24: Additional examples generated by DiT/XL-22 using Standard CFG (w 4) and Power- Law CFG (w 8., a 0.7). Image pairs start from the same noise (same seed). The resulting pairs represent Standard CFG on the left and Power-Law CFG on the right.
Figure 25: Gen. images conditioned on the class pineapple with Standard CFG
(w 4).
Figure 26: Gen. images conditioned on class pineapple with Power-Law CFG (w = 8, a 0.7).
46

---

Published as a conference paper at ICLR 2026
Figure 27: Gen. images conditioned on the class water ouzel, dipper using Standard CFG with 4.
Figure 28: Gen. images conditioned on the class water ouzel, dipper using Power-Law CFG
with w = 8,, = 0.7.
Figure 29: Gen. images conditioned on the class vine snake using Standard CFG with w 4..
Figure 30: Gen. images conditioned on the class vine snake using Power-Law CFG with w
8., a 0.7.
47

---

Published as a conference paper at ICLR 2026
G.8.4 GENERATED IMAGES BY MMDIT MODEL (DIFFUSION OBJECTIVE, RESOLUTION 512x512)
Figure 31: Images generated conditioned on the textual prompt Glowing mushrooms in a dark forest. using Standard CFG with (top two rows) and Power-Law CFG with w = 10, a = 0.8 (bottom two rows).
Figure 32: Images generated conditioned on the textual prompt Stunning, breathtaking view of galaxy or nebula using Standard CFG with (top two rows) and Power-Law CFG with 10, a = 0.8 (bottom two rows).
48

---

Published as a conference paper at ICLR 2026
H FURTHER NOTES ON NON-LINEAR CFG
14
Figure 33: (q) versus time the Gaussian binary mixture with d 16 and o² 4. The dashed line are obtained by the standard linear CFG with = 0. 8, 16 from bottom to top. The dotted line are obtained with the Power-Law non-linear scheme f(x) wr -75 with = .5, 1.5 from bottom to top. The full lines are obtained with the non-lineas guidance of Eq.(4) with y = 4 and w = 8, 16 from bottom to top. The Rescaled Power-law non-linear scheme departs from g = at large time on trajectory similar to the linear scheme and to the Power-Law non-linear scheme. But it gives smaller bias at
The first non-linear CFG proposal, the power-law CFG with Φr(s) = wa and a > -1 (in main paper focus 0 but the guidance be applied in fact for any a -1) results in the following guidance scheme:
(S (( S,)))
(43)
As mentioned, the l distance between scores 85. = (F, (z)) is exponentially small both at the beginning of the backward process (as both conditional and unconditional distributions are standard Gaussian clouds) and before exiting Regime (as shown Section 4), after which remains zero. This non-linear scheme automatically switches off in Regime II and has the following properties: choosing a < 0 provides guidance which speeds up convergence to the target at early times, while > 0 dampens the guidance for small 8S, and strengthens it for large 8S, In practice, we found positive values for a to perform best. In numerical experiments for finite dimension biases the distribution obtained t 0 (see Fig.35).
One would like to have different non-linearities applying the regimes > lo and f.g. One possibility is to use the following version, which extends to more general effective distributions Po(ā)e 2 (2s(t) (2x())
with non-standard s(t) and (t)..
Rescaled Power-law CFG. Here, by denoting with (-) the expectation w.r.t. the effective distri- bution Po(ā)e -² 222(()) (()))(2=())) the score difference be expressed as S, (x)) (1/(s(t)o(t)}) (0) (.) where st)) and (t)) are related to the functions f() and g(t by s(t) exp fo f()) and (t)) So drs(r))/s(T)². Therefore the non-lineat function depends on the difference between the estimators of the initial value i, given Z(t), in the class and in the full distribution. This difference typically function that decreases with the time of the backward process. This suggests to use non-linear CFG of the form
5
RPP (2. c) = 5, c) + (((,,c) - ((
[T) - (d)=)"
= (5, + (5, 51
5. - (.) (())))(1)>)
(44)
with positive 7. As will show in Figures 33-34, this non-linear guidance term has interesting performance in terms of combining a rapid drift toward the desired class c at early stages of the backward process together with small bias in the finite distribution in finite dimensional problems.
49

---

Published as a conference paper at ICLR 2026
The behavior of both versions is portrayed in Figure 35: both non-lin. versions yield smaller bias at t = O. Furthermore, Figure 35 also displays additional experiments highlighting the benefits of non-linear versions.
Figure 34: We perform the same experiment in Fig. 33. Left: the value of Kā) (a) Right: value of [ - (z)),, with the same linestyle and color code as Fig. 33.
Jonsen Shonnon Divergence
Projection onto the moan
Score difference
162
3124
2= (CF6)
0.45
-8.
0.75
0.9
50-
255
Jecsert Shannon Divergence
Proection onin the mean
Scree differece
6.5 (3PS)
150-
0.33
135
9
2.0
Kn IFG
Figure 35: Real-world experiments using DiT/XL-2 (Peebles & Xie, 2023) trained on ImageNet-
1000 (Deng et al., 2009): randomly selected class with w = 4, using DDPM (Ho et al., 2020) with 250 sampling steps, averaged over samples. First column: Power-Law CFG. Second column: Rescaled Power-Law CFG (44). Left column: Jensen-Shannon Divergence between the embedded data points corresponding to randomly selected class and the generated samples as a function of reverse time Middle column: mean dot product of the normalized class centroid and the diffusion trajectories Z ///l (both in the latent space) as a function of reverse time т. Right column: Evolution of the distance between conditional and unconditional scores. From all three plots, we can see that using first (second) version of non-linear CFG with a < (7 > 0) results in paths that have smaller JSD, estimated in Wang et al. (2009), throughout the whole trajectory and smaller overshoot of the distribution's mean at 7 = 0. We can also see that the score difference J,S, (z, (x)) has the same qualitative behavior as numerical simulations of Gaussian mixtures.
50


Reviews:
[{"review_id": "JoZvf5inet", "reviewer": "Reviewer_canA", "summary": "The paper observes the problem of mean overshooting and variance shrinkage resulted by classifier-free gudiance during Diffusion Sampling process. The main results are drawn from the connection of dynamical regimes and classifier-free guidance. Main observations are conducted on  the mixture of Gaussian (2 modes) with the same weight and variance. This gives us the division of CFG into three distinct steps (1) guidance before speciation (2) align before exiting regime I (3) follow unguided path. However, when the dimensionality is limited, the observation (2) is no longer correct and over push the conditional information. To solve this problem in lower-dimension setting, the authors propose power-law guidance where it will enhance the guidance when the guidance signal is strong, and purging the guidance when the guidance is weak. The results show the significant improvement", "questions": "Please see the weakness. I would encourage the authors to simplify the paper.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The paper is well-written\n2. The theoretical part is well-written\n3. The experiments are comprehensive and up-to-dated.\n4. The improvement is quite significant\n5. The proposed method is simple", "weaknesses": "The main drawback of the paper lies in its over-complication. In my opinion, simplicity should be viewed as a strength rather than a weakness. The proposed method is inherently simple, and the results are largely empirical. Therefore, I strongly suggest simplifying the presentation and avoiding unnecessary theoretical analysis that does not contribute to the core message.\n\n1. **Overly Strong Assumptions:**  \n   The assumptions made in the paper are excessively restrictive and unrealistic.  \n   - The use of a mixture of two Gaussians is not representative of most real-world datasets.  \n   - The assumption that both Gaussians share the same variance is also impractical and lacks justification.  \n   - Furthermore, assuming the means to be +m and –m is arbitrary and overly specific.  \n   In contrast, *Biroli (2024)* assumes only the existence of two classes without making any assumptions about their distributions, means, or variances. The authors may want to consider relaxing or revising these assumptions to enhance realism and generalizability.\n\n2. **Unnecessary First Key Finding:**  \n   The first key finding adds confusion rather than clarity. It does not appear essential to the overall narrative or conclusions. The paper would benefit from removing this section and instead starting directly from the second finding (e.g., Figures 8 and 12), focusing on the proposed method for addressing low-dimensional cases.\n\n3. **Weak Connection to Previous Findings:**  \n   The third key finding seems only loosely connected to the earlier results. It is unclear how the concepts of “Regime I” and “Regime II” relate to the presented equation. This section requires clearer logical linkage and explanation.\n\n4. **Experimental Results and Resolution:**  \n   The experiments report results for resolutions of 512 and 256. According to the paper’s own claims, one might expect the method to perform even better on lower resolutions such as 64×64 or 32×32. If this is indeed the case, the authors should provide empirical evidence to support this claim.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8oMARq0ymj", "reviewer": "Reviewer_DodZ", "summary": "This paper provides a theoretical analysis of how CFG affects the generation of mixtures of Gaussians via diffusion models. The authors arrive at the result that in the scenario of exact score estimation the effect of CFG on the generated distribution tends to zero as the number of data dimensions tends to infinity. Conversely, they find that in finite/small dimensional cases CFG causes the generated means to shift and the variances to shrink. The authors propose to adjust CFG according to the norm of the score difference in order to mitigate the above effects and evaluation on various image generation models/datasets.", "questions": "See above\n\nMy main issues with the paper are (according to my current understanding)\n1. The proposed approach is weakly linked with the theoretical contribution.\n2. The concrete form of the power-law CFG is not well motivated.\n\nI'm open to increasing my score if the authors can help me understand better the above issues. I have also not read the appendices in detail.\n\nAdditionally, I would like to clarify, is the score difference a vector norm or an elementwise difference?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The theoretical contributions advance the understanding of CFG.\n- The presentation of the paper is clean and professional. Theoretical results are presented in a structured and understandable manner.\n- The scale and variety of the experiments are appropriate.", "weaknesses": "The paper load for ICLR this year has been large, and so I have not been able to spend as much time as I would like on reviewing. I encourage the authors to correct any errors/misunderstandings I may have with regards to the paper.  \n\n\n1. **weak motivation for proposed approach**\n    1. The authors motivate their Power-Law CFG as a method to reduce mean overshoot and variance shrinkage (implying generation is in the *finite dim* regime), but then justify their approach by saying it retains the *high dim* properties of CFG that they've derived. It seems like they are suggesting that their approach won't \"break\" the nice high-dimensional properties of CFG, but then are focused on addressing a problem that only exists in low/finite dim regime. I find this link between the theoretical and practical part of the paper quite weak.   \n    1. It is unclear to me how the proposed approach reduces the mean overshoot and variance shrinkage, as the authors don't provide an explanation in the main body of the paper. \n    1. From what I can tell, any heuristic scaling of CFG [1] (e.g. according to timestep) will also retain the nice theoretical properties, which hardly makes power-law scaling unique. Besides, practically speaking it seems like the proposed approach behaves similarly to existing heuristics, gradually increasing CFG as generation progresses [1].\n    1. The intuitive description of power-law scaling is misleading. First of all the assertion that score difference large -> reliable, small -> unreliable is not clearly explained or motivated. Secondly, $x_t=\\alpha_t x_0 + \\sigma_t \\epsilon$, $S_\\theta = -\\epsilon_\\theta/\\sigma_t$, that is to say from a noise prediction perspective the magnitude of the score estimate depends on the noising schedule. As noise level $\\sigma_t$ decreases over the course of generation this will have a direct impact on the power-law scaling. Importantly, the $\\sigma$ scaling has nothing to do with the actual difference in spatial content predicted by the conditional and unconditional models and is instead dependent on the diffusion time. In fact, the authors acknowledge that applying their scaling directly to flow velocity results in different behaviour, but do not offer much additional insight. \n\n\n2. **Tepid experimental results**\n    1. Although improvements over vanilla CFG seem notable, improvements over other improved CFG methods are much more marginal. Moreover, due to the aforementioned weak motivation there isn't really a clear explanation for why to prefer power-law scaling to competing approaches. \n\n3. **Lack of intuition to support theory (minor)**\n   1. For a general reader interested in using/improving CFG, I feel like this paper would benefit from more intuitive explanations to help understanding. Personally, I still have not been able to build an intuition for why the effect of CFG disappears when dimensionality tends to infinity, making it harder to appreciate the contribution. (I have some vague ideas of distributions tending to hyperspherical shells, leading to different behaviour, but nothing concrete.)\n\n[1] Wang et al, Analysis of Classifier-Free Guidance Weight Schedulers, TMLR 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RpjvH99UE1", "reviewer": "Reviewer_hSQr", "summary": "The paper analyzes Classifier-Free Guidance (CFG) for diffusion/flow generative models. Using a high-dimensional dynamical-regime framework, it proves that in sufficiently large dimensions CFG reproduces the correct conditional distribution and accelerates convergence before “speciation” (transition between Regime I and II). In finite dimensions, it formally characterizes mean overshoot and variance shrinkage and attributes them to a more confining effective potential. Motivated by this, it introduces a nonlinear “power-law” CFG that scales the score difference by $|\\cdot|^\\alpha$, which provably mitigates both artifacts while preserving the high-dimensional guarantee. Experiments span Gaussian-mixture simulations and ImageNet/T2IM models (DiT, EDM2, MMDiT), showing improved FID/recall and robustness to $\\omega$.", "questions": "refer to weakness", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper's main theoretical punch is the \"blessing-of-dimensionality\" result. The authors formally prove for the first time that CFG can generate the exact target distribution under appropriate conditions (infinite dimensions).This is a huge deal because it formalizes why this heuristic is so stable and effective in high-dimensional domains like images. (Eq. (4)–(6); Sec. 4.1; Appx. B–D).\n\n2.  The authors' theory also provides a clear explanation for well-known flaws in CFGs. They show that they are mathematically necessary consequences of finite dimensionality and directly relate them to the curvature of the effective potential (Section 4.2; Equation (35)).\n\n3.The authors propose a theoretically derived solution, the power-law CFG, which is very simple. It requires only the addition of a `$|\\cdot|^\\alpha$` term, and it also has theoretically driven reassurances (Equation (7)).", "weaknesses": "1. Theoretical assumptions regarding accurate scores; practical mismatch issues remain unresolved.\nThe paper obtains true scores under high-dimensional guarantees and finite-dimensional analysis assumptions. The paper acknowledges that it cannot explain why (linear) CFGs perform well with imperfect estimators (Section 3.2, last paragraph; Conclusions/Limitations). Direct robustness analysis of the scores to the error/noise of nonlinear rules (e.g., perturbation bounds) is also lacking.\n\n2. The scope of the study focuses on Gaussian mixture structures.\nThe core derivation of this paper relies on mixture models with isotropic covariances and separable scaling (Section 4; Appendices B-D); extensions to non-GMM real data are controversial but unproven. While Appendix C discusses multi-component and non-centered mixture models, empirical validation of multi-class schemes focuses primarily on standard vision datasets rather than controlled non-isotropic cases.\n\n\n3. While $\\alpha$ is considered robust ($\\alpha \\approx 0.9$), the method introduces a second hyperparameter; more extensive ablations (dataset, sampling step size, $omega$ range) are shown in Appendix G, but some edge cases (e.g., extremely small $omega$ or very low step size) require minimal exploration. (Figure 4; Appendix G).\n\n4.  I note that you cite an extension of flow matching with velocity prediction (Appendix G.2.1 notes), but the main theoretical section does not provide a complete derivation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EHgchchgnj", "reviewer": "Reviewer_QJyR", "summary": "This paper shows that the CFG artifacts of overshoot and variance shrinkage that have been observed in low dimensions vanish in high dimensions, via a theoretical analysis of dynamical regimes. They then propose a nonlinear extension of CFG that mitigates the low dimensional artifacts while preserving the high dimensional behavior.", "questions": "The Biroli dynamical regimes analysis applied to the CFG deals with a mixture of two Gaussians. How can we be sure the conclusions carry over to more complex multimodal distributions?\n\nHow does this analysis relate to the common empirical observation that conditional diffusion models often start to behave like unconditional denoisers at low noise levels? (Does the analysis actually rely on that behavior? Or can it help explain it?)\n\nL328: Do you require $0 < \\alpha \\le 1$ or can $\\alpha$ be greater than zero?\n\nClarification: (L332) When the difference between conditional and unconditional scores is zero, CFG automatically reproduces the conditional score. So the goal of the power-law is to further (nonlinearly) damp small (but nonzero) score differences?\n\nWhy is Power-Law CFG helpful in high-dimensional settings like ImageNet? I thought the analysis showed that CFG is already \"correct\" in high dimensions (i.e. recovers conditional distribution without distortion)?\n\nMinor edits:\nL 82-83 did you swap descriptions of mean overshoot and variance shrinkage?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is clear and well-written. The problem is well-positioned within the literature (particularly wrt Chidambaram and Biroli). It is nice to see the question of why CFG 'works' despite the known issues in low dimensions being addressed. The dynamical regimes analysis is interesting. The improved diversity on ImageNet is promising.", "weaknesses": "Please see questions.\n\nAs the authors note in the limitations, although the theory can help improve diversity while maintaining fidelity/quality, it still cannot explain how CFG improves fidelity/quality.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)