Feedback Guidance of Diffusion Models
Authors: Felix Koulischer
Florian Handke
Johannes Deleu
Thomas Demeester
Luca Ambrogioni
Venue: NeurIPS 2025 poster ()
Keywords: Diffusion models
Classifier-free guidance
Diffusion guidance
Primary area: deep_learning
TL;DR: We introduce Feedback Guidance, a trajectory specific guidance scheme that relies on the models own prediction to dynamically adapt the value of the guidance scale.

Abstract: While Classifier-Free Guidance (CFG) has become standard for improving sample fidelity in conditional diffusion models, it can harm diversity and induce memorization by applying constant guidance regardless of whether a particular sample needs correction. We propose **F**eed**B**ack **G**uidance (FBG), which uses a state-dependent coefficient to self-regulate guidance amounts based on need. Our approach is derived from first principles by assuming the learned conditional distribution is linearly corrupted by the unconditional distribution, contrasting with CFG's implicit multiplicative assumption. Our scheme relies on feedback of its own predictions about the conditional signal informativeness to adapt guidance dynamically during inference, challenging the view of guidance as a fixed hyperparameter. The approach is benchmarked on ImageNet512x512, where it significantly outperforms Classifier-Free Guidance and is competitive to Limited Interval Guidance (LIG) while benefitting from a strong mathematical framework. On Text-To-Image generation, we demonstrate that, as anticipated, our approach automatically applies higher guidance scales for complex prompts than for simpler ones and that it can be easily combined with existing guidance schemes such as CFG or LIG.

Feedback Guidance of Diffusion Models
Felix Koulischer
Florian Handke²
Johannes Deleu'
Thomas Demeester
Luca Ambrogion
Ghent University imec
2 Donders Institute for Brain Cognition and Behaviour, Radboud University
Abstract
While Classifier-Free Guidance (CFG) has become standard for improving sample fidelity in conditional diffusion models, can harm diversity and induce memoriza- tion by applying constant guidance regardless of whether particular sample needs correction. We propose FeedBack Guidance (FBG), which uses state-dependent coefficient to self-regulate guidance amounts based on need, Our approach is derived from first principles by assuming the learned conditional distribution is lin- early corrupted by the unconditional distribution, contrasting with CFG's implicit multiplicative assumption. Our scheme relies on feedback of its own predictions about the conditional signal informativeness to adapt guidance dynamically dur- ing inference, challenging the view of guidance as fixed hyperparameter. The approach is benchmarked on ImageNe15 12x5e where it significantly outperforms Classifier Freeee Guidance and is competitive Limited Interval Guidance (LIG) while benefitting from strong mathematical framework. On Text-To- Text-To-Image gen- eration, we demonstrate that, as anticipated, our approach automatically applies higher guidance scales for complex prompts than for simpler ones and that it can easily combined with existing guidance schemes such as CFG or LIG. Our code is available at this link
Introduction
At the heart of the image and video generation revolution led by diffusion models Sich--Dickstein et al.
[2015). Ho et al. [2020). [Song et al. [2)21ab). Karras et al. [2022 lies the conditioning algorithm
most well-known the guidance mechanism Dhariwal and Nichol 2021 Ho and Salimans 2021]
The need for diffusion guidance stems from the inherent challenge of learning conditional distributions
with limited paired data, which why further emphasizing the signal using parameter referred to
as the guidance scale proves advantageous in practice Dhariwal and Nichol [0021]. This parameter allows to push the predictions from the conditional model further from those of the unconditional model, essentially reinforcing the difference between them Ho and Salimans (0211) Karras et al. (2024a) This more well-known as the Classifier Free Guidance (CFG) algorithm Ho and Salimans 2021T. It is only after the introduction of the guidance algorithm, that diffusion models gained impressive performance they are nowadays known for. Recent works have however made clear that applying guidance the entire sampling trajectory is not only unnecessary but also harms performance Kynkäänniemi et al. 2024). Sadat et al.) [2024). By applying guidance in the beginning of the generative process, the model tends to converge to specific condition -edetdeendeeeedennt low frequency details Dieleman 22024), Ho and Salimans [202]) Kynkäänniemi et al. [2024), severely reducing the diversity of the generated samples. On the other hand, applying guidance towards the end of the generative process is largely unnecessary as both unconditional and conditional estimates converge,
1 Joint Senior Authors Corresponding author: felix.koulischer@uueer
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Dynamic guidance scale
2-
D
1.5-
X[
0.2
0.6
0.8
0.4
Diffusion time
Diffusian time
(a) Example conditional diffusion trajectories
(b) Corresponding dynamic guidance scale
Figure 1: Illustrative diffusion trajectories and their hypothetical guidance scales in a 1D setting. Trajectories farther from the mode near the decision window (red, orange) receive stronger guidance, whereas those clearly heading toward the right mode (yellow) receive negligible guidance.
both being equally capable of denoising the high-frequency features Kynkäänniemi et al. 2024]] Karras et al. [202aa]
These results can be understood thcoretically based the recently developed theory of spontancous
symmetry breaking phase transitions [aya and Ambrogioni [0023] Sclocchi et al. 2024 Ambrogioni
(025) This theory predicts that called "speciation transitions during generative diffusion
correspond to particular 'decision points' when some features the generations, such as for example
class identity, maximally sensitive guidance Biroli al. (2024), , [i and Chen (2024] Handke
et al. 2025). Since the timing of these critical windows depends on both the data and the conditioning signal, it natural consider dynamic forms of guidance where the guidance scale is determined state-dependently.
In this work, we provide principled methodology to obtain dynamic guidance formulas. Our scheme relies on feedback of a quality estimation its current predictions, which is why we refer to our scheme FeedBack Guidance (FBG) in analogy with control theory. As shown in Fig. [ if trajectory is estimated as more likely to be originating from the unconditional model than from the conditional model, the guidance scale increases correct the towards the correct class. Similarly to the empirically justified work on Limited Interval Guidance (LIG Kynkäääniemi et [2024), self-regulated guidance scale is only present at intermediate noise levels, which now follows from first principles.
and realign the sample
Preliminaries
To introduce our FeedBack Guidance scheme (FBG), the working principles behind Denoising Diffusion Probabilistic Models (DDPMs) and ClassiferrFFee--uiddae (CFG) are required. These are summarized in sections 2.1 and 2.2 In section 2.3 overview of the current stand of literature
with respect to diffusion guidance is given.
2.1 Denoising Diffusion Probabilistic Models (DDPM)
Diffusion models Sohl-Dickstein al. 2015 Ho al. 20200] Song et al. [2021a b), Karras et al. 2022 are progressive denoisers, that aim to invert a forward noising schedule by learning the score function, defined as the gradient the ligeliohd of the marginals V_ log (æ:; 8). In the case of Variance Exploding (VE) DDPM, this forward noising process iteratively adds Gaussian noise to clean data distribution. This way, the underlying data distribution q(x,, 0) progressively noised until a Gaussian of very large variance is obtained, i.e. until after T steps g(x; T) ~ Nxx; 0, of I), with σT >> 1. For any forward noising schedule {or (orf" whereby 2+++1 = 2t + Oz41 (with c ~ N(E 0, and of = of - of). the forward process can be directly sampled at any step t, according to q ~ Na To. o, with o1 = Ei=1 of of The goal is to learn score based model, that is able to iteratively reverse this forward process. This backward process can

---

be decomposed into a Markov chain
p( p(æT)
p. (0 x;; i=++1
II
Crucially, each step of the Markov chain is by construction approximately Gaussian pe(# a 0)
with o2- = o - - - Instead of modeling the mean, Pc-1+
it is common to train denoiser that predicts the final denoised output at any diffusion stage Zozz (x In the case of a VE-scheduler, the two are connected by the identity:
Ti- at of
oi of
1
Zott
(2)
It should also be noted that the sought after score funtion is proportional to the learned denoiser.
2.2 Classifier-Free Guidance
Learning highly complex conditional distribution, such as that required for Text-To To-Image (T2I), is a challenging task. In most situations it is only possible to poorly approximate this target distribution, which can be seen from the vague predictions generatedby sampling a "pure" conditionally trained diffusion model, such as the one present in Stable diffusion Rombach et al. [2022]. To amplify the conditioning signal present during the denoising process, diverse solutions exist. The most widely used method is that of diffusion guidance, of which the simplest example is that of Classifier (-Free)
Guidance Dhariwal and Nichol (02T]. Ho and Salimans [002]] The guidance mechanism used in
all variants of diffusion, from Flow Matching Lipman al. [0033] Zheng et al. [2023] Discrete Diffusion Models Schiff et al. (2025). The reasoning behind guidance is to sharpen the marginals towards the the postenor likelihood ps, (c(x() using exponent À referred to the guidance scale, i.c. to consider X-sharpened conditional marginal distribution 5e (2)z)c)
po,, ( )po ( xx When À is equal to one this reduces to sampling the conditional model, while setting À >>> 1 further sharpens the marginals towards regions that better satisfy the condition To obtain Classifier-Fin Free scheme, independent of the posterior Pe. (c) it suffices to rewrite the posterior probability as ratio of the conditional and unconditional likelihoods po, (c(.a) po.a((xac)//po.c score-based perspective this results in the well known guidance equation:
From
Va log P. (x (c) = V. p (xx) + AV log (o, V log + A logPo, - Va log log
In the first formulation the gradient of a classifier appears, if this likelihood is directly parametrised using pretrained networks one obtains what is refered to training-free free guidance Dhariwal and Nichol [2021], J. Shen et al. [2024a]. The main challenge of these approaches is that they leverage a
model trained solely clean samples to offer guidance on noisy samples Shen et al. [2024a] Ye et al.
2024].
The second equation is that of Classifier Free Guidance Ho and Salimans [021] which has
main inconvenient that requires joint conditionaluuncoddiioonn model training Ho and Salimans
2021
Rombach et al. 20022] Karras et al. 2022 2024b).
2.3
Related Work
Conditional generation, and in particular the topic of guidance, is very active research field. As this paper is centered around Classifier- Guidance and its derivatives in the context of Diffusion Models, this section will provide a concise exploration of the field's key developments, with readers seeking comprehensive context directed to existing in-depth survey literature Anonymous [2024], Adaloglou and Kaiser 2024]. A prominent research direction preserves the framework of CFG while introducing various predefined time-varye varying profiles to replace the rigid constant guidance scale Sadat al. [2024), KKynkäänniemi al. (2024), i Wang al.) (2024) Xia et al. (2024). Noteworthy, due to its computational advantage, simplicity and its effectiveness. is choosing a limited interval in which guidance should be applied Kynkäänniemi et al. (2024). Precisely where this limited interval is located depends on the underlying quality of the conditional model. The better the model, as is
"qquivalently, the noise é can be predicted, the two connected by the identity &aj orcs
3

---

the case for the EDM2 models on which that particular paper focuses, the later the guidance can be activated. Different intervals are found when evaluating performance using the FID Feeeel et al.) [2017] or vinov Stein et al. [023). the latter being much wider and earlier.
Less researched the approach
state-dependent guidance scale, which has been shown to be
theoretically optimal in the context of negative guidance Koulischer et al. [2025] Kim et al. [2025].
but has only been heuristically proposed for positive guidance Brack et al. [2023] Shen et al. [2024b].
Our work is the first to derive a state- and time-dependent guidance scale from first principles.
Another advance is autoguidance, which replaces the expensive unconditional model by a much weaker one et al. [2024a]. The weaker model may be smaller version of the same architecture, an undertrained model, or one incorrectly conditioned at earlier timesteps Karras et al. 2024a Kaiser ct al. [2024). Li al. [2024. This approach reinforces not only class-specifes details but also image quality, relying on the smaller model's errors being of similar nature as those of the stronger one, simply larger. Our Feedback scheme can be adapted to produce equations fully compatible with these
principles.
A newly emerging research direction, has proposed step away from guidance whole and instead perform tree-search over a limited amount of trajectories and selectively choose the most promising ones using reward models Guo et al. 2025 Ma al. [2025]. These approaches are still very recent, but might lead to an entire different era of conditional diffusion generation, in which instead of relying on guidance, the model has the ability focus on the most promising trajectories to obtain the most of the learned models. Our feedback approach, and in particular the posterior estimation algorithm, could potentially provide meaningful way of ranking these paths using solely the pretrained diffusion models.
3 Feedback Guidance
This section presents the theoretical foundation of Feedback Guidance. In section 3.1 introduce framework that reformulates guidance formulas as the result of assumptions about systematic errors in the learned distributions. In section 3.2 we demonstrate that adopting an additive rather than multiplicative error assumption naturally produces a dynamic guidance mechanism. The resulting state- and time-dependent guidance scale requires posterior likelihood estimation, outlined in section In section 3.4 the key hyperparameters of FBG are discussed.
3.3
3.1
Interpreting guidance schemes through error assumptions
Here, we conceptualize guidance formulas as the result of inverting an error model that determines how the unconditional distribution 'corrupts' the estimated conditional distribution. This form of corruption is to be expected since the model is far less frequently trained on a given class or prompt, which implies that a large part of the training signal is unconditional. In the case of CFG, by reversing Eq. and substituting 7 1 it becomes clear that CFG implicitly assumes that the learned conditional distribution, denoted by the subscript this work (o.r(<:(c), a multiplicative mixture of the true conditional and unconditional distributions:
Po. (x c) (xx - (e (c)
Pt (xx) ( p
px, (c)
In the case when s 1 (low guidance regime), the modelled conditional corresponds to the true conditional. In that setting, sampling the learned conditional is sufficient. However, when y
Compute scores u(zale), p(xx))
Mix scores + Take step
C; A
1ex (x (c) = ((x)) + >e (u(arle) - m(xx)) xx A
Update guidance scale value A = Update(x Ae, p(xzlc), ((x)))
Figure 2: Schematic of Feedback guidance (FBG). The state space consists of both æ; and X, which are updated iteratively during the denoising process. The guidance scale is updated by tracking the posterior ratio thanks to Eq. (IT), which can then be inserted in Eq. (8).

---

(strong guidance regime), the modelled conditional resembles the unconditional distribution, which precisely why strong guidance required during sampling.
3.2 Feedback Guidance
In control theory jargon, both standard CFG and LIG examples of open-loop controllers since the guidance scale is not a function of the current state Φt. This means that the guidance formula will equally affect all states, regardless of their quality and class alignment. We argue that might lead to over-saturation and stereotypical generations in situations where the conditional model is already good enough to be sampled on its own, as is the case for simplistic or memorized prompts. Here, we derive guidance formula that implements form of feedback, or closed-loop control. From an error perspective we assume that the learned conditional distribution corresponds to an additive mixture of the true conditional and unconditional distributions:
Pe, (æ,c( (1 1 (x)) + (pp
(5)
The additive assumption can be seen as less restrictive than the multiplicative one as it allows the learned conditional distribution Pa, (x, to be non-zert zero in regions where the true conditional distri- bution pe (ar (e) is zero, a feat the multiplicative assumption is incapable of. Due to the joint training pipeline, and the fact that training pairs often contain more than single element, such an overlap of the learned distributions is in practice highly likely. Assuming well-modeled unconditional distribution, i.e. (æ.) 2 pe (3x), this implies sampling:
Pt (axle) (xlc) - (1 (x))
(6)
In other words, we propose removing a portion of the unconditional distribution from the modeled conditional before sampling. This, similarly to the approach taken in CFG, helps strengthen the conditioning signal, as regions that do not satisfy e are pushed towards zero-likelihood Of key interest for sampling is the score function of the underlying conditional distribution Va (xrle),,, which can be derived using the chain rule
Va logg(().c)) = Ve logpo,r(xx)) + M(xi, t) V log (o.(((c))))))) - Va, log po.p..(aa)..
(7)
with as guidance scale:
po.((elas)/pnx(() A(x1, po.:(cl=:)/oo.rr (c)
(8)
The additive error model results in state- and time-dependent guidance scale that can be expressed terms of the posterior likelihood (e)((c) The guidance scale equal to one when the posterior likelihood is high, and exhibits asymptotic behavior the posterior approaches 1 1 The mixing parameter π determines when guidance is deemed necessary: if i is close to one, indicating a well learned distribution, guidance is only activated when P8, (c) xx) reaches very low values. In contrast, for poorly learned distributions, with smaller values of т, guidance is easily activated as soon as the posterior decreases. It should be noted that if 0 < po, (c)arr) < 1 the guidance scale is negative. We argue that in the continuous case this situation would never arise since the guidance scale would need to cross the asymptote, which should in turn increase the posterior. To avoid this happening in the discretized case, we clamp the posterior to a minimum value Pmin. which in practice implies clamping the guidance scale at Amax
In practice, our dynamic guidance scale defined by Eq. (8) can easily be added on top of any pre- existing guidance method such as CFG or LIG. To make it clear which methods are used, we introduce the following notation: FBG corresponds to using solely Eq. (8) as guidance scale. FBGCFG and FBGLIG respectively correspond to adding some base CFG or LIG on top. We refer to FBG as all approaches that use variants of Eq. (8) for guidance. The corresponding error models assumed for these schemes are given in Appendix
AA detailed derivation provided Appendix
The two connected by the identity Pmin log ((1 T)) Amaz () for more details Appendix C.2
5

---

3.3 Posterior approximation by tracking the Markov Chain
Our novel dynamic guidance scale X(x, relies on the posterior likelihood (el(<)) which is in general not available using score-based models. Leveraging recent ideas by from Koulischer et al. [2025). we approximate the required posterior p(c(æ;) by estimating the required likelihoods by tracking the diffusion Markov Chain, defined by Eq. ① during the denoising process. Key for this estimation is that the likelihood ratio between the conditional and unconditional models can be updated iteratively through:
log pa, (.p.,.c/aaaae (c)))) log po. po.+11 + log pa, p (141, c)
log po., (z, (x++1)
(9)
Both markov transitions likelihoods are parametrised as gaussians, resulting in:
log p (c) = log (og_po.--1(//)++1) 20 2/1-11
(1 1 - (o,c(00+11)1)
(10)
This equation estimates the posterior by comparing conditional and unconditional model performance at each denoising step, effectively computing likelihood ratio weighted by the inverse noise variance d As the transition kernel sharpens, the posterior estimates become increasingly decisive, allowing for more abrupt shifts in the probability assessment. A crucial advantage of computing the
posterior using the scheme describe above is that it causes negligible computational overhead as all required quantities, in particular (4..1 (X14)) and f (2)+1l(), are already computed.
By tracking the posterior likelihood during inference, we estimate state- and time-dependent guidance scale and feed it back to the denoiser. Crucially, posterior computation and denoising are
staggered, effectively solving a joint ODE-SDE system (Skreta al. 20055 Karczewski et al.) [2025] The closed-loop diagram shown in Fig. and described in detail in
summarizes our approach.
This control diagram progressively unrolled during the denoising process, implying a succession of
computing the score functions, mixing them, applying denoising step. updating the value of the
guidance scale and repeating a fixed amount of times until a fully denoised image obtained.
3.4 Defining Practical Hyperparameters
The previously described posterior likelihood estimation via Eq. 10 however suffers from self- reference bias: when using the conditional model's prediction (æ; (4,00(22+1()) the sampling trajectory, the model effectively evaluates its performance on its own output, artificially inflating its perceived accuracy. This creates circular reasoning problem where the conditional model always appears superior because it evaluates a trajectory it created. We address this by introducing a linear bias term -ō (Eq. [I]. which allows the unconditional model's predictions to receive appropriate consideration. This adjustment forces the system to recognize that the conditional model's apparent superiority stems from self-comparison rather than objective performance, creating more balanced sampling process that better represents the true posterior distribution. In practice, this forces the posterior to decrease the early stages of diffusion, enabling the activation of guidance.
log Pt (clar;) log P++ (c/a)+1)
20? (I|ar (1,0 ||,r (1,0 )lP) [12) (11)
The complex non-linear interplay between the three hyperparameters of our approach -m, and 8- makes it challenging to predict how changing one parameter affects the overall guidance profile. To address this issue, we propose more intuitive parameterization that allows users to directly control the characteristics of directly tuming T and 8. express them functions of two more interpretable parameters: to and f,. Here, to represents the normalized diffusion time at which the guidance scale reaches a predefined reference value A (set 3 without loss of generality), while i represents estimant of the normalized diffusion time at which the guidance reaches its maximum value. Details regarding the hyperparameters provided in Appendix C.2
the guidance profile through normalized diffusion timesteps. Instead

---

FID (L)
FD Dinov2 (L)
Prec. (1)
Rec. (T)
Guidance scheme
Stoch.
PFODE
Stoch.
PFODE
Stoch.
PFODE
Stoch.
PFODE
CFG
5.00
2.97
100.2
88.4
0.85
0.84
0.73
0.75
Weight scheduler
4.58
2.75
103.1
97.1
0.84
0.83
0.74
0.76
CFG++
/
3.66
/
87.8
/
0.86
/
0.73
LIG
3.59
2.31
88.5
77.1
0.86
0.86
0.75
0.77
puGe (ours)
3.76
2.50
89.0
75.6
0.86
0.86
0.76
0.77
FBGLI (ours)
3.62
2.45
87.9
74.6
0.87
0.86
0.75
0.76
Table 1: Evaluation of different guidance methods using EDM2-XS using a stochastic ("Stoch.') and 2nd -oddrr Heun sampler ("PFODE"). FID and FD values refer to the model optimized under the respective metrics. Precision and Recall are computed on 10,240 samples with nearest neighbour with models optimized under FDDina22 Kynkäänniemi al. 2019). Stein et al. [2023]
4 Results
We validate our novel state-dependent dynamic guidance scheme on ImageNet5I 12x5 1 using EDM2 models, where it consistently outperforms CFG and remains competitive with LIG, while benefitting from strong mathematical framework. To assess generality, we additionally compute FIDs on MS-COCO in the T2I setting. Alongside these quantitative results, we provide qualitative examples showing that the self-regulated feedback guidance scale naturally increases with prompt complexity.
4.1 Comparison of Guidance Schemes on Class Conditional Generation
For class-conditional experiments, we use EDM2-XS Karras et al. 2024b al trained on Ima- geNet512x512 Deng et al. 2209 with 64 function evaluations. Performance is measured via FID Heusel et al. [2017]. J. Dinovv Stein et al. [2023), and Precision/Recall Reccll Kynkäääniiemi
al.)
[2019]. providing a comprehensive view of the quality-diversite diversity trade-off K Kynkäänniemi et al. [2019] Stein et al. [023]_ Kynkäänniemi et al. [2024). Consistent with our posterior estimation framework based on Markov chain sampling. we focus on stochastic samplers but note similar results using the probability flow ODE with a 2od-order Heun solver Karras et al. 2022 2024b). Baselines are
optimized per sampler: CFG Ho and Salimans [202T] CFG++[Chung et al. [2025] and the linear guidance weight scheduler Wang et al. [2024) via grid search over the guidance scale, and LIG Kynkäänniemi et al. [2024] via joint search over max and guidance scale, followed by amin tuning")
For Feedback Guidance, we sweep (o. to - 4, and m. We note that CFG++ was originally only tested
on text-to-imant image generation Chung al. 2025 while the linear guidance weight scheduler was only
benchmarked using the FID as metric Wang et al. 2024].
To visualize parameter effects, we present FDDinoV2 sweeps as heatmap in Fig. (FID results
in Appendix F). Optimal hyperparameters are listed in Table B and vary across metrics (FID vs. FDDina22) and samplers. Consistent with LIG, optimal FDD performance requires earlier and longer guidance activation, corresponding to larger to and to t1 for FBG.
FBG outperforms CFG, CFG++ and the linear scheduler both FDDinoV2 and FID, while remain- ing competitive with LIG. To assess the quality-diversity trade-off, we compute Precision- curves on 10,240 images with 5 nearest neighbor with optimal FDDinoV2 settings. CFG and LIG are swept over guidance scales 1-4, while FBG is swept over to with fixed ly - 11 = 0.125 (8/64 steps). Results confirm that CFG improves quality at large cost to diversity. LIG better preserves diversity due to its narrow late-stage guidance, and FBG achieves CFG-like Recall but with substantially higher Precision, offering a better quality-diversity balance.
Finally, we optimize FBGLIG by fixing the LIG interval and solely varying the guidance scale, with m and to taken from FBG and f adjusted. While full grid search could further improve results, this hybrid outperforms its components on FD which aligns closely with human perception Stein al. [2023). highlighting the complementarity of our approach. All optimized metrics are provided in Table metrics per sampler are written in bold and best performing overall further underlined.
and [and
additional details on the various methods are provided in Appendix F Best performing
'Contrary to Kynkäänniemi et al. 2024 we find that late stte-stage guidance is not only unnecessary but also harmful, particularly for FDpinoV2.
7

---

Optimall:i
CFG LIG FBG
CFG: 100.5
0.9-
Preciston (Quality 1.)
High guidance
FDDinav2
1. 0.0.255
0.8-
guidance
[R.8.88.
0.547
0.563
0.578
0.594
0.609
0.6
0.7
0.8
Recall (Diversity 1)
(a) Din minimum
(h) Precision and Recall
Figure 3: (a) Grid search over to and f1 with FDDinv calibrated to the best value among CFG, LIG, and FBG. (b) Precision-Recall sweeps each method's DDnono optimum: CFG/LIG sweep guidance scale, FBG sweeps to at fixed to 4, Guidance strength is indicated by color intensity.
4.2 Guidance on Text-To- Tex--To-Image
In the context of Text-To To-Image (T2I), our approach evaluated using Stable diffusion Rombach itt al. [2022), for which VE-scheduler is implemented Karras et al. [2022] 2024b]. To remain consistent with the theory stochastic sampler using 32 function evaluations used. The purpose of this section is not to investigate to what extent Feedback Guidance may outperform CFG or LIG in terms of image quality, but merely to demonstrate the promise of the approach.
On its own, the conditional model used in T2I applications is of far lower quality than is the case for the EDM2 models, which precisely why practice much larger guidance scales are required HH and Salimans [202T]. Rombach et al. (2022). For FBG this implies that has to be chosen much smaller, all images shown this section use value of π 0.85 in combination with to = 0.75 and 4 = 0.5. In practice, we also find it helpful to remove the offset from the posterior approximation towards the end the generative process We find that using a limited amount CFG with ACEG = 1.5 help retrieve low frequency features such sharp colors, without significantly harming diversity, which why in this context we propose to use FBGCFG for visual evaluation. To assess approach beyond visual inspection, on 3k
compute FID, FD V2 and Aesthetic-See Scoor
prompts Lin et al. [2014), Heusel et al. 2017]. Oquab et al. 20024). Results are given in Tab. and show similar trends to those from the cass-conditional setting, underscoring the method's 's generality. Although MS-COCO captions are relatively simple and thus less ideal for testing guidance methods, most effective complex prompts, it remains the standard benchmark for qualitative T2I evaluation.
Guidance Scheme
FID (L)
FD FDDino22 (1)
Aesthetic Score (t)
CFG
19.64
54.56
5.65
LinCFG LIG
19.15
53.26
5.71
18.81
54.25
5.74
FBGpure (ours)
18.63
53.11
5.75
FBGCEG (ours)
18.63*
52.14
5.75
Table 2: Comparison of methods across FID, FDDino22, and Aesthetic Score. Evaluated using 3k
prompts from the MS-COCO dataset Lin et al.
(0144] using SDv2 Rombach et al. 20222.
We now emphasize two key, novel, properties of our feedback guidance approach:
Prompt specificity FBG: A key advantage of Feedback Guidance is that it adapts denoising behavior per prompt, unlike fixed fixed-profile methods that treat all prompts identically. Well-learned prompts (e.g., "The Starry Night") should receive minimal guidance, whereas complex, descriptive
"To be specific in this context choose J 0 iff 0.3.
haspaa///44444eeeeeeeeeerrrrrr/o predictor--2-- The optimised hyperparameters given in Table
"Moeel accesible at

---

"The starry night"
Avcraige quidance scale
Diftuiion Eme
3.0
2.5
"Chame te with
Guidan stals
graffiti"
2.0
0.5
Diitusion Lime
(a) Examples of dynamic guidance scale
(b) Average guidance for various prompt difficulties
Figure 4: Analysis of FBG the context T2I. In (a) the dynamic guidance scale 32 samples are shown using prompts: memorized one ( "TTh starry night by Van Gogh") and more difficult one "A chameleon blending into graffiti-covered wall"). In (b) the average guidance scale applied when using FBG is shown function of various prompt difficulties specified in Appendix G
CFG (A = 2.5)
Conditional model
(ours)
CFG 2.5)
Prompt average
Guidance scale
Image
Image
Image
image
0
60
0.6
Diffusion time
Figure 5: Guidance scale for different trajectories using the prompt: "A snail crawling on green leaf with water droplets". ". If the conditional prediction is good the guidance is low (top two images). In contrast when the conditional prediction is poor, the guidance scale increases (bottom two images).
prompts require stronger guidance. test this, we construct a 60-prompt dataset with four difficulty levels: memorized, easy, intermediate, and very hard. More detail on this dataset provided in Appendix G We report the average guidance scale obtained with 32 samples per prompt and find, as shown in Fig. 4b that FBG applies the strongest guidance for the most difficult prompts, expected. provide further examples Appendix F.
Trajectory specificity of FBG: Another key property of FBG is its state, or trajectory, dependence. For the same prompt, two trajectories might receive entirely different levels of guidance. This is illustrated in Fig. 5 if the conditional model is, by chance, already close to the desired result, minimal amount guidance is applied, whereas if the conditional model far off guidance much more present.
Limitations and Future Work
Our FBG approach opens several new directions in both theory and practice of guidance. The adopted additive mixture model was chosen based on its mathematical simplicity, but likely not close approximation of the true systematic biases in trained models. More precise error models could potentially obtained by studying the training dynamics of the models and tracking the relative error of the learned conditional and unconditional scores, which may lead to more accurate

---

dynamic guidance formulas. Similarly, our choice of prior is solely based on simplicity, and other options should be investigated both theoretically and empirically. Our evaluation currently focuses on FDDM2-XS models. Future work should investigate larger architectures such as EDMM2-L or DiT Peebles and Xie [2022), and extend quantitative T21 evaluation to broader prompt datasets, such as LAION5B Schuhmann et al. [0022. to better assess guidance performance across different prompt complexities.
Conclusion
In this work, we introduced novel view of the commonly used guidance mechanism, interpreting guidance as a way of rectifying the errors made by the learned conditional model. By replacing the implicitly assumed multiplicative error of Classifier-Free Guidance (CFG) with an additive one, we obtained Feedback Guidance (FBG), a state- and depeedent guidance mechanism that dynamically relies on the model's own prediction to estimate how much guidance is needed during inference. This work challenges the view of guidance as a fixed global scheme and instead allows different trajectories and conditions behave differently. Our results demonstrate that FBG significantly outperforms CFG, and is competitive with Limited Interval Guidance (LIG) while relying on solid theoretical grounds.
Acknowledgments
This research was partly funded by the Research Foundation Flanders (FWO-Vlaandereh) under grant G0C2723N and by the Flemish Government (AI Research Program).
References
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In Proceedings of the 32nd International Confer- ence on Machine Learning, volume 37, 2015.
mlr press/
-337sool-diccsteelne html
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic mod- els. In Advances in Neural Information Processing Systems, volume 33, 2020. URL
https::/proceedings.ss www.:/eeweeenn...............e 4cbbcifcc554arff00eeF11101111111
Jamiing Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In Interna-
tional Conference on Learning Representations, 2021a. URL http:tttss//ppeeeeettt net/
Forum?id=st 1iaii
Yang Song, Conor Durkan, lain Murray, and Stefano Ermon. Maximum likelihood training of score-based diffusion models. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and Wortman Vaughan, editors, Advances in Neural Information Processing Systems, vol- ume 34, 2021b. URL httttp:://ppcceedinggg. wwwrips.eupp.............. cc/paper files/paper/ 2021/file Ttt.ialeeeSSaaabbeeeeeooooo htt.eaaccccccccccaaooe5555222222 (022:11144400003331o/oo000002330 Papeeee--PPp..ee pdf
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the de- sign space of diffusion-bart generative models. In Advances in Neural In- formation Processing Systems, volume 35, pages 05565-26577, 2022.
URL
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans image synthesis. In Advances in Neural Information Processing Systems, volume 34, 2021. URL flle///aaaddeeeaaaaarrrrroaaaaaa
File/
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. In NewrlPS 202/ Workshop on Deep Generative Models and Downstream Applications, 2021. URL https::
net Eorum?idi Forumm2idd-o@@xxxEEW Q
10

---

Tero Karras, Mika Aittala, Tuomas Kynkääänniemi, Jaakko Lehtinen, Timo Aila, and Samuli Laine. Guiding diffusion model with a bad version of itself. In
Advances in Neural Information Processing Systems, volume 37, 2024a.
URL
https: httpp::/prroceeding....ot
Tuomas Kynkäänniemi, Miika Aittala, Tero Karras, Samuli Laine, Timo Aila, and Jaakko Lehtinen. Applying guidance in a limited interval improves sample and distribution quality in diffusion models. In The eightt Annual Conference on Neural Information Processing Systems, 2024.
URL https: /ttp::///ppeeeeieww
net/ hwtp://eee/foottat///ee/m Ad=nA
Seyedmorteza Sadat, Jakob Buhmann, Derek Bradley, Otmar Hilliges, and Romann M. Weber. CADS: Unleashing the diversity of diffusion models through condition-annealee sampling. The Twelfth International Conference on Learning Representations, 2024. URL https: /ttp///peeevveeeeeeeeeeee
net/ ZMoNra k2x
Sander Dieleman. Diffusion is spectral autoregression. Blog post, 2024. URL https:: /sander
a 81/2024/99/02/ http:////////ppcaaaaaaaaaaaaaaaa -utorrgressiion ottp:///oooo/ppaaaaaaaaaattrror. html
Gabriel Raya and Luca Ambrogioni.
Spontaneous symmetry breaking in generative diffusion models. In seventhh Conference on Neural Information Processing Systems, 2023, URL
btttss://openreviee net http://wwwne//ooomddddll----ee
Antonio Sclocchi, Alessandro Favero, and Matthieu Wyart. A phase transition diffusion models reveals the hierarchical nature of data. arXiv preprini arXiv:2402. 2024.
Luca Ambrogioni. The statistical thermodynamics of generative diffusion models: Phase transitions, symmetry breaking, and critical instability. Entropy, 27, 2025. URL httppt:///wwtt mdpi
com/1099 4300022/3/233111
Giulio Biroli, Tony Bonnaire, Valentin de Bortoli, and Mare Mézard. Dynamical regimes of diffusion models. Nature Communications, 2024. URL
541467-024-34281
Marvin Li and Sitan Chen. Critical windows: non-asymptotic theory for feature emergence in diffusion models. arXiv preprint arXiv:2403.0 2024.
Florian Handke, Felix Koulischer, Gabriel Raya, and Luca Ambrogioni. Mcasuring semantic in- formation production in generative diffusion models. In ICLR 2025 Workshop on Deep Gen- erative Model in Machine Learning: Theory, Principle and Efficacy, 2025. URL https V/openreview. net.
R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image syn- thesis with latent diffusion models. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. URL
http:/ol...eeecomuuuttrrooooiiio org/
10. ww.11109//PR22BB30 2022. 01042
Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling, The Eleventh International Conference on Learning Repre- sentations, 2023. URL /ttts:://ppereeeeee
.nt/ wtr.//orrmii.oooo 1 PQWRRDCTTT
Qinqing Zheng, Matt Le, Neta Shaul, Yaron Lipman, Aditya Grover, and Ricky T. Q. Chen. Guided flows for generative modeling and decision making. arXiv preprint arXiv:2311.13444 2023.
Yair Schiff, Subham Sekhar Sahoo, Hao Phung, Guanghan Wang, Sam Boshar, Hugo Dalla-torre, Bernardo P de Almeida, Alexander M Rush, Thomas PIERROT, and Volodymyr Kuleshov. Simple guidance mechanisms for discrete diffusion models. In The Thirteenth International Confer- ence on Learning Representations, 2025. URL https: htttps://ppenreview.cco. htt.//ormmmmmm
15MrJ6g5G1
Yifei Shen, Xinyang Jiang, Yezhen Wang. Yifan Yang, Dongqi Han, and Dongsheng Li. Understand- ing and improving training-fit loss-based diffusion guidance. arXiv preprint arXiv:2403, 12404, 2024a.
11

---

Haotian Ye, Haowei Lin. Jiaqi Han, Minkai Xu, Sheng Liu, Yitao Liang, Jianzhu Ma, James Zou, and Stefano Ermon. TFG: Unified training-free guidance for diffusion models. In The Thirty- eighth Annual Conference on Neural Information Processing Systems, 2024. URL https
Tero Karras, Miika Aittala, Jaakko Lehtinen, Janne Hellsten, Timo Aila, and Samuli Laine. Analyzing and improving the training dynamics of diffusion models. In Proc. CVPR, 2024b.
Anonymous. Conditional image synthesis with diffusion models: A survey. Submitted to Trans- actions on Machine Learning Research, 2024. URL id=ewwNKwh665K Under review.
Nikolas Adaloglou and Tim Kaiser. fusion models. theaisunmer.com, classifier free guidance)
An overview of classifier-free guidance for dif-
2024. URL https://thealsume unmee..ommmmcooo
Xi Wang, Nicolas Dufour, Nefeli Andreou, Marie-Paule Cani, Victoria Fernandez Abrevaya, David Picard, and Vicky Kalogeiton. Analysis of classifier guidance weight schedulers. arXiv preprint arXhv:2404.130400 2024.
Mengfei Xia, Nan Xue, Yujun Shen, Ran Yi, Tieliang Gong, and Yong-Jin Liu. Rectified diffusion guidance for conditional generation. arXiv preprint arXiv:2410.18733 2024.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge to lo- cal nash equilibrium. In Advances in Neural Information Processing Systems. 2017. URL hatpps:/pproceedinosssseuu wwwp://ogs..eurrrs......... co/pper files/paper/201 File/Ba Ba Taeer//e6SSTTTTeerrraa7333373763
George Stein, Jesse C. Cresswell, Rasa Hosseinzadeh, Yi Sui, Brendan Leigh Ross, Valentin Vil- lecroze, Zhaoyan Liu, Anthony L. Caterini, Eric Taylor, and Gabriel Doaiza-Ganem. Expos- ing flaws of generative model evaluation metrics and their unfair treatment of diffusion mod- els. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL net. www.ne//ooommmm000em2m
Felix Koulischer, Johannes Deleu, Gabriel Raya, Thomas Demeester, and Luca Ambrogioni. Dynamic negative guidance of diffusion models. In The Thirteenth International Conference on Learning Representations, 2025. URL wttps///e/pp..eeiieeemmoo n forum ?id= 6p 740yApl
Mingyu Kim, Dongjun Kim, Amman Yusuf, Stefano Ermon, and Mi Jung Park. Training- ffree safe denoisers for safe use of diffusion models. arXiv preprint arXiv:2502.0800 08011,, 2025.
Manuel Brack, Felix Friedrich, Dominik Hintersdorf, Lukas Struppek. Patrick Schramowski. and Kristian Kersting. SEGA: Instructing text-to-image models using semantic guidance. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https
//openreview.co www.:nrrwvieee.eeeeeeeeeee fneet//orrmm1ddkiaaa.a
Dazhong Shen, Guanglu Song, Zeyue Xue, Fu- Yun Wang, and Yu Liu. Rethinking the spatial
inconsistency in classifier-free diffusion guidance. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024b. doi: 10. 1109/CCVPR52733.2224.0000
Tim Kaiser, Nikolas Adaloglou. and Markus Kollmann. The unreasonable effectiveness of guidance for diffusion models. arXiv preprint arXiv:2411. 10257, 2024.
Tiancheng Li, Weijian Luo, Zhiyang Chen, Liyuan Ma, and Guo-Jun Qi. Self-guidance: Boosting flow and diffusion generation on their own. arXiv preprint arXiv:24/2.05887, 2024.
Yingqing Guo, Yukang Yang, Hui Yuan, and Mengdi Wang. Training-free guidance beyond differ- entiability: Scalable path steering with tree search in diffusion and flow models. arXiv preprint 122. 2025.
Nanye Ma, Shangyuan Tong, Haolin Jia, Hexiang Hu, Yu-Chuan Su, Mingda Zhang, Xuan Yang, Yandong Li, Tommi Jaakkola, Xuhui Jia, and Saining Xie. Inference-time scaling for diffusion models beyond scaling denoising steps. arXiv preprint axXhv:2501.09732, 2025.
12

---

Marta Skreta, Lazar Atanackovic, Joey Bose, Alexander Tong, and Kirill Neklyudov. The super- position of diffusion models using the itō density estimator. In The Thirteenth International Conference Learning Representations, 2025. UR..htps:: Sictps://openrviit ew www. f
id=2o58Mbqkd2)
Rafal Karczewski, Markus Heinonen, and Vikas Garg. Diffusion models as cartoonists: The curious case of high density regions. In The Thirteenth International Conference Learning Representations, 2025. URL https::///// http:::://ppeeevvieee opwprevieeeeeeeeeeee/ee/ee
Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall metric for assessing generative models. In Advances in Neural Information Pro- cessing Systems, volume 32, 2019. URL
neur iisuripp.oo httccc//pppee paper. pddf
Fil wespppapr/20111 9/ 9twfl/////. 023 01113344500000000000000000000010
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: large-scale hier- archical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, 2009. doi:
Hyungjin Chung, Jeongsol Kim, Geon Yeong Park, Hyelin Nam, and Jong Chul Ye. CFG++: Manifold-constrained classifier free guidance for diffusion models. In The Thirteenth International Conference on Learning Representations, 2025. URL hetttps://opezeeveer net/ forum?
Id=E77uvb0Tts
Tsung-Y Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision ECCV 2014. Springer International Publishing. 2014. ISBN 978-3-319-10002-1
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V. Vo, Mare Szafraniec, Vasil Khali- dov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El-Nouby, Mido Ass- ran, Nicolas Ballas, Wojeiech Galuba, Russell Howes, Po- Yao Huang, Shang- Wen Li, Is- han Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Herve Jegou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski. DINOv2: Learning robust vi- sual features without supervision. Transactions Machine Learning Research, 2024. URL https::/apenrev/ ew wwt.:etttooteeee mttt:oorummid 6zFt.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In 2022 IEEE/CVF
International Conference Computer Vision, 2022.
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade W Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa R Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarezyk, and Jenia Jitsev. LAION-5b: An open large-scale dataset for training next generation image-text models. In Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks net forum 2iciummiiiMM333 4 4
Track, 2022. URL /tltps://openreeeee wwwperrvwww....eeeo........mmma.
Arwen Bradley and Preetum Nakkiran. Classifier-free guidance is a predictor-corrector In NeurlPS 2024 Workshop on Mathematics of Modern Machine Learning, 2024. URL https 7/openreview oot.://orrrmm...2222.a...t..
Muthu Chidambaram, Khashayar Gatmiry, Sitan Chen, Holden Lee, and Jianfeng Lu. What does guidance do? a fine-grained analysis in a simple setting. In The Thihty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL httthtt:: /tttpp:://openrevee...o www.:/eeee/frurnae
Id=AASSHBSaP1
Sinho Chewi, Alkis Kalavasis, Anay Mehrotra, and Omar Montasser. Ddpm score matching and distribution learning, arXiv preprint arXiv2504.05161. 2025.
13

---

A Detailed derivation of Feedback guidance scheme
Our Feedback Guidance scheme is based the assumption that the learned conditional distribu- tion (x. can be expressed as an additive mixture the true conditional and unconditional distributions, Pr (c)/lc) and Pe (3:e), with mixing coefficient
p ( (pp(( + (1 (x)).
(12)
Our objective is to sample from the true conditional distribution, Assuming a well-learned uncondi- tional model, this can be written as:
Pr (xx))c x po,c(2z/c) - (1 ((11-))a (x:))
(13)
For score-based generative models, the relevant quantity the score function Va a log Pr (x)
Applying the chain rule and using V væp(x) = p(z) V l(z), we obtain:
Va i log (atale) = V log (ax (1 - p
V (mpo,,(asslc)) (c) - (1 x) a
(o, (1 (x2)
po.((x,lc) log (0 - (1 -
log (xx))
(0,1( - -1-10 (x))
(0,e(20e(c)
= V log))(()) + (0,c( (1 (xxe
(V log (o, Va log Pt (ar:))
(14)
Defining the feedback guidance scale A( t) through Eq. 88 a familiar-looking guidance equation is obtained:
V log (= = V (x.,) + X(x,, t) (V log Ve log (xx)))
(15)
This final expression summarizes our approach of feedback guidance. This equation generalizes the typical classifier-free guidance scheme to possess both state- and time-dependent guidance scale A(xe, 1), enabling adaptive control over the denoising process.
B Theoretical shortcomings of CFG resolved by FBG
Despite being standardly used in practice, CFG remains vastly misunderstood Bradley and Nakkiran (2024), , Chidambaram
[2024] The common loose understanding of guidance that to reinforce the conditioning signal of the models the aim is to sample from X-sharpened ditribution, i.e. from p(z|c) = p2z) p(c[z))> From this it typical derive V log p(x|c) and obtain the standard linear combination of conditional and unconditional scores used in all variants of guidance Dhariwal and
Nichol [202T] Ho and Salimans [202T).
What it is often overlooked is that this mixing of the distribution is defined locally, at specific noise level defied by By assuming an equivalent multiplicative mixing at every noise level, as for instance the case when using a constant guidance scale in CFG, the sampled marginals do not correspond to the predefined forward process. In essence the common misunderstanding is that the mixing operation commutes with the noising operation, i.e. that mixing the noise-free distributions and noising them equivalent to first noising the distributions and then only mixing them, which is erroneous. In other words, in the case of multiplicative mixture one can not simply mix the clean distributions, add a subscript everywhere and expect these to sample from the desired marginals. When following the score functions defined using the CFG equation. one is not sampling from the intuitively sharpened data distribution p(z|c) = p(z)p(c(x)> Bradley and Nakkiran 20224) Chidambaram al. (2024).
To sample the y-sharpened conditional distribution in the clean image space using the forward process defined by the diffusion kernel k(x;, æ.) the marginals at timestep t should correspond to:
pe(2:lc)
dxooo (x)l)) 2x))
(16)
((oplo)((ool)))) (occc)poooppuu (zx) 1- > xx))
14

---

However, when sampling using the score function defined by CFG we are implicitly assumiig that the marginals at timestep t keep the mixing property:
Pr.cr((al(c) po..(zz/c) po.(
= ( (zo (a))) mi)) (/ dzroño (azo) (3o)) mi)
(17)
These two expressions are not equal and can fact differ significantly especially at higher noise levels, at which the overlap between Pt e) and (x))) is significant the information of the underlying distributions has been nearly entirely removed, ie. both distributions start to ressemble gaussian distributions. This implies that the trajectories sampled under CFG simply do not correspond to the predefined forward process, especially at high noise levels. The main consequence this is that sampling using CFG can be very misleading.
Mathematically the non-commutabiln is due to the non-commutability of the multiplication and convolution operations. The proposed additive error at the heart of our FBG approach does not suffer from the same flaws. Thanks to the commutability of the addition and the convolution, the mixing and noising operations become inter- changeable. This implies that when using the scores provided by Feedback guidance the predefined distribution is retrieved
of the mixing and noising operations in the case of CFG
Resolving this issue is not the main purpose of this work, which is why we for instance propose com- bining FBG with other guidance scheme such as CFCHo and Salimans [202]] LICKynkäänniemi et al. [2024]. Nonetheless, we believe this to be an insightful discussion worth mentioning and exploring.
C Posterior likelihood estimation
The estimation of the posterior likelihood is central to the proposed Feedback Guidance. Multiple approaches in the literature provide ways estimating such densities Koulischer et al. [2025] Skreta
et al. 22025] Chewi et al. 22255, Karczewski et al. (2025). Seeing the similarities of the present work with the Dynamic Negative Guidance (DNG) approach, we keep the same notation Koulischer
et al. (2025). For an intuitive explanation of the estimation,
refer the reader to the afore mentioned
paper. The continuous limit is then derived in C.T The key hyperparameters introduced in the main body described in more detail in
C.1 Continuous limit
In this section, we derive the continuous-time limit of the posterior approximation used in Feedback Guidance (FBG). To this end, it suffices to consider the posterior update under the guided prediction:
ft-1- = ((x,)) + {(u((x)))) - (ax))) + o1-1jt with e ~ NO, I). This results in:
1
log p(c/px, = logpr(c(xx.) - 20,
(1|x)- - p(x; e) 1| ||:-11 | 11=
log 20,-11t
(11A((0))) + X(xx, (((((a)))) - ml(xa)) + 00-1jc-- - ((aale))l?
- -Il((z)) + A(xx,, (H(()()) ((xx) + -1- ((xx))11
logg:(c((0)) - 20? 20,-1/1
T
(I(A((2e,) 1) - -1)(((())c) - m(xx)) + 1
IA(x,,, (Ma -m(xz)) 01-1)
= logp:(c(20))
T
((1 2A(x,, ())l)((()))))) 1==x))12+ 20,-1116: (μ(2))e) jx(xz)))
20 20,-111
(18)
The term added to the log posterior likelihood can be decomposed into separate contributions. The first corresponds to deterministic measure, evaluating how much the conditional and unconditional
$ At least in the case that an exact posterior is available and that the to-be be-sampled distribution is valid, i.e.
satisties positivity constraints.
15

---

20.0
With clamping
Average reweighting
Pure conditional
50-
Guidance scale
Aodedd term
MA
-2.3
-25
-3.0
-50
-7.5
-75
-10.0
10-1
Normalised posterior pick]]
-100
pia
Diffusion
(a) Guidance function of posterior
(b) Average term added each timestep
Figure 6: In (a) the guidane scale is plotted as a function of the posterior. Here it is important to note that we argue that in practice the negative part of the curve never reached as in a continuous process the crossing of the assymptote would result in an infinite amount of guidance. In practice to avoid this happening upon discretisation a maximal guidance value can be set, shown in the light blue line. In (b) an average of the added term is shown. The red line corresponds to the euclidean difference, whereas the light-blue line includes the reweighting by the transition kernel variance o2-1j
predictions differ []x((x;) (a( )1² while the second is a stochastic term that measures the overlap between this difference and the added stochastic noise. The former measures how much on average the predictions differ, while the latter measures if by chance the stochastic process has favored one over the other.
Using this understanding, a continuous form the equations is recognisable. For this one has to realise that in the continuous case the backward and forward transition kernels become of equal variance, i.e. lima:-so of t-dtlt df Therefore up to first order one can approximate that oz- oip-de =o oi dož /dl- 20,0r. This results in the following continuous integral:
log logp(c)(x:)= = 1/ 40,d,
(2\(x,, 8) 1)) ((x.) f" (p(_))( u(xx)) dW
(19)
The first integral is a path integral, capturing the cumulative deterministic contribution over the diffusion path. The second integral is stochastic Itô integral over the Wiener process dW, capturing how the stochastic noises influence over the posterior likelihood.
C.2 Understanding the hyperparameters for the posterior estimation
The hyperparameters T, and 5 play central role in Feedback Guidance (FBG), but their inter- dependence and abstract nature can hinder accessibility and ease of tuning. To mitigate this, we propose a reparametrization of the temperature and offset parameters, T and 6, in terms of the mixing parameter и and two new hyperparameters--A and I -wwhich correspond to normalized diffusion times (1 = 0 denotes clean data; / = denotes fully noised data).
Understanding the influence these parameters requires analyzing how the posterior, and thereby the guidance scale A(x, t), depends on them.
Effect of mixing parameter
The mixing parameter controls the point at which the guidance scale A(z, t) becomes large. Specifically, À diverges when p(c|æ;) = - Thus, adjusting shifts the asymptotic region of the guidance curve. In Fig. 7a we illustrate this behavior by plotting the posterior values at which
A = 3 for various values of л. As increases, stronger confidence (i.e., lower posterior values) is required to activate a given level of guidance. Intuitively, when क - 1, the conditional model closely approximates the true posterior, and additional guidance is only necessary when uncertainty is high.
16

---

Omse temm (5 55,7731
5.6 Mth 0-
Omhettam (5 5.33
5.5 wth e-
Otbet (5 0.350
5.6 with 03
A: 3mm 5.90
1 0.91
A- 3m 5.9531
3 0.953
Log pootore 1200
Am (m 5.593
cm 0.983
0-
0.5
0.0
0.4 Norma bed ditucion time
1.0
Normaleed dilutton Bine
(a) dependence
(b) n. to dependence
Figure 7: Illustration of the interplay between π and in (a) and to in (b). When specifying to the value of is chosen such that the intersection of the red and blue lines is located tu-
Motivation for Offset parameter
In the early diffusion steps, when noise dominates, the trajectory of the generated sample is best approximated by the conditional model itself, i.e., vt-1 pxrl(e))))). Neglecting stochasticity, the posterior update simplifies to:
log pecl(()) = log (
(1|2
14,0 (1,0 1 - -M,0(2042111
2/141
(20)
< log
||æ,
(1,0(1++1)11
20
20 f|t+1
This leads to monotonically increasing posterior, which artificially inflates model confidence and suppresses guidance activation. To address this, we introduce linear transformation with an offset
lcgp)(c(x,,) = log Pr+ (c/2
T 20
(1I (4.00(24410))) 11,, - -m,0( -
///11
(21)
This correction is especially important in the early diffusion regime, where the signal-to-noiss ratio is low and the posterior estimate is dominated by the offset. Under the EDM scheduler, where of e [0.002, 80), the transition variance of of-11 of oi //?) spans an extremely wide range. exacerbating this effect.
To make this more interpretable, we define 8 in terms of π and new hyperparameter Lo. defined as the timestep which the guidance scale reaches reference value Avet 3 under a purely linear model. This yields:
N Stcpps to) log (1-7)- Ayel Apet 1) 1)
8
(22)
Reparameterising the temperature
We reparametrize using and t1, where represents the timestep which the additive Euclidean term matches the magnitude of the offset 6. To estimate this, we compute the average contribution A of the Euclidean difference across sampled trajectories (see Fig. 6b). For the EDM2- XS model, we find Rs 10 around = 0.5.
This gives:
17

---

Algorithm Foedback Guidance (FBG)
Input:
Pre-trained conditional and unconditional network with prediction (0x()))) and (a) mixing factor it, the two timestep hyperparameters to and t and maximal guidance scale value Amax
Derive d, T from to, t and Pmin from Amax
Set hyperparameters (App. C.2
ær <W I
Initializo state
log [(pp(arr
Initialize posterior and guidance scale
(#T) P(C-T)-(1-m)
plc
for T, T...., do
(e..nnn(as(c) = (a. ) (c( (w))))) 21-1 = 14 (z, (e) -- with z N((,,))
Compute and mix scores DDPM Step Update the log pesterior
logp(c(x(-1) = 2)
2o 2 1-
21 10 (2 || (o)
log logp(c/xx--) max (log(([(2=-1)) log Proin)
Clamp the pesterior Update the guidance scale
Ae (x = 1-plo/a
p(c)a.-1)
end for
20'
T
(23)
While A is not tunable hyperparameter, using improves interpretability: t now corresponds to the point at which guidance begins to decrease, marking the transition to effective conditional denoising. Summary
By reparametrizing the abstract hyperparameters and in terms of normalized diffusion times to and f,. we provide a more intuitive interface for tuning Feedback Guidance. This improves both usability and interpretability, which we consider essential for practical deployment.
D Pseudocode of Feedback Guidance
Our Feedback Guidance procedure is summarized in Algorithm 1 At each denoising step, given noisy state Pt, we compute both the unconditional prediction (2te)) and the conditional prediction po (c) These predictions then combined using guidance scale determined by the previously estimated posterior through eq.(8. The resulting mixture is used to predict the next, less noisy state .4-1- After this step, the posterior updated based on the state, and the process is repeated for a fixed number of iterations until a fully denoised image is produced.
Our code, compatible with the EDM2 repository Karras al. 2024bb a is provided the reviewers as supplementary material.
E Stochastic sampling for Variance Exploding Diffusion Models
is well known the forward process can be freely chosen. Two very standard cases are those of a Variance Preserving (VP) and that of Variance Exploding (VH) forward process. In the VP case, the information is progressively destroyed by both downscaling the fea- tures by factor Va, and adding normal noise with standard deviation VI- i.c. 2 Varaa, + ai- with be described by #2+1 ~ ((#++1/=))) with g( (a2) = =N(a/a,a,, (1 T) Thanks to nice property of the Gaussian function, this Markov chain can be reparameterised Ø4+1 ~ q(*++1 (xo) with (x))+1/=0)) = (( (a (1 - I) and a, = II Oe.. This forward process can then be inverted according to #t-1 ~ p(p(((111 ax)) with p p( (x+) = NN4,, T). In this case, it is well (x, 1-a: Gt) and ã. = 1-8.-1 Br vT-a,
ANO,, This forward transition alternatively
known that = var
1-ā: The case of VE is far less often described using the discrete markov chain framework, which is why we think it wise to derive the precise shape of æ p( (xx)) with p(x;-1 (axz) = =W of I).
In the VE case, the forward process simply consists of adding gaussian noise of increasing scale
18

---

2++1 = +x Vo+ off with e ~ (0 1). Similarly to the VP case, the previous process can
be reparameterise xt+1 ~ a σo) with g( 20) = (ao, o and dip = Σ = O'
To obtain the form of p(zz-1 24) - obtainable thanks to the conditioning zo. One finds:
need to obtain g( zo) which is
( 100)
p( p De. zo) = g( 1,
q( [ xo)
[t 8t-
x
20
2
[x 2o 6
xx exp
+
2 o
o
0
= exp
2 07
+
||
(24)
0 1-1
t-1
2
Pi-1
2
Ct
+
0
xt- + c(2 (0)
at-1 14
exp
20
1/e
Rewriting the clean data points To using the ground truth noise e through Zo 2 - oge,. This is done because this is precisely how our denoising network will be parametrized. Using this, one recognizes:
i- (o, 0
1-1
(25)
0
1t = 84 - (1
0,6t
0
Or alternatively using the score function, i.e. C2 di V
log Pt, we have:
N4 = 24+ (aî )2-1) Vn log pt
(26)
Or equivalently, the very intuitive equation:
Et 1tt
Va log
(27)
0 0
2
F Additional results and ablations
In this uppendix all the additional ablations and obtained results are provided and described in more detail.
F1 Detailed description of class-conditional experiments
First and foremost the hyperparameters of the different guidance schemes that minimize the FID or FDDmov2 are provided in Table [ It should also be noted that CFG++ Chung et al. 2022
was originally not analyzed in the context of class-conditional image generation such we do on Imagenet, explaining the performance observed in Table [ On the other hand, the guidance weight-schedulers introduces by |/aanng al. [2024] were only verified using the FID-metric, much less sensible to late stage guidance than the Dimovy,, explaining the underperformance that regime. For CFG Ho and Salimans [021 sweep over the guidance scale is performed a resolution of = 0.1.
To compare our method with adaptive CFG weight schedulers, we follow Wang et [024] and benchmark against their best-performing variant, the linearly increasing scheduler, which we denote as LinCFG. The reported guidance scale corresponds to the aitectory-averagee value, and we sweep over X with a resolution of 0.1. As expected, the optimal scales of LinCFG strongly correlate with those standard CFG.
For CFG++ Chung et al. 2025 we perform sweep with finer resolution of X = 0.025. Unlike other guidance schemes, CFG++ constrains [0, 1). since it modifies not only the score function prediction but also the coupling between forward and reverse diffusion processes. This design
19

---

Guidance scheme
CFG
LinCFG
CFG++
LIG
FBG pure
FBGLIG
A
A
A
A
max
min
π
Tt
Ot
X
Stoch. (FID)
1.4
1.5
/
2.8
1.6
0.15
0.999
1.10
0.56
1.4
4.64
PFODE (FID)
1.4
1.5
0.35
2.2
2.9
0.41
0.999
1.61
0.60
2.6
2.7
Stoch. (FDDin)V2)
2.1
2.1
2.9
6.8
0.48
0.999
4.07
1.29
2.6
2.34
PFODE (FDDinoV2)
2.3
2.2
0.6
2.8
16.6
0.80
0.999
6.46
1.17
1.6
1.61
Table 3: Optimal hyperparameters for different sampling approaches. To facilitate the comparison
between the schemes we follow the nomenclature introduced of the EDM framework Karras et al.] [2022] and refer to the noise levels Cta Ot, instead of normalized diffusion times to and i for FBG. For FBGra the unspecified parameters (T,, lo, max and min)) are left unaltered from the separately optimised methods.
precludes its use with purely stochastic DDPM samplers, which do not rely on forward noising during denoising. In essence, CFG++ is conceptually distinct from conventional guidance schemes and could, in principle, be combined with methods such as LIG or other weight schedulers. We include this benchmark to provide a more comprehensive comparison with alternative approaches proposed in the literature.
For LIG a joint sweep over the guidance scale and the starting point of guidance is done. The sweep over the guidance scale is performed at a resolution of À 0.25 and the starting point max is chosen at the discretised step values. The influence of the end point of guidance is not analysed, instead a low value of min = 0.28 is chosen. As higlighted in the work in which the method is introduced, increasing min leaves the FID unaltered and is simply beneficial from a computational point of view Kynkäänniemi et al. (2024).
For FBG a joint sweep over to and to f is performed defined as the normalised diffusion times corresponding with the discrete step values. For instance in the case of stochastic sampling where 64 sampling steps are used we perform a sweep at a resolution of to = 1/64 ~ 0.0156. We prefer to define 1 as function of to. their difference gives an estimate for how large the guidance interval is. This is then repeated for three values of π = 0.999, 0.9999, 0.99999. These values were chosen after a preliminary finetuning by hand. We find that although the guidance profiles do differ slightly between different choices of π, mainly being sharper as π increases, the optimal FID/FDDimov remain very similar. Due to this we choose to focus our limited resources value of π 0.9999. To illustrate the weak dependence of FBG on values of π a sweep is performed using the optmal values for to and & given in table The results are shown in Fig. 9 where it can be seen that both FID and FDDinoV2 values remain fairly constant over a wide range of r-values.
proper ablation
fixed
It should also be noted that at some point in the research process we tried adding late start to the offset parameter, i.e. to set 8 = 0 for t ofturt- and slightly modify the way we parameterise 6 as function of to such that its interpretation remains true. This however did not significantly modify the performance of the approach, so this research track was dropped to avoid any unnecessary convolutions. For all methods we find that the FDDpiovz-optima hyperparameter values result in a much higher amount of guidance than the FID--optmmzed values, hinting that the metrics are not sensitive to the same features [Stein et al. [2023). This fact highlights that only providing one of the two metrics when benchmarking a new approach might not provide the full story.
Figure is the corresponding figure Fig. 3 in the main body, but for the FID-optimised stochastic sampling, rather than for FDDinov2-
F.2 Detailed description of T2I experiments
First and foremost the hyperparameters of the three guidance schemes that minimize the FID or DDDmmv2 are provided in Table 4
For CFG Ho and Salimans 2021 sweep over the guidance scale is performed at resolution of A= 0.25.
For LIG Kynkäänniemi et al. 20244, we follow the recommendations of Kynkäänniemi et al. [2024) and perform a joint sweep over the guidance scale and Tmax at resolution of À = 0.25. Thereafter, Jmin is optimized. As expected, we find that the FD optimal interval is much larger and earlier. For FBG, we perform a sweep over lo and to - 4 at resolution of 0.05. We then perform a sweep
over on logarithmic scale, similar to that used in Fig. 9
20

---

FID-Optimal:
CFG
LIG
FBG
CFG: 5.01
0.9
0.08
Precision (Quality
High guidance
0.064
FID
0.8
Low guidance
0.048
3.7:
0.397
0.413
0.429 0.444
0.46
LIG: 3.59
0.6
0.7
0.8
Recall (Diversity
(a) FID minimum
(b) Precision and Recall
Figure 8: Grid search over the two hyperparameters to and t1. The FID is shown calibrated within the best performing value using LIG and CFG.
CFG (Stoch.)
5
FBG (Stoch.)
CFG (Stoch.)
100
FBG (Stoch.)
CFG (PFODE)
CFG (PFODE)
x
FBG (PFODE)
FD-DinoV2
FBG (PFODE)
4
FID
90
2
10
10
10
10
10-2
10-3
10-4
10
1 1 n
1 1-1
(a) Optimised FID
(b) Optimised FD
Figure 9: Illustration of the weak dependence of FBG when parametrised using to and f1. The sweep over ı performed at the optimal values for to and t given to in Table 3
Guidance scheme
CFG
LinCFG
LIG
FBG
pure (ours)
FBGCrG (ours)
X
X
Omax
Gmin
π
lg
4
X
to
/
Stoch. (FID)
2.25
3.25
4.0
2.4
0.08
0.9
0.55
0.4
1.0
0.55
0.4
Stoch. (FDpinev2)
2.5
3.0
4.0
3.94
0
0.9
0.65
0.45
2.0
0.5
0.375
Table 4: Optimal hyperparameters when optimizing for FID and FDpinoV2 on MS-COCO using Stable DIffusion 2. For FBGcrG. the hyperparameter is left unaltered from FBG We also note that in the context of FID optimisation, FBGçre reduces to FBGpure- that that additional classifier guidance on top of FBGpree is harmful, which not the case when optimizing using the FD
21

---

Azerage
Guidence scole
Diftusion time STEPS
Figure 10: Comparing the guidance scale predicted by FBG and that of LIG in the FDDinov2 optimized setting. Despite possessing similarities the two seem to operate in different regimes.
We would also here like to emphasize that we believe the MS-COCO benchmark to be suboptimal when comparing different guidance schemes. This because the prompts of MS-COCO Lin et al. 2014] are quite uniform and fairly standard, which preciscly corresponds settings when guidance is not needed as much. This explains why the optimized guidance scales are much smaller than the ones typically used for sampling complex prompts, such as the ones used in handerafted dataset as shown in Fig. 4 This benchmark however remains the standard used in the literature, which is why choose to report it here.
F,3 Combined guidance schemes
The proposed Feedback Guidance scheme can be easily combined with other preexisting guidance Limited Interval Guidance
schemes such Classifier-Free Guidance Ho and Salimans 2022 I
Kynkäänniemi et al. 2024].
In the context of Text- Toxt-To-Image low frequency features of an image, such as sharp colors, without drastically harming the diversity. In the context of Imagenet generation using EDM2-XS, FBG with LIG, is optimal. Preliminary results indicate that joint methods easily outperform their parts. That such results are obtained in this context should not surprise the reader, both method despite having some similarities, behave very differently as illustrated in Fig. TO To simplify hyperparameter tuning of FBGLIG, we suggest use the optimal time interval parameters of LIG with slightly less guidance and to slightly reduce t to for FBGpure- In essence, both of these subtle changes are responsible for less guidance the respective schemes, which makes sense as the are later on combined.
observe that adding base level of CFG can help to retrieve the
find that using FBGLIG which combines
The optimal values for the sweep over λLιG and t are given Table The other parameters are chosen the same as the separately optimized methods. We do not exclude the possibility that full grid-search over the entire joint hyperparameter space might yield better trade-offs between the two guidance schemes.
An advantage of the error assumption model proposed is that it allows for very flexible view of guidance. For instance, for the FBGCFG approach in fact simply coresponds to assuming that the true conditional distribution can be rewritten as:
(x(l/c)) = (pr,o( - ((1-x)po
(28)
Pr, (mx
Pt,s
This implies that the learned distribution satisfies the following algebraic equation:
Pro(xx - 111---111kmpp00pp0 -)p = (c) (xx))~1
(29)
Similarly, FBGLG assumes the same form of error with as only distinction that À becomes a time dependent function that is equal one outside the guidance interval specified by Jwin and maax-
22

---

E4
Illustrative samples (EDM2-XS)
(a) FID-optimal CFG
(b) DDumovr-optine CFG
(c) FID-optimal LIG
(d) FD FDpuovz-optimal LIG
(c) FID-optimal FBG (ours)
FDouevz-optimal FBG (ours)
Figure 11: Grids containing samples generated using the different guidance schemes CFG, LIG and FBG (ours). Results displayed under the same seed and when he perraaramettess are optimised for both FID and FDpinov--ppimml performance.
23

---

F,5 Illustrative samples (T2I)
To facilitate comparison of our newly introduced FBGpure and FBGcrG schemes with CFG, we provide illustrative samples. Prompts are drawn from our curated dataset spanning four difficulty levels (memorized basic < intermediate hard; see Appendix [G.. For each level, two prompts are randomly selected, and we display four samples generated with: (i) the conditional model Rombach al. 2022 (ii) CFG with guidance scale 3.5 Ho and Salimans (02T) (iii) FBGpure with (IT 0.9, to 0.75, f1 0.5), and (iv) FBGCFG with an additional fixed scale 1.5.
3.51
FIG CER
CR 25)
(a) Memorized: "The great wave by Hokusai"
Memorized: "The Taj Mahal"
(c) Easy: "A sushi platter"
(d) Easy: "A pinecone resting on a forest floor"
Figure 12: Different samples for randomly selected prompts of memorized and easy prompts. The used prompts are written underneath the images.
For memorized/easy prompts, visible in Figure 12 CFG images often exhibit oversaturated colors and overly smooth textures, whereas FBG variants largely avoid these artifacts by deactivating guidance when the conditional prediction is already accurate.
24

---

Contitoraiic mice
CPe 2t 3.1
HG pure
HG CHi1A-15 LS
Candii meds
dRd Do D
FBG zur
FRG c 1.0
(a) Intermediate: "A fox wearing reading glasses sitting under with open book"
(b) Intermediate: "A dense jungle with oversized
insects and ruins covered
(c) Hard:
croissant slowly unrolling itself into a spiral staircase, with tiny chefs walking each layer, in detailed isometric art style"
(d) Hard: "A city built inside giant canyon where each layer of rock houses a different civilization, all lit by bioluminescent flora"
Different samples for randomly selected prompts of intermediate and hard prompts. The used prompts written underneath the images.
Figure 13:
Harder prompts show the strength of FBG, e.g., capturing bioluminescent lights in Fig. [3d as its dynamic scale allocates stronger guidance only when needed. Such challenging prompts are far more informative for comparing guidance schemes than simpler MS-COCO captions [in et al. [2014). It should be however noted that both schemes struggle with following all the details present in the prompts.
Next, we present qualitative comparisons between images generated with CFG and FBGCFG, along with the corresponding dynamic guidance scales across different prompt difficulty levels. As discussed in the main paper, the guidance scale in FBG increases with prompt complexity.
For memorized prompts (e.g., Figures [4 and [5), the dynamic guidance scale remains near one, preserving sample diversity. In contrast, CFG tends to overemphasize prompt-specife details, leading to oversaturated features such as excessively bright colors.
For challenging prompts (e.g., Figures 18 and 19). FBG applies much higher guidance scale, successfully enforcing the generation key prompt-specific features, such as the phoenix in Figure
19 that underrepresented in CFG outputs.
25

---

In both cases, the fixed guidance scale used by CFG is suboptimal: it is too strong for memorized prompts and too weak for difficult ones. These results reinforee our central claim that guidance should not rely on global, fixed scale, but instead adapt dynamically, activating only when needed enhance fidelity, while remaining inactive to preserve diversity when the conditional model already performs well.
Guid ance scale
Diffusion time
CFG 3.5)
(b) FBGCrG (A 1.5)
(c) Guidance scales FBGcrG
Figure 14: Different samples for the memorized prompt: "Girl with pearl by Vermeer"
Guidance scale
Diffusion
(a) CFG (A 3.5)
(b) FBGGrG (A = 1.5)
(c) Guidance scales FBGcrG
Figure 15: Different samples for the memorized prompt: "The Eiffel Tower in Paris"
Guidance scale
Diffusion time
(a) CFG (A = 3.5)
(b) FBGGRG (A 1.5)
(c) Guidance scales FBGGNG
Figure 16: Different samples for intermediate prompt: "A jumping through a hoop made of fire"
26

---

Guidance scale
Diffusion time
(a) CFG (A = 3.5)
(b) FBGCRG (A 1.5)
(c) Guidance scales FBGCEG
Figure 17: Different samples for the intermediate prompt: "A parrot with steampunk goggles flying
through a thunderstorm above 19th-century shipwreck, in dramatic oil painting style"
Guidance scale
Diffusion time
(a) CFG (A = 3.5)
(b) FBG (A 1.5)
(c) Guidance seales FBG
Figure 18: Different samples for the intermeiate prompt: "A volcanic landscape with rivers of lava flowing under a starry sky"
Guidance scale
Diffusion time
CFG 3.5)
(b) FBG (A 1.5)
(c) Guidance scales FBGera
Figure 19: Different samples for the intermeiate prompt: "A floating island chained to the earth by golden vines, casting a shadow shaped like phoenix over the ocean below" "
27

---

Difficulty
Category
Prompt
Memorized
Artwork
The starry night by Van Gogh
Memorized
Brand
The Nike brand logo
Memorized
Location
A photo of the Taj Mahal in India
Basic
Animal
An orange cat sitting on couch
Basic
Food
A pizza slice on white plate
Basic
Nature
A bird nest with three blue eggs
Intermediate Intermediate
Animal Food
A dolphin jumping through a hoop made of fire
A floating sushi platter arranged in the shape of a koi fish, hovering over pond
Intermediate
Nature
A desert landscape with an abandoned train half-buried in the sand
Very hard
Animal
A parrot with steampunk goggles flying through
a thunderstorm above 19th-century shipwreck, in dramatic oil painting style
Very hard
Food
A glass teapot filled with herbal tea where each herb leaf is shaped like a different mythical creature, photographed on white marble city built inside a giant canyon where
Very hard
Nature
each layer of rock houses different civilization,
all lit by bioluminescent flora
Table 5: Example prompts from the dataset, sorted by difficulty and category
G
Prompt dataset
To analyse the sensitivity of our dynamic guidance scale to the complexity of the given prompts, a small scale prompt dataset is introduced. It contains 60 prompts of 4 difficulty levels: memorized, basic, intermediate and hard. Each complexity level is divided into 3 categories/topics containing 5 prompts each.
For the memorized prompts we use: well known artworks, brands and locations. For the other three we use: animal, food and nature images.
The prompts themselves are generated using ChatGPT and further minimally modified to make the prompt easier verify. The main difference between basic and intermediate prompts is that the latter contain highly unlikely combinations (such as "A giraffe playing basketball on rollerskates") that the model has most likely not seen (or only rarely) as such the training data. The main difference between intermediate and hard prompts is mainly the amount of details contained in the prompt. The more details such as colours, numbers or different elements are added, the more unlikely becomes that the conditional model will be to satisfy all prerequisites on its own.
The dataset is available in the official repository at this link Examples of each difficulty level and each category are given in Table 5
H Used resources and LLM use
For the stochastic sampler all experiments are run using on a NVIDIA Tesla V100-SXM3-32GB GPU. Generating 50k images in batches of 64 using the EDM2-XS model Karras et al. [024bba as required for valid FID benchmark Heusel et al. [2017] Stein et al. 2023 takes 7h30 on such node. For the 2nd -order Heun sampler of the PFODE Karras al. [002 2024b a NVIDIA GeForce
RTX 4090 GPU is used. Generating 50k images in batches of 64 using the EDM2-XS model Karras
et al. 2024b a), as required for valid FID benchmark Heusel al. 201 7. Stein et al. 2023 J. takes
8h on such a node. For the T2I results using Stable diffusion Rombach et al. [022]] we rely on an
NVIDIA Tesla V100-SXM3-32GB GPU. The experiments performed only require the generation of
3k images per hyperparameter setting, which takes around 2h on such node.
28

---

During the writing of this document, publicly available LLMs of different sources were used to rewrite, or polish, existing text. Typically, first draft was written by one of the authors and then polished after comments from the others, thereafter the text was, in some cases, condensed or slightly modified, on a paragraph level. on paragraph level, our original ideas as well as the intended flow of the paper remains closest to ours. The propositions of the LLMs were only accepted when in full alignment with the handwritten text, and were otherwise discarded.
our belief that by solely correcting the document with an L.MM
29

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: As discussed the work, our approach the first theoretically grounded state- and time-dependent guidance scale. The obtained trajectory-specinn guidance scale exhibits behavior similar the ad-hoc proposed Limited Interval Guidance. When compared CFG and LIG using generated samples and robust metrics, we find that our approach outperforms CFG and performs on par with the state-of-the-art LIG method. FBG, with its state- and time-dependent guidance scale that accounts for how well the model denoises the current state, is not only highly novel but also promising.
our firm belief that
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: The paper contains a limitation section which provides fair and honest evaluation of the proposed work.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: The theoretical introduction of the additive error model and the resulting Feedback Guidance scheme is central to this work and therefore derived in detail.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: A pseudocode of Feedback Guidance is provided in Appendix D. The introduced hyperparameters are discussed in detail at the end of section 3.4 and fur- ther explained Appendix C.2. The code is available at: battps://qitaee com/ Fel xKoul scher wwwccerFFeeebaaeeeeeeeeeeeeeeeae Fee/ebeeebbaeeeee Gu www.cca---iinnneeeeeeeeeeeeeeeee dance - Dif-fiffiioono Modell
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification; The method is thoroughly described and should be reproducible from the docu- wwm/
ment. The code is further provided at: Model
Feedback- dence
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: As afore mentioned, the introduced hyperparameters are discussed in detail the end of section 3.4 and further explained Appendix C.2. opti- mised values are clearly given in table 3 and 4. The code and the small scale prompt dataset are available at:
http:///...b....o..ooooe/o ixoou scher/ Models
Feedback of
30

---

7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Due to the expensiveness of the FID/FDDimvv metrics requiring 50k images per data point, only single value is reported. As all approaches evaluated the same seeds, this should however ot affect the relative performance of the different methods.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
This information is disclosed in Appendix H
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of ott.iiggttggee///errrrr...o/oppo
Answer: [Yes]
Justification: The proposed work conforms in every respect with the NeurlPS Code of Ethics.
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [NA]
The paper presents theoretical contribution to diffusion guidance techniques, focusing on foundational aspects without proposing a concrete application or system for deployment. AS such, the work does raise immediate concerns related malicious use, fairnsss, privacy, security. While future practical applications of improved diffusion methods may have societal implications, our contribution is abstract and methodological, without direct path to societal impact at this stage.
Justification:
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (c.g., pretrained language models, image generators, scraped datasets)?
Answer: [NA]
This work purely theoretical and does not involve the release of models, datasets, or other assets that carry a high risk for misuse. As such, no safeguards are necessary applicable.
Justification:
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All models, datasets and code that are used are properly referenced in the main document.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
31

---

Justification:
The code as well as the small scale
prompt
dataset
are both available at: https://github
con/Felxxxouiischer Monn-
Gue of F fusi
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: No human evaluations were performed.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: No human evaluations were performed.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMSs if it is important, original, non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification:No LLMs were used as core methodological component this work. Any use of LLMs was limited to standard writing assistance and did not affect the scientific content the paper.
32


Reviews:
[{"review_id": "blrqwvIeup", "reviewer": "Reviewer_UFxC", "summary": "This paper proposes a state-dependent guidance scale method, which can automatically adjust the current guidance scale according to the posterior distribution after sampling to obtain the optimal value. Experiments on class-to-image (ImageNet 512x512) and text-to-video tasks have demonstrated that this method achieves significantly better results than the conventional CFG (Classifier-Free Guidance).", "questions": "1. Please add more experiment on text-to-video generation, DiT-XL/2 on ImageNet 256x256, as the proposed method is training-free and data-free, conducting on video model is accepetable.\n2. Impact of Prompt. In text-to-video, what is the result if prompt is missing, wrongly provided, or more detailed provided.\n3. More Analysis of more iterations of mix scores + take step + guidance scale.\n4. More experiments of few-step generation when the number of total denoising steps are decreased.\n5. Add more evaluation metrics such as HPS v2, aesthetic score, clip score.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths​\n1. Quality: The training-free, data-free, and online method boosts generation quality based on CFG with simple, reproducible code. Extra forward pass costs are acceptable for quality gains.​ The experiment includes both FID and FID-dino on sampled about 10k images, which is reliable.\n2. Clarity: Methodology and implementation are clearly presented.​\n3. Significance: Offers an efficient way to improve generative model performance without extra training/data.​\n4. Originality: Automatically adjusting guidance scale via posterior distribution is a novel approach.​\n\nWeaknesses​\n1. Experiments only on EDM-XS due to limited computing power fail to match the goal of achieving optimal CFG, raising doubts about the method's necessity for accurate models.​ Small-scale experiments undermine the method's significance; testing on DiT-XL/2, SiT is required.​\n2. Only evaluating with FID and FID-dino is not enough. If the proposed guidance scale method could work, it should also improve human preference.", "was_revised": true, "final_justification": "The authors initially conducted experiments only on small models. The rebuttal results on large models such as EDM2-L demonstrate that all guidance approaches are more similar to each other.​\nAs a training-free general CFG method, it is feasible to conduct quick evaluations on video models to validate its generalization. However, the authors failed to address this issue.​\nConsidering the lack of results on HPSv2, limited improvements in metrics such as FID and CLIP score, along with the introduced complex pipeline (mix scores + take step + update guidance), I can only retain my score and judge it to be between 3 and 4.", "rebuttal": "We would first like to thank the reviewer for their detailed analysis, which recognizes the quality, originality and significance of our work. In particular, we appreciate the reviewer’s acknowledgments of our efforts to provide clear and usable code, as well as realizing that our proposed scheme comes with negligible computational overhead when compared to CFG.\n## Response to Weakness #1: limited experiments\nWe first address the reviewer's concern that the small-scale experiments may not be sufficient for assessing the method's significance. We agree that strengthening the paper in this regard would significantly improve its quality and impact.\n### Extension from small EDM2-XS to larger EDM2-L  \nTo address this, during the rebuttal period we conducted the same analysis previously performed with the EDM2-XS model using the much larger EDM2-L model, providing more comprehensive and qualitative insights. The initial results, computed on 10k generated images, are presented in Table 1 below. The extension towards 50k generated images, as well as the addition of the Precision and Recall metrics, is under way but will not be ready in time for the initial rebuttal.  \nWe can observe that the results on EDM2-L are more similar for the different methods, compared to the small model. This is because the EDM2-L model is far more qualitative and requires much less guidance, a result expected from our interpretation of guidance as an error correcting scheme. As a consequence, all guidance approaches are more similar to each other.\n\n**Table 1: Performance comparison on EDM2-L (10k images)**\n\n| Method | FID | FD$_{\\text{DinoV2}}$  \n|--------|---------|----------------------|  \n| CFG | 8.15 | 119.70 | \n| LIG | 7.78 | **112.68** | \n| FBG |**7.75** | 114.90 |\n\n### Additional T2I results with Stable Diffusion 2 on MS-COCO 30k  \nSimilarly, to further support our findings in the T2I setting, we evaluated the FID, FD$_{\\text{DinoV2}}$ alongside the average CLIP and Aesthetic Scores for different guidance schemes (CFG, LIG, and our proposed FBG) using Stable Diffusion 2 on the standard MS-COCO 30k dataset. These results, currently limited to a subset of 3k prompts, are reported in Table 2. Results on all 30k prompts will become available next week.  \nThe MS-COCO30k dataset contains prompts that originate from captioned images, which all remain fairly standard. We expect our approach to work even better in context where the prompts are much more varied. To remain consistent with the literature we however still chose to evaluate on this standard benchmark, on which FBG still slightly outperforms alternative approaches.\n\n**Table 2: Quantitative evaluation of T2I on MS-COCO benchmark (3k images)**\n\n| Method | FID | FD$_{\\text{DinoV2}}$ | CLIP-Score| Aesthetic Score|  \n|--------|---------|----------------------|-----------|--------|  \n| CFG | 19.64 | 53.55 | 0.310 | 5.650 |  \n| LIG | 18.81 | 53.55$^*$| 0.311 | 5.742 |  \n| FBG |**18.62** | **52.93** | 0.311 |**5.753**|\n\n$^*$In this setting, the best LIG interval corresponded to applying guidance on the whole trajectory.\n\nWe hope that these results help convince the reviewers of the more general validity of the method, as well as of our commitment to incorporate their valid critique to improve the current submission.\n## Response to Weakness #2: no human evaluation\nWe fully agree with the reviewer that in terms of quality metrics, the end goal remains human evaluation. Unfortunately, in the short amount of time available during the rebuttal, performing such an analysis remained unfeasible. Instead, we rely on common practice in the field, which has demonstrated the strong correlation between human preferences and FD metrics, more particularly the FD$_{\\text{DinoV2}}$ [1]. In the context of T2I we believe that the a much wider ranger of metrics is preferable, which is why we chose to also provide both CLIP-Score and Aesthetic Scores. We hope that these additional metrics help convince the reviewer of the validity of the proposed evaluation approach.\n\n[1] Stein et al., “Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models”  \n## Response to Question 1: experiments on text-to-video?\nRegarding the analysis of larger models, we refer the reviewer to Response to Weakness #1. Unfortunately, due to the short time available during the rebuttal phase, testing in the context of T2V, or on DiT, was not feasible with the computational resources available. We hope that our evaluation setting of choice (in line with the tradition of image benchmarks and metrics, to validate research on diffusion models) remains acceptable to the reviewer.\n## Response to Question 2: on underspecified prompts\nAs no experiments have been performed in the context of T2V, we can not comment on this specific setting. However, the questions remains interesting and insightful in the context of T2I. Our dynamic guidance method adjusts according to the guidance needed for each prompt. This does not mean that less detailed (“easier”) prompts are less well followed; on the contrary, their lack of specificity allows for more diverse secondary features. The key point is that not all prompts require the same level of guidance, and low guidance does not necessarily imply low quality. In this regard FBG provides a key edge over established methods such as CFG which always applies the same amount of guidance regardless of the amount of detail and/or complexity of the prompt. This is well highlighted by the results shown in Fig. 4 of the paper.\n\n## Response to Questions 3-4: on iterations of: mix scores + take step + update guidance\n\nAn important note is that the graph displayed in Fig. 2 is unrolled in time, implying that applying more iterations corresponds to a change in the total number of steps used in the diffusion process. Theoretically, the more steps are used the more accurate the posterior approximation becomes. To avoid any future confusion for readers we have decided to add the following sentence after line 204: \n>“This control diagram is progressively unrolled during the denoising process, implying a succession of computing the score functions, mixing them, applying a denoising step, updating the value of the guidance scale and repeating a fixed amount of times until a fully denoised image is obtained.”\n\nAs our main focus lied on a theoretically grounded method, we preferred testing the use of different solvers (2nd order Heun and stochastic) which operate in fundamentally distinct ways. The analysis of varying the number of sampling steps, such as analyzing the case of few-step generation which we considered as non-central to our work, was left as future work.\n\n## Response to Question 5: add prompt alignment metrics\n\nTo beter evaluate the quality of the proposed approach we evaluated the T2I results using a wider range of metrics, including the FID,FD$\\_{\\text{DinoV2}}$ , CLIP-Score and the Aesthetic Score. We strongly agree with the reviewer that in the case of T2I with the absence of human evaluation, using at least a variety of metrics is key. However, we would like to point out that in the context of Imagenet, which contains almost exclusively well-centered, well-lit images, these metrics can be seen as slightly less relevant than the provided FD$_{\\text{DinoV2}}$ and the Precision and Recall curves, which also measure the quality-diversity trade-off of the method [1]. We hope that the reviewer is satisfied by the additional metrics, on which FBG also outperforms concurrent approaches.\n\n## Summary  \nWe appreciate the reviewer's concerns and questions, and are convinced our responses as well as the additional experimental results have the potential to improve the manuscript considerably. We hope the reviewer reassesses the manuscript in light of this rebuttal, to see whether it warrants increasing their original score."}, {"review_id": "ATbhZTZupM", "reviewer": "Reviewer_qaVs", "summary": "This paper addresses the limitation of classifier-free guidance (CFG) where its use of constant guidance scale throughout the process can reduce generation diversity. This paper proposes a novel feedback guidance (FBG) method that can adaptably determine the guidance scale throughout the denoising process, based on the feedback of a quality estimation of its current predictions. This makes FBG state-dependent (e.g., closer to conditional guidance leads to lower scale, while closer to unconditional guidance leads to higher scale; higher scale for complex prompts, while lower for easier prompts). FBG is also time-dependent, where it is only active in the middle of the denoising process, due to the limitation of injecting CFG in the early and late stages. Experiments are conducted on ImageNet using EDM2-XS model, and results have demonstrate superior performance over CFG and comparable performance with LIG.", "questions": "Please see the previous strengths and weaknesses section. Addressing the four aspects mentioned in the weaknesses section can effectively address my concerns.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**:\n\n1. The task is well-motivated. Considering the important role guidance plays in diffusion models, improving on the standard CFG can provide wide benefits to the field.\n2. The presentation is overall clear, apart from some potential small improvements. This makes the paper easy to follow. Especially, the visualizations are helpful for delivering the insights and demonstrating the rationale.\n3. The proposed method is novel and effective, surpassing CFG, while achieving comparable results with LIG.\n4. The proposed method is also highly adaptable, resulting in using state- and time-dependent guidance scales. Another practical benefit is its combinable with other guidance schemes.\n5. Finally, a notable advantage of this work is its solid theoretical underpinnings.\n\n**Weaknesses**:\n\n1. Due to the design of the proposed FBG, it necessitates the computation of the “feedback” to determine the guidance scale. This will bring additional computational costs in addition to the original inference process. \n2. It’d better of Figure 1 can include labels or annotations of each line in the plot or as more detailed descriptions in the caption to improve on its clarity. Similarly, in Table 1, it is suggested to explain in the caption what underline and bold mean when they are used for highlighting the numbers.\n3. FBG’s performance does not always surpass LIG. However, this is not a major concern as the performance is comparable and FBG benefits from its solid theoretical backgrounds.\n4. Would the proposed method work if using DiT as the diffusion backbone? It would be interesting to see more experimental results using a variety of settings such as whether the proposed FBG is effective in larger architectures, in one-step generative models, etc. The inclusion of these and the proof of its scalability would make this work more influential.", "was_revised": true, "final_justification": "The rebuttal has effectively addressed my concerns by providing additional clarifications and results; combined with the advantages listed in the strengths section, I would maintain my support of this paper's acceptance.", "rebuttal": "We would first like the reviewer for the in depth reading of our manuscript. In particular, we are glad that they appreciate the novelty of our novel state- and time-dependent guidance approach as well as its solid theoretical underpinnings.\n\n## Response to Weakness #1: Computational overhead\n\nOne of the key advantages of the iterative posterior estimation approach is that it comes at negligible additional cost as it solely relies on precomputed quantities, namely the conditional and unconditional score functions already required for the denoising itself. To further emphasize this point we have added a sentence in the main document in section 3.3 mentioning: \n> *“A crucial advantage of computing the posterior using the scheme describe above is that it causes negligible computational overhead as all required quantities, in particular $\\mathbf{\\mu}_c(\\mathbf{x}_t|c)$ and $\\mathbf{\\mu}(\\mathbf{x}_t)$, are already computed.”*\n\n## Response to Weakness #2: Description of Fig. 1/Table 1\n\nWe thank the reviewer for this valuable advice. We have included descriptive labels for the three lines present in Fig 1. which describe the three trajectories using the adjectives: “good”, “okay” and “poor”. Their meaning in the context of the generative trajectories is then further explained in the caption:\n>*\"Figure1: Illustrative diffusion trajectories and their hypothetical guidance scales in a 1D setting. Trajectories located further from the mode close to the decision window, such as the trajectories denoted by \"okay\" and \"poor\", receive higher guidance. On the contrary trajectories that are clearly going to the right mode, such as that denoted by \"good\", receive negligible guidance\"*\n\n Regarding Table 1, the caption has been modified to explain the use of bold and underlined:\n > *\"We underline the best performing method per sampler, and further emphasize the best overall method using bold values.\"*\n\n## Response to Weakness #3: FBG compared to LIG\n\nWe acknowledge that when measured using the FID, FBG does not outperform LIG, as mentioned in the abstract and recognized by the reviewer. Indeed, we solely claim performance comparable to LIG with as main advantage a robust theoretical underpinning.\n\n## Response to Weakness #4: Testing FBG on larger architectures  \nFinally we wish to address the reviewer's concern that the small-scale experiments may not be sufficient for assessing the method's significance. We agree that strengthening the paper in this regard would significantly improve its quality and impact. It should also be noted that the described theoretical framework is certainly not limited to a specific architecture and that FBG is fully compatible with models such as DiT. We believe this versatility, gained by rigorous mathematical foundations, to be one of the main strengths of our paper.\n\n### Extension from small EDM2-XS to larger EDM2-L  \nTo address this, during the rebuttal period we conducted the same analysis previously performed with the EDM2-XS model using the much larger EDM2-L model, providing more comprehensive and qualitative insights. The initial results, computed on 10k generated images, are presented in Table 1 below. The extension towards 50k generated images, as well as the addition of the Precision and Recall metrics, is under way but will not be ready in time for the initial rebuttal.  \nWe can observe that the results on EDM2-L are more similar for the different methods, compared to the small model. This is because the EDM2-L model is far more qualitative and requires much less guidance, a result expected from our interpretation of guidance as an error correcting scheme. As a consequence, all guidance approaches are more similar to each other.\n\n**Table 1: Performance comparison on EDM2-L (10k images)**\n\n| Method | FID | FD$_{\\text{DinoV2}}$  \n|--------|---------|----------------------|  \n| CFG | 8.15 | 119.70 | \n| LIG | 7.78 | **112.68** | \n| FBG |**7.75** | 114.90 |\n\n### Additional T2I results with Stable Diffusion 2 on MS-COCO 30k  \nSimilarly, to further support our findings in the T2I setting, we evaluated the FID, FD$_{\\text{DinoV2}}$ alongside the average CLIP and Aesthetic Scores for different guidance schemes (CFG, LIG, and our proposed FBG) using Stable Diffusion 2 on the standard MS-COCO 30k dataset. These results, currently limited to a subset of 3k prompts, are reported in Table 2. Results on all 30k prompts will become available next week.  \nThe MS-COCO30k dataset contains prompts that originate from captioned images, which all remain fairly standard. We expect our approach to work even better in context where the prompts are much more varied. To remain consistent with the literature we however still chose to evaluate on this standard benchmark, on which FBG still slightly outperforms alternative approaches.\n\n**Table 2: Quantitative evaluation of T2I on MS-COCO benchmark (3k images)**\n| Method | FID | FD$_{\\text{DinoV2}}$ | CLIP-Score| Aesthetic Score|  \n|--------|---------|----------------------|-----------|--------|  \n| CFG | 19.64 | 53.55 | 0.310 | 5.650 |  \n| LIG | 18.81 | 53.55$^*$| 0.311 | 5.742 |  \n| FBG |**18.62** | **52.93** | 0.311 |**5.753**|\n\n$^*$In this setting, the best LIG interval corresponded to applying guidance on the whole trajectory.\n\nWe hope that these results help convince the reviewers of the more general validity of the method, as well as of our commitment to incorporate their valid critique to improve the current submission.\n\n## Summary  \nWe appreciate the reviewer's concerns and questions, and are convinced our responses as well as the additional experimental results have the potential to improve the manuscript considerably. We thank the reviewer for reassessing the manuscript in light of this rebuttal."}, {"review_id": "HYGD270Kun", "reviewer": "Reviewer_o9ey", "summary": "This paper introduces a diffusion guidance approach that adaptively adjusts the guidance strength $\\lambda$ during sampling. The core idea lies in assuming an additive factorization of the learned conditional into true conditional and unconditional distributions. The guidance scale can then be derived by letting the model to estimate the posterior $\\log p(c|\\mathbf{x}_t)$. The method has been evaluated on class-conditioned and text-conditioned image generation, demonstrating enhanced performance in terms of quality-diversity tradeoff compared with CFG and LIG.", "questions": "Q1. It is still not clear that empirically how to select $t_0$ and $t_1$. More specifically, $t_0$ and $t_1$ should also be dynamically adjusted for different prompts instead of simply tuned as hyperparameters which potentially leads to suboptimal performance.\n\nQ2. How does the proposed approach perform on diffusion models with larger scales? Can it be adopted for flow matching models?\n\nQ3. Can the authors provide more comprehensive comparisons on text-to-image task? This is a critical venue to showcase the advantage of the proposed approach, given that CFG is commonly adopted as an effective and easy-to-use guidance approach.\n\nQ4. It is unclear how $\\lambda$ in Eq. 8 is derived. Could the authors elaborate more on the details?\n\nQ5. I am still confused on the rationale of the additive factorization of $p_{\\theta,t}(x_t|c)$. It would be helpful if the authors can elaborate more on the comparison of additive versus multiplicative. In particular the statement in line 156 \"gives non-zero probability to regions outside the conditional $c$\" seems confusing.", "limitations": "yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "## Strengths\n\n1. The paper is well motivated by arguing that the open-loop guidance methods like CFG are suboptimal. At the same time, the fixed guidance scale is CFG does not take into account the dynamic nature throughout the diffusion sampling.\n\n2. The idea of dynamically adjusting the guidance weight based on the model's self-estimated confidence of the class posterior seems interesting.\n\n3. The experiments demonstrate interesting observations of the dynamic nature of the proposed approach in terms of enforcing different guidance scale at different diffusion steps and with different difficulties of the prompt.\n\n## Weaknesses\n\n1. The experimental evaluation is more or less illustrative and lacks comprehensiveness. In particular, the class conditional generation experiments are only performed using EDM2-XS, while the text-conditioned generation experiments have no numerical results. It is thus unclear how much gain can be achieved by the proposed approach.\n\n2. While being theoretically motivated, heuristics have been adopted such as introducing additional hyperparameters like $\\delta$ and $\\pi$ which leads to difficulty in hyperparameter tuning. By constrast, baselines such as CFG does not require extensive tuning of these additional hyperparameters, which seems like an overhead. More discussions should be conducted to show that introducing these variables leads to gains that outweigh the inconvenience.\n\n3. The presentation leads to various vague and confusing statements. Please refer to Q4-Q5 in Questions.\n\nMinor: multiple typos, e.g., Algorithm 1, line 1, the blue highlight should be $\\mu_\\theta(x_t)$. Table 1, what is DCFG? Line 167, should be with $\\lambda(x_t,t)$ as guidance scale.", "was_revised": true, "final_justification": "The rebuttal partially addresses my questions. However these concerns remain.\n\nThe experiments on larger-scale model are not completed. For the numbers provided, the performance is quite close for all guidance approaches, which raises uncertainty on the actual scalability and applicability of the proposed approach.\n\nAs a guidance method, I believe both effectiveness in terms of performance and convenience in terms of to what degree the method is easy to adopt are crucial factors that affect its applicability. In this respect, the additionally introduced hyperparameters further pose challenges to the practitioners who want to choose a guidance approach to perform for their downstream application. It would be more helpful if more in-depth analyses such as how these hyperparameters affect different tasks.\n\nThere are also other minor issues about presentation that are possibly fixable but indeed have raised concerns during reading. e.g., incorporating more details in the derivation to avoid confusion.\n\nTherefore I give a borderline score and believe that the paper can be carefully revised and further improved.", "rebuttal": "First and foremost, we would like to thank the reviewer for their in depth review of our manuscript. We appreciate that the reviewer recognizes the novelty of the dynamic posterior dependent guidance scale. We are happy that they found the work interesting, and also thank them for having taken the time to come up with interesting and challenging questions that, in our eyes, will improve the paper.\n## Response to Weakness #1: Requirement for more exhaustive experiments\nWe first address the reviewer's concern that the small-scale experiments may not be sufficient for assessing the method's significance. We agree that strengthening the paper in this regard would significantly improve its quality and impact.\n### Extension from small EDM2-XS to larger EDM2-L  \nTo address this, during the rebuttal period we conducted the same analysis previously performed with the EDM2-XS model using the much larger EDM2-L model, providing more comprehensive and qualitative insights. The initial results, computed on 10k generated images, are presented in Table 1 below. The extension towards 50k generated images, as well as the addition of the Precision and Recall metrics, is under way but will not be ready in time for the initial rebuttal.  \nWe can observe that the results on EDM2-L are more similar for the different methods, compared to the small model. This is because the EDM2-L model is far more qualitative and requires much less guidance, a result expected from our interpretation of guidance as an error correcting scheme. As a consequence, all guidance approaches are more similar to each other.\n\n**Table 1: Performance comparison on EDM2-L (10k images)**\n| Method | FID | FD$_{\\text{DinoV2}}$  \n|--------|---------|----------------------|  \n| CFG | 8.15 | 119.70 | \n| LIG | 7.78 | **112.68** | \n| FBG |**7.75** | 114.90 |\n\n### Additional T2I results with Stable Diffusion 2 on MS-COCO 30k  \nSimilarly, to further support our findings in the T2I setting, we evaluated the FID, FD$_{\\text{DinoV2}}$ alongside the average CLIP and Aesthetic Scores for different guidance schemes (CFG, LIG, and our proposed FBG) using Stable Diffusion 2 on the standard MS-COCO 30k dataset. These results, currently limited to a subset of 3k prompts, are reported in Table 2. Results on all 30k prompts will become available next week.  \nThe MS-COCO30k dataset contains prompts that originate from captioned images, which all remain fairly standard. We expect our approach to work even better in context where the prompts are much more varied. To remain consistent with the literature we however still chose to evaluate on this standard benchmark, on which FBG still slightly outperforms alternative approaches.\n\n**Table 2: Quantitative evaluation of T2I on MS-COCO benchmark (3k images)**\n| Method | FID | FD$_{\\text{DinoV2}}$ | CLIP-Score| Aesthetic Score|  \n|--------|---------|----------------------|-----------|--------|  \n| CFG | 19.64 | 53.55 | 0.310 | 5.650 |  \n| LIG | 18.81 | 53.55$^*$| 0.311 | 5.742 |  \n| FBG |**18.62** | **52.93** | 0.311 |**5.753**|\n\n$^*$In this setting, the best LIG interval corresponded to applying guidance on the whole trajectory.\n\nWe hope that these results help convince the reviewers of the more general validity of the method, as well as of our commitment to incorporate their valid critique to improve the current submission.\n## Response to Weakness #2: Introduction of $t_0$ and $t_1$\nOur Feedback Guidance (FBG) approach, like Limited Interval Guidance (LIG), has three hyperparameters, making initial tuning harder than CFG. However, unlike LIG, FBG is derived from first principles, offering insights into why guidance is needed. Its main advantage is that it automatically detects when and where strong guidance is required, removing the need for a changeable guidance scale—unlike CFG or LIG, which use fixed schedulers independent of prompt or state. Once valid hyperparameters are set at the model/application level, FBG self‑regulates guidance across prompts and timesteps, focusing on parts needing the strongest correction, as shown in our T2I results. We acknowledge the concern about introducing these hyperparameters, which is why we added:\n>“The hyperparameters introduced for FBG are conceptually interpretable and need to be set only once. They allow the model to self-regulate guidance strength across prompts and timesteps, reducing the need for extensive prompt specific tuning compared to fixed-scheduler approaches.”\n## Response to Weakness #3: Unclear statements\nWe have gone through the manuscript in close detail again to remove any possibly confusing statements, such as the typos mentioned (such as the “DCFG” present in Table 1, which was the previous acronym we had internally used for our FBG scheme and which was unfortunately left unchanged before the submission). In this regard, we also spend close attention to Q4 and Q5, for which additional sentences/derivations have been added to the main document.\n## Response to Question #1: Selection of $t_0$ and $t_1$\nWhile $t_0$ and $t_1$ require manual tuning, we designed them to be highly interpretable, simplifying this process. As outlined, they correspond to the average moments guidance starts and stops (similar to $t_{\\text{start}}$ and $t_{\\text{end}}$ in LIG), and intuitive reasoning—e.g., earlier or wider intervals producing greater changes—helps calibrate them. Figures 3a and 8a further show that across a reasonable range of $t_0$/$t_1$ values, FBG outperforms CFG on EDM2‑XS using FD$_{\\text{DinoV2}}$ and FID. While making $t_0$ and $t_1$ dynamic (e.g., automatically adjust to earlier timesteps for low‑frequency features like landscapes) is a promising direction we are highly interested in, we believe our fixed‑parameter approach already represents a significant advance over static guidance schemes.\n## Response to Question #2: Larger scale models + Flow Matching models\nFor the analysis of larger scale models we refer to our answer of Weakness #1.  \nRegarding the adaptability to Flow Matching Models, although left as future work, it is our belief that a very similar Feedback Guidance scheme can easily be derived and described in this context. The novel ideas introduced, such as the dynamic posterior dependent guidance scale, are in no way connected to the specific choice of diffusion models described in this paper. We are very excited about future works looking into extending the framework of Feedback Guidance to Flow Matching models, Stochastic Interpolators or Schrödinger Bridges.\n## Response to Question #3: Evaluation of T2I\nIn the context of T2I, we have focused on obtaining FID/FD$_{\\text{DinoV2}}$ values as well as CLIP and Aesthetic Scores on MS-COCO 30k, on which, similarly to what was obtained using EDM2-XS, FBG outperforms CFG and performs on par with LIG. We also agree with the reviewer that careful visualizations are key to understanding a paper. To improve on the current, limited, illustrative examples, we have have prepared multiple additional images similar to Fig. 4a and Fig. 5 that will be added to the appendices. Unfortunately, to remain in line with the NeurIPS rebuttal guidelines, we can not share them at this stage.\n## Response to Question #4: Unclear derivation of the dynamic guidance scale\nSeeing that this was pointed out by two of the reviewers, we have added two additional lines in the derivation of the main body as well as a detailed derivation in the appendix. Equation 7 now reads:  \n$$ \n\\\\begin{split}  \n\\\\nabla_x\\\\log p(x_t \\\\mid c)&=\\\\nabla_x\\\\log\\\\big(p_{\\\\theta,t}(x_t|c)-(1-\\\\pi)p_{\\theta,t}(x_t)\\big)\\\\\\\\&=\\\\frac{1}{p_{\\\\theta,t}(x_t|c)-(1-\\\\pi)p_{\\\\theta,t}(x_t)}\\\\big(\\\\nabla_x p_{\\\\theta,t}(x_t|c)-(1-\\\\pi)\\nabla_x p_{\\\\theta,t}(x_t)\\\\big)~\\\\\\\\&= \\\\nabla_x\\\\log p_{\\\\theta,t}(x_t) +\\\\lambda(x_t, t)\\\\Big(\\\\nabla_x\\\\log p_{\\\\theta,t}(x_t \\\\mid c)-\\\\nabla_x\\\\log p_{\\\\theta,t} (x_t)\\\\Big) ~\\\\  \n\\\\end{split}  \n$$\nWe have also added an additional inline equation: \n>  \"To obtain the last equation, we make use of the identity $\\nabla_x p(x) = p(x) \\nabla_x \\log p(x)$ and then reorder the terms to obtain a familiar looking guidance equation.\"\n\nWe thank the reviewer for highlighting this, as the theoretical underpinning is fundamental to our work, making a clear derivation of our guidance scheme is essential.\n## Response to Question #5: Rationale behind additive vs. multiplicative assumption\nThe interpretation of guidance schemes through error assumption models is in our eyes one of the most relevant contributions of our work. We agree with the reviewer that explaining the rationale behind the choice of an additive error model is important, which is why we have added two sentences in the main body. \n>“The additive assumption can be seen as less restrictive than the multiplicative one as it allows the learned conditional distribution $p_{\\theta,t}(x_t|c)$ to be non-zero in regions where the true conditional distribution $p_t(x_t|c)$ is zero, a feat the multiplicative assumption is incapable of. Due to the joint training pipeline, and the fact that training pairs often contain more than a single element, such an overlap of the learned distributions is in practice highly likely.“\n\nWe would however like to emphasize that our point is not that the additive error assumption is fully representative of the true error of the learned model, but merely that this new assumption might lead to different properties for the guidance scheme. In the future, analyzing different, more intricate, error models might lead to further findings, which we look forward to analyzing.\n\n## Summary  \nWe appreciate the reviewer's concerns and questions, and are convinced our responses as well as the additional experimental results have the potential to improve the manuscript considerably. We hope the reviewer reassesses the manuscript in light of this rebuttal, to see whether it warrants increasing their low original score."}, {"review_id": "7WlyKsNQMW", "reviewer": "Reviewer_maqm", "summary": "This paper study the classifier-free guidance (CFG) in diffusion model. The traditional guidance scheme harm the diversty and memory-inefficient due to the constant guidance design. To tackle this problem, the authors propose a novel state-dependent dynamic guidance scheme - FeedBack Guidance (FBG)，which uses a state-dependent coefficient to self-regulate guidance amounts based on need. Experiment on the ImageNet benchmark demonstrate its effectiveness.", "questions": "- CFG is usually used to control the prompt-following capability of the generated image. Should it measure the text-to-image alignment of the generated image to evaluate the effectiveness of the proposed method?\n- What is the meaning of Precision and Recall in the context of the experiment section? What about the $FD_{DinoV2}$?\n- The proposed FBG works in a state- and timestep-dependent manner, so how is the extra overhead brought by this scheme compared with the naive CFG scheme?\n- The proposed method shows timestep-aware guidance derived from the professional derivation. I'm curious about how the performance comparison is with the simple tiemstep-aware dynamic weighing CFG scheme, like weight scheduler [1], which comes from the empirical observation.\n-  The implementation details of the Hybrid CFG-FBG?\n\n[1]  WANG, at, al. Weight-Scheduler: Analysis of classifier-free guidance weight schedulers.", "limitations": "yes", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**:\n- Good writing, and easy to follow\n- The idea is novel and interesting, and more importantly, supported by solid theoretical proof\n\n**Weaknesses**:\n- The experiments are insufficient, without comparison with other CFG-related studies [1,2]\n- Some terms (e.g. $FD_{DinoV2}$) have no explanation, and some derivations are skipped too much (e.g., Sec. .3.3)\n- The experiments are insufficient, with limited comparative methods.\n\n[1]  WANG, at, al. Weight-Scheduler: Analysis of classifier-free guidance weight schedulers. \n\n[2] Chung, et al. CFG++: Manifold-constrained classifier free guidance for diffusion models.", "was_revised": true, "final_justification": "The issue with this work is the inadequacy of the experiment. The author primarily compares the proposed method with the LIG while ignoring other CFG-related methods. In the rebuttal stage, although the author provides some justification for the experiment, it is not fully convincing via the internal experiment without any quantitative results as evidence. In addition, the theoretical derivation of this work is somewhat unclear. Therefore, I have decided to retain my initial rate.", "rebuttal": "We would like to start by thanking the reviewer for their acknowledgement of the novelty of our approach. Furthermore, we are glad that they found the paper both interesting and easy to follow.\n## Response to Weakness #1: Comparison to other baselines\nThe reason that our main focus was on comparing our method to CFG and LIG is because these two guidance methods are respectively the most popular and the most performant in the literature. We have, internally, played with other alternative fixed schedulers such as those described in the “Analyzing weight-schedulers” paper [1], but our results have always demonstrated that choosing a finite guidance interval as proposed in LIG always surpassed these. Our understanding is that guidance at the beginning of the diffusion process always has a negative impact on FID, which means that only the sine, linear or $\\Lambda$ guidance shapes are prominent options. All three are however active too early, having detrimental effect on the performance. Furthermore, we wanted the main focus to be on the spatio-temporal dependence of our guidance scheme, which is why we did not compare to methods such as CFG++, a reference which we have now aded to the related work section.\n\n[1] WANG, at, al. Weight-Scheduler: Analysis of classifier-free guidance weight schedulers.\n## Response to Weakness #2: Unclear derivations/metrics\nFor the description of the metrics, such as FD$_{\\text{DinoV2}}$ or Precision/Recall, see our answer to Question #2.\n\nTo make sure that the derivations are easily tractable, we have added two additional lines in the derivation of the main body as well as a detailed derivation in the appendix. Equation 7 now reads:  \nSeeing that this was pointed out by two of the reviewers, we have added two additional lines in the derivation of the main body as well as a detailed derivation in the appendix. Equation 7 now reads:  \n$$ \n\\\\begin{split}  \n\\\\nabla_x\\\\log p(x_t \\\\mid c)&=\\\\nabla_x\\\\log\\\\big(p_{\\\\theta,t}(x_t|c)-(1-\\\\pi)p_{\\theta,t}(x_t)\\big)\\\\\\\\&=\\\\frac{1}{p_{\\\\theta,t}(x_t|c)-(1-\\\\pi)p_{\\\\theta,t}(x_t)}\\\\big(\\\\nabla_x p_{\\\\theta,t}(x_t|c)-(1-\\\\pi)\\nabla_x p_{\\\\theta,t}(x_t)\\\\big)~\\\\\\\\&= \\\\nabla_x\\\\log p_{\\\\theta,t}(x_t) +\\\\lambda(x_t, t)\\\\Big(\\\\nabla_x\\\\log p_{\\\\theta,t}(x_t \\\\mid c)-\\\\nabla_x\\\\log p_{\\\\theta,t} (x_t)\\\\Big) ~\\\\  \n\\\\end{split}  \n$$\nWe have also added an additional inline equation: \n> \"To obtain the last equation, we make use of the identity $\\nabla_x p(x) = p(x) \\nabla_x \\log p(x)$ and then reorder the terms to obtain a familiar looking guidance equation.\"\n\nWe thank the reviewer for pointing this out, seeing that the theoretical underpinning of our approach is fundamental to our work it is of uttermost important that the derivation of our guidance scheme is clear.\n## Response to Weakness #3: Insufficient Experiments  \nWe wish to address the reviewer's concern that the small-scale experiments may not be sufficient for assessing the method's significance. We agree that strengthening the paper in this regard would significantly improve its quality and impact.\n### Extension from small EDM2-XS to larger EDM2-L  \nTo address this, during the rebuttal period we conducted the same analysis previously performed with the EDM2-XS model using the much larger EDM2-L model, providing more comprehensive and qualitative insights. The initial results, computed on 10k generated images, are presented in Table 1 below. The extension towards 50k generated images, as well as the addition of the Precision and Recall metrics, is under way but will not be ready in time for the initial rebuttal.  \nWe can observe that the results on EDM2-L are more similar for the different methods, compared to the small model. This is because the EDM2-L model is far more qualitative and requires much less guidance, a result expected from our interpretation of guidance as an error correcting scheme. As a consequence, all guidance approaches are more similar to each other.\n\n**Table 1: Performance comparison on EDM2-L (10k images)**\n| Method | FID | FD$_{\\text{DinoV2}}$  \n|--------|---------|----------------------|  \n| CFG | 8.15 | 119.70 | \n| LIG | 7.78 | **112.68** | \n| FBG |**7.75** | 114.90 |\n\n### Additional T2I results with Stable Diffusion 2 on MS-COCO 30k  \nSimilarly, to further support our findings in the T2I setting, we evaluated the FID, FD$_{\\text{DinoV2}}$ alongside the average CLIP and Aesthetic Scores for different guidance schemes (CFG, LIG, and our proposed FBG) using Stable Diffusion 2 on the standard MS-COCO 30k dataset. These results, currently limited to a subset of 3k prompts, are reported in Table 2. Results on all 30k prompts will become available next week.  \nThe MS-COCO30k dataset contains prompts that originate from captioned images, which all remain fairly standard. We expect our approach to work even better in context where the prompts are much more varied. To remain consistent with the literature we however still chose to evaluate on this standard benchmark, on which FBG still slightly outperforms alternative approaches.\n\n**Table 2: Quantitative evaluation of T2I on MS-COCO benchmark (3k images)**\n| Method | FID | FD$_{\\text{DinoV2}}$ | CLIP-Score| Aesthetic Score|  \n|--------|---------|----------------------|-----------|--------|  \n| CFG | 19.64 | 53.55 | 0.310 | 5.650 |  \n| LIG | 18.81 | 53.55$^*$| 0.311 | 5.742 |  \n| FBG |**18.62** | **52.93** | 0.311 |**5.753**|\n\n $^*$In this setting, the best LIG interval corresponded to applying guidance on the whole trajectory.\n \nWe hope that these results help convince the reviewers of the more general validity of the method, as well as of our commitment to incorporate their valid critique to improve the current submission.\n## Response to Question #1: Prompt alignment metric\nCFG serves not only to improve prompt alignment but also to maintain overall image quality, as DMs underperform on all metrics without guidance. Its key advantage is letting users trade image diversity for stronger alignment by adjusting the guidance scale. This makes FBG’s compatibility with other methods like CFG and LIG particularly valuable: FBG can improve baseline quality, while the guidance scale remains available for users seeking stronger alignment. Given the relevance of metrics like CLIP‑score and Aesthetic scores in T2I, we report these alongside FID and FD in our Stable Diffusion experiments.\n## Response to Question #2: Metric explanations\nWe thank the reviewer for pointing out the absence of explanations regarding the nature of the used metrics. After all, these are not as common as the FID/CLIP-Score and should therefore be explained. To remedy this we have added a sentence describing the FD$_{\\text{DinoV2}}$ metric:\n>“The FD-DinoV2, similarly to the FID, measures the Fréchet Distance between generated images and reference images inside the DinoV2 embedding space. The DinoV2 embedding space is trained on a large diverse unlabeled dataset, in stark contrast with the InceptionV3 network used for FID which is solely trained on ImageNet classes and is therefore insensitive to certain features such as human faces. ”\n\nas well as a sentence describing the Precision-Recall metrics: \n> “The precision and recall metrics, very useful to analyze the quality-diversity tradeoff of a method, measure respectively how many of the generated images lie close to the reference image manifold (quality) and inversely how many of the reference image lie close to the generated image manifold (diversity). ”\n## Response to Question #3: Computational overhead of FBG\nOne of the key advantages of the iterative posterior estimation approach is that it comes at negligible additional cost as it solely relies on precomputed quantities, namely the conditional and unconditional score functions already required for the denoising itself. To further emphasize this point we have added a sentence in the main document in section 3.3 mentioning: \n> “A crucial advantage of computing the posterior using the scheme describe above is that it causes negligible computational overhead as all required quantities, in particular $\\mathbf{\\mu}_c(\\mathbf{x}_t|c)$ and $\\mathbf{\\mu}(\\mathbf{x}_t)$, are already computed.”\n## Response to Question #4: Comparison to other deterministic weight schedulers\nThe obtained state- and time-dependent guidance scale shares some aspects with simple predefined guidance schemes, such as LIG or some schemes described in the “Analyzing CFG weight schedulers” work [1]. In particular, we find that our guidance scale is small during the early stages of diffusion, increases during intermediate stages and finally decreases towards later stages. These trends are perfectly in line with the best performing weight-schedulers described in the aforementioned work [1] as well as with the scheduler proposed by LIG.\n## Response to Question #5: Implementation of FBG_CFG\nThe Hybrid FBG-CFG scheme is implemented in the provided code under the name of *‘Hybrid_CFG_FBG’*. We have further added an algorithm, as well as a paragraph in the appendix section describing how precisely the two guidance schemes can be combined. Essentially, the easiest way to understand this hybrid scheme is to see the total guidance scale as the sum of the two terms, i.e. $\\lambda_{\\text{tot}}(x,t) = \\lambda_{\\text{FBG}}(x,t) + \\lambda_{\\text{CFG}}$.\n## Summary  \nWe appreciate the reviewer's concerns and questions, and are convinced our responses as well as the additional experimental results have the potential to improve the manuscript considerably. We hope the reviewer reassesses the manuscript in light of this rebuttal, to see whether it warrants increasing their original score."}, {"review_id": "KvS5Q1Nnxh", "reviewer": "Reviewer_WucL", "summary": "This paper introduces Feedback Guidance (FBG), a new method for dynamically adjusting the guidance scale in conditional diffusion models. Unlike traditional Classifier-Free Guidance (CFG), which uses a fixed guidance scale throughout the sampling process, FBG adapts the guidance based on the current state of the sample and the progress in the denoising process.", "questions": "None", "limitations": "-Additive error model is a simplification.\n-Posterior estimation has a self-reference bias, mitigated using a correction term (δ).\n-Evaluation limited to small models (EDM2-XS); large-scale testing pending.", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n-Principled Theoretical Framework: Introduces a state- and time-dependent guidance mechanism derived from first principles, contrasting with heuristic-based methods like CFG or LIG.\n-Improved Quality-Diversity Trade-off:Feedback Guidance outperforms Classifier-Free Guidance (CFG) in both fidelity (FID) and diversity (Recall), and performs competitively with Limited Interval Guidance (LIG).\n-Adaptivity to Sample and Prompt: Guidance dynamically adjusts based on the current sample state and prompt complexity, enabling fine-grained control over generation without manual tuning.\n-Trajectory-Specific Behavior: Unlike static methods, FBG can vary guidance per sample trajectory, which helps avoid over- or under-guiding individual outputs.\n-Empirical Validation Across Tasks: Results are shown for both class-conditional generation (ImageNet) and text-to-image generation (Stable Diffusion 2), demonstrating broad applicability.\n-Extensible and Modular: Can be easily combined with other methods (e.g., CFG + FBG or LIG + FBG), and integrates well with existing diffusion frameworks.\n\nWeaknesses\n-Computational Overhead: Estimating posterior likelihoods during sampling introduces additional computation, which may slow down inference compared to simpler fixed-guidance methods.\n-Posterior Estimation Bias: The approach relies on self-evaluation of the conditional model, which can lead to biased posterior estimates. This is partly mitigated by a correction term, but still a limitation.", "was_revised": true, "final_justification": "The rebuttal has partially addressed my concerns. I still find the experiments to be not entirely convincing, but I remain positive about the well-motivated approach and still lean a bit towards acceptance.", "rebuttal": "First and foremost we would like to thank the reviewer for their detailed yet concise feedback. We appreciate that you recognize that our method is supported by strong mathematical foundations as well as the fact that it brings interesting insights in the context of prompt/trajectory-specific behavior.\n\n## Response to Weakness #1: Computational overhead\n\nOne of the key advantages of the iterative posterior estimation approach is that it comes at negligible additional cost as it solely relies on precomputed quantities, namely the conditional and unconditional score functions already required for the denoising itself. To further emphasize this point we have added a sentence in the main document in section 3.3 mentioning: \n> “A crucial advantage of computing the posterior using the scheme describe above is that it causes negligible computational overhead as all required quantities, in particular $\\mathbf{\\mu}_c(\\mathbf{x}_t|c)$ and $\\mathbf{\\mu}(\\mathbf{x}_t)$, are already computed.”\n\n## Response to Weakness #2: Posterior estimation parametrisation\n\nThe specific choice of parametrizing the posterior through the additional bias parameter $\\delta$ is described in detail in appendix B2. While it is true that we could have made other choices, we believe that the choice of a linear transformation is well defended in the paper. It should also be pointed out, that even though this is the simplest possible choice, it still manages to lead to a method outperforming standard baselines, which we believe to be a strong demonstration of the potential of our approach. In the future, we are looking forward to seeing more work analyzing the potential of different parametrizations of the posterior.\n\n## Response to Weakness #3: Limited evaluation\n\nFinally, we wish to address the reviewer's concern that the small-scale experiments may not be sufficient for assessing the method's significance. We agree that strengthening the paper in this regard would significantly improve its quality and impact.\n\n### Extension from small EDM2-XS to larger EDM2-L  \nTo address this, during the rebuttal period we conducted the same analysis previously performed with the EDM2-XS model using the much larger EDM2-L model, providing more comprehensive and qualitative insights. The initial results, computed on 10k generated images, are presented in Table 1 below. The extension towards 50k generated images, as well as the addition of the Precision and Recall metrics, is under way but will not be ready in time for the initial rebuttal.  \nWe can observe that the results on EDM2-L are more similar for the different methods, compared to the small model. This is because the EDM2-L model is far more qualitative and requires much less guidance, a result expected from our interpretation of guidance as an error correcting scheme. As a consequence, all guidance approaches are more similar to each other.\n\n**Table 1: Performance comparison on EDM2-L (10k images)**\n\n| Method | FID | FD$_{\\text{DinoV2}}$  \n|--------|---------|----------------------|  \n| CFG | 8.15 | 119.70 | \n| LIG | 7.78 | **112.68** | \n| FBG |**7.75** | 114.90 |\n\n### Additional T2I results with Stable Diffusion 2 on MS-COCO 30k  \nSimilarly, to further support our findings in the T2I setting, we evaluated the FID, FD$_{\\text{DinoV2}}$  alongside the average CLIP and Aesthetic Scores for different guidance schemes (CFG, LIG, and our proposed FBG) using Stable Diffusion 2 on the standard MS-COCO 30k dataset. These results, currently limited to a subset of 3k prompts, are reported in Table 2. Results on all 30k prompts will become available next week.  \nThe MS-COCO30k dataset contains prompts that originate from captioned images, which all remain fairly standard. We expect our approach to work even better in context where the prompts are much more varied. To remain consistent with the literature we however still chose to evaluate on this standard benchmark, on which FBG still slightly outperforms alternative approaches.\n\n**Table 2: Quantitative evaluation of T2I on MS-COCO benchmark (3k images)**\n\n| Method | FID | FD$_{\\text{DinoV2}}$ | CLIP-Score| Aesthetic Score|  \n|--------|---------|----------------------|-----------|--------|  \n| CFG | 19.64 | 53.55 | 0.310 | 5.650 |  \n| LIG | 18.81 | 53.55$^*$| 0.311 | 5.742 |  \n| FBG |**18.62** | **52.93** | 0.311 |**5.753**|\n\n$^*$In this setting, the best LIG interval corresponded to applying guidance on the whole trajectory.\n\nWe hope that these results help convince the reviewers of the more general validity of the method, as well as of our commitment to incorporate their valid critique to improve the current submission.\n\n## Summary  \nWe appreciate the reviewer's concerns and questions, and are convinced our responses as well as the additional experimental results have the potential to improve the manuscript considerably. We hope the reviewer reassesses the manuscript in light of this rebuttal, to see whether it warrants increasing their original score."}]

Author rebuttal:

Meta-review: This paper proposes a guidance approach for diffusion models called Feedback Guidance (FBG). In contrast to classifier-free guidance (CFG), which treats the guidance strength as a fixed hyperparameter, FBG dynamically adapts the guidance strength over the denoising trajectory. The paper provides a theoretical framework and derivation of the approach.

Empirically, the authors evaluate FBG on class-conditional generation on ImageNet 512x512 using the EDM2-XS model. They compare FBG to vanilla classifier-free guidance and limited interval guidance (LIG), which is another approach to dynamic guidance that serves as a strong baseline. Overall, FBG outperforms CFG and performs similarly to LIG. FBG has better theoretical grounding than LIG, which is a heuristic approach.

The paper includes interesting results showing how the guidance scale changes during denoising for different prompts, which demonstrates that it can differ on a per-prompt basis.

Reviewer UFxC mentioned that the paper was clear, novel, and seems useful as it does not require extra training. This reviewer raised concerns regarding the scale of the experiments (only performed on EDM2-XS), the lack of evaluation metrics such as HPSv2, CLIPScore, and Aesthetic score, and the lack of text-to-video results. The authors provided some experiments on the larger EDM2-L model in their rebuttal, and added results using FBG for Stable Diffusion 2 on MS-COCO30k. In general, I do not think that text-to-video experiments are necessary for such a paper, as this is not commonly done in related work.

Reviewer qaVs noted that the paper is novel, well-motivated with solid theoretical underpinnings, and clearly written. The reviewer also noted that the proposed method is novel and effective, outperforming CFG and attaining similar performance to LIG.

They raised concerns about the computational cost of FBG and the lack of evaluation of a DiT diffusion backbone, a larger model, or a one-step generative model. The rebuttal addressed this reviewer’s concerns.

Reviewer o9ey noted that the paper is well-motivated and the idea of dynamically adjusting the guidance weight is interesting.

This reviewer raised concerns regarding the small model EDM2-XS used in the evaluation, the hyperparameters that need to be tuned for the FBG, and the clarity of parts of the exposition. In the rebuttal, the authors provided results on EDM2-L and Stable Diffusion 2. While the results show that different guidance methods perform somewhat similarly to each other, FBG still behaves similarly to LIG and both perform better than CFG. The reviewer was not fully convinced by the larger-scale results, but they nevertheless show that FBG does work in this setting. In the rebuttal, the authors acknowledged that their method has three hyperparameters, but LIG does too, so this does not seem like a key issue.

Reviewer maqm noted that the paper is well-written, novel, and backed by theory. This reviewer raised concerns about insufficient comparisons with other CFG methods (Weight-Scheduler and CFG++) and missing explanations in the derivations. 

I agree that adding quantitative comparisons to more guidance baselines would further support the usefulness of FBG. The rebuttal provided some additional results and clarifications about derivations and metrics. This reviewer was not fully convinced by the rebuttal.

Reviewer WucL noted that the proposed method is based on a principled theoretical framework (in contrast to heuristic methods like CFG or LIG), and that it outperforms CFG and performs competitively with LIG. The reviewer also noted that the paper includes results for class-conditional generation on ImageNet and text-to-image generation that demonstrate the applicability of FBG, and that it shows that FBG can be combined with CFG or LIG.

Reviewer WucL raised concerns regarding the computational overhead, posterior estimation bias, and scale of experiments. The rebuttal addressed some of these concerns.

Overall, this paper is interesting, novel, and proposes a theoretically grounded guidance method that outperforms CFG and is competitive with LIG (while having more theoretical foundation than LIG). The main drawback is the lack of complete experiments using larger-scale models. But even in the present form, it would be useful for the community.
Decision: Accept (poster)