Dynamic Classifier-Free Diffusion Guidance via Online Feedback
Authors: Pinelopi Papalampidi
Olivia Wiles
Ira Ktena
Aleksandar Shtedritski
Emanuele Bugliarello
Ivana Kajic
Isabela Albuquerque
Aida Nematzadeh
Venue: ICLR 2026 Poster (2026)
Keywords: Diffusion
text-to-image generation
classifier-free guidance
Primary area: generative models

Abstract: Classifier-free guidance (CFG) is a cornerstone of text-to-image diffusion models, yet its effectiveness is limited by the use of static guidance scales. This ``one-size-fits-all'' approach fails to adapt to the diverse requirements of different prompts; moreover, prior solutions like gradient-based correction or fixed heuristic schedules introduce additional complexities and fail to generalize. In this work, we challenge this static paradigm by introducing a framework for dynamic CFG scheduling. Our method leverages online feedback from a suite of general-purpose and specialized small-scale latent-space evaluators—such as CLIP for alignment, a discriminator for fidelity and a human preference reward model—to assess generation quality at each step of the reverse diffusion process. Based on this feedback, we perform a greedy search to select the optimal CFG scale for each timestep, creating a unique guidance schedule tailored to every prompt and sample. We demonstrate the effectiveness of our approach on both small-scale models and the state-of-the-art Imagen 3, showing significant improvements in text alignment, visual quality, text rendering and numerical reasoning. Notably, when compared against the default Imagen 3 baseline, our method achieves up to 53.8% human preference win-rate for overall preference, a figure that increases up to to 55.5% on prompts targeting specific capabilities like text rendering. Our work establishes that the optimal guidance schedule is inherently dynamic and prompt-dependent, and provides an efficient and generalizable framework to achieve it.

Published as a conference paper at ICLR 2026
DYNAMIC CLASSIFER--REE DIFFUSION GUIDANCE VIA ONLINE FEEDBACK
Pinelopi Papalampidi', Olivia Wiles'; Ira Ktena² Aleksandar Shtedritski Emanuele Bugliarello', Ivana Kajič', Isabela Albuquerque', Aida Nematzadeh! 'Google DeepMind, Ellison Institute of Technology, 'University of Oxford ginelopi@google com
ABSTRACT
Classifier-free guidance (CFG) is a cornerstone of text-to-image diffusion models, yet its effectiveness is limited by the use of static guidance scales. This "one-size- fits-all" approach fails to adapt to the diverse requirements of different prompts: moreover, prior solutions like gradient-based correction or fixed heuristic sched- ules introduce additional complexities and fail to generalize. In this work, we chal- lenge this static paradigm by introducing a framework for dynamic CFG schedul- ing. Our method leverages online feedback from suite of seneral-purposs and specialized small-scale latent-space evaluators- as CLIP for alignment, a discriminator for fidelity and human preference reward model-to to assess gener- ation quality at each step of the reverse diffusion process. Based on this feedback, we perform greedy search to select the optimal CFG scale for each timestep, creating a unique guidance schedule tailored to every prompt and sample. We demonstrate the effectiveness of our approach both small-scale models and the fate--ofthe--a art Imagen 3, showing significant improvements in text alignment, vi- sual quality, text rendering and numerical reasoning. Notably, when compared against the default Imagen 3 baseline, our method achieves up to 53.8% human preference win-rate for overall preference, a figure that increases up to to 55.5% on prompts targeting specific capabilities like text rendering. Our work establishes that the optimal guidance schedule is inherently dynamic and prompt-dependent, and provides an efficient and generalizable framework to achieve it.
INTRODUCTION
The remarkable progress in text-to-image synthesis, powered by diffusion models (Ho et al. 2020 Song et al. 2023). has unlocked unprecedented creative potential. However, generating images from diffusion models requires hundreds of sampling steps to achieve sufficient generation quality. Consequently, a critical frontier of research is not only in training more powerful models, but also in enhancing inference in terms of efficiency and controllability without the need for costly retraining.
A cornerstone of controlling the generation process at inference time is classifier-free guidance (CFG: Ho & Salimans 2022) which has become the de facto standard in image generation. CFG provides a mechanism to amplify the influence of the text prompt, allowing to trade diversity for stronger adherence to the conditioning signal via single guidance scale. However, the guidance scale is typically either set to a single, static value for the entire generation process or is defined as schedule depending only on the sampling timestep based on empirical observations (Kynkäänniemi ct al. 2024: Chang et al. 20233 Sadat et al.] 2023 [Wang et al. (2024). In all cases, CFG is reduced to a "one-size sit-siiefffttlal strategy that overlooks the nuanced demands different prompts during inference. For example, a prompt requiring complex compositional arrangements may need strong guidance for text alignment, whereas a prompt focused on a specific artistic aesthetic might benefit from lower guidance to preserve visual fidelity and diversity. We empirically validate this hypothesis and further find that generating specific, challenging attributes like legible text within an image often
"Equal contributions
Work done while Google DeepMind.

---

Published as a conference paper at ICLR 2026
responds poorly to standard guidance strengths. This rigidity forces an undesirable compromise, where optimizing for one aspect (e.g., aligmment) often degrades another (e.g., aesthetics).
In this paper, we challenge the notion of a static guidance scale in diffusion models. We hypothesize that the optimal trade-off between prompt alignment and visual quality is not fixed, but is a dynamic function of the prompt's content, the current generation stage, and the diffusion model itself. To realize this, we propose a framework that dynamically selects the optimal CFG scale using online feedback from efficient latent evaluators. We employ suite of these evaluators measure distinct generation capabilities: both general-purpose (alignment, visual quality) and specialized ones such as text rendering and numerical reasoning. Crucially, these evaluators operate directly on noisy latents within the diffusion process, providing rich feedback with negligible computational overhead.
We leverage a greedy search-based optimization at each sampling step to evaluate discrete set of candidate CFG scales. We select the one that maximizes composite score from our latent evalua- tors. This procedure generates a dynamic CFG schedule tailored specifically to each prompt and its evolving sample. Interestingly. the average trend of our schedules aligns with empirical heuristics from prior work (Kynkäänniemi et al., 2024; Wang et al.) 2024), lending external validity to our approach. However, the key to our superior performance lies in the adaptability of our approach.
Our experiments on text-to-image model similar to Stable (Rombach et al. 2022) across the Gecko (Wiles et al.) 2024) and MS COCO (Lin al. 2014) benchmarks demonstrate that our method improves both alignment and visual quality simultancously. This stands in sharp contrast to
prior methods, such gradient guidance (Nichol al. 2022, Kim et al. 2023) fixed heuristic
schedules [Kynkäänniemi et al. 2024 Sadat et al. 2023 which typically improve one aspect at the expense of the other.
To demonstrate the generality and scalability of our approach, we apply it to the SoTA Imagen model (Team et al.] 2024). On the challenging Gecko and GenAl-Bench (Li 2024) prompt sets, human raters preferred generations from our method over the default Imagen 3 baseline in 53.6% and 53.8% of comparisons, respectively. The high quality of SOTA models also motivates extending our framework with more specialized, capability-basee evaluators. By incorporating a human preference reward model, and text rendering and numerical reasoning specific evaluators, we achieve even more fine-gra fine-grineed control. For the MARIO-eval (Chen et al., (2023a) benchmark requiring legible text, and the GeckoNum (Kajić et al.) 2024) one requiring counting skills, this specialized guidance boosts the human preference rate up to 55.5% and 54.1 % over default sampling, respectively.
Our contributions can be summarized as follows:
We propose a novel framework for dynamically optimizing the CFG schedule during gen- eration and introduce suite of latent evaluators that provide online feedback directly on noisy diffusion latents while increasing the computational requirements only by 1% in con- trast to 400% for pixel-space equivalent.
We show that prior empirical observations on CFG schedules fail to generalize across dif- ferent model families, prompt sets, and generation skills. In contrast, our method signif- wtweee/ffssooeeuuivlllleee
icantly improves sampling on both 3 across general-purposs and skill-specific prompt sets. We empirically demonstrate how our method's superiority lies in its adaptability and how the optimal CFG values change depending on the requirements of the prompt.
model and SoTA Imagen
2 RELATED WORK
Evaluation of text-to-image models. Evaluating the output of text-to-image models is a signif- icant challenge itself. Beyond traditional metrics, such as FID (Heusel et al. 2017) for image quality, and CLIPScore for alignment, that cannot offer fine-grained feedback on sample quality, recent work has developed VQA-based systems as autoraters (Wiles et al. 2024: Hu et al. [023] Yarom et al. 2024 Lin et al. 2024). These autoraters show strong correlation with human percep- tion, but their reliance on large language models (LLMs) makes them too computationally expensive for use during the iterative inference process, relegating them to post-hoc analysis. This motivates the search for evaluators that are both effective and efficient enough for online, step-by-step guid- ance. The most related work in this direction is that of Becker et al. (2025). Xu et al. (2023) Na
2

---

Published as a conference paper at ICLR 2026
Algronen
High CFG
Algrment
High CFG
X
Rumadd condaring
Ramard
rendering:
Three cats and
dog
Alignment: Roward Text rendorig:
Algnmert:
holiing benner that reads
Reward
rendering ng:
Diflusion model
1Dywnis flassffor-free
Algrment:
Reward 0.3 Text rondoring
Reward:
rendering:
Diffuson
Guldance!
Alignment: 0.4
Algnment: 0.5
X
Roward 0.05
Roward 0.6
Text rondering 0.1
Text rondoring
CFG
Dynamic CFG. We propose to perform greedy search over multiple CFG scales and select the one that maximizes the latent evaluators' scores at each sampling step. The evaluators are small-scale and operate directly the diffusion latent space increasing the computational overhead during inference by only 1%. Finally, for combining scores by multiple evaluators, we propose an adaptive weighting dependent on the denoising timestep.
Figure 1:
eet al. (2024), and (Singhal et al. (2025). Becker et al. (2025) employ CLIP for evaluation directly
in the latent space but they only assess denoised latents before the final decoding step. Xu et al.
(2023) and Na et al. (2024) use a discriminator for evaluating visual quality during sampling for rejecting poor quality samples or restart the process earlier on. Finally, Siinhal et al. (2025) and
Kim et (2025) propose FK steering and DAS, respectively, for improving sampling starting from multiple random seeds and evaluating the intermediate "potentials" of samples. We introduce flexible framework for combining feedback from multiple general and capability-speciie
evaluators to enable more fine-grained, multi-faceted control. Crucially, in contrast to prior work, we do not increase the NFEs and aim improving a single seed instead choosing or steering multiple seeds. Our method is orthogonal to work that rejects bad initial seeds.
Guided image generation. Classifier-free guidance (Ho & Salimans) 2022) has emerged as useful way of trading-off sample quality and diversity using a single parameter. Recent work has focused on tuning the CFG values: Kynkäänniemi et al. (2024) apply guidance only for limited time interval, and Chang et al. (2023) find that using a linearly increasing CFG schedule improves diversity. To improve sample quality and aligmment, [Sadat al. (2023) use custom CFG sched- ules, while Wang et al. (2024) find that tuning such schedules per model and prompt set further improves results. In attempt correct for mistakes caused by CFG, Nichol et al. (2022) propose to additionally employ classifier guidance via a noise-conditioned CLIP model which gradients push samples towards the direction of the prompt. In the opposite end of the spectrum, Kim et al. (2023) propose a similar method using a discriminator for increasing visual fidelity. However, combining CFG with auxiliary model guidance increases complexity, makes manual hyperparameter tuning more strenuous and does not offer different guidance strength depending on the prompt.
METHOD
3.1 PRELIMINARIES
Diffusion models are a class of generative models that learn to reverse noising process and are defined by two Markov processes. The forward process iteratively adds Gaussian noise to the data To with T increasingly noisy steps. At timestep E [1, T] noise is added to To as follows: X; Xxxo /1 C;,E,, ~ (O, 1), where a, E (0, 1) are pre-defined schedule parameters. The learned backward process gradually denoises Iy towards the data distribution p( p(Zdata). ). After training a diffusion model po po(u)) to fit the data distribution, we sample from it starting with Gaus- sian noise: &n
Va (x+ vI a966 Xt, ()). where 6x(X,, () is the model's 's noise prediction.
3

---

Published as a conference paper at ICLR 2026
3.2 ONLINE EVALUATORS
Given a noisy latent sample 2; at denoising step I, we compute a score e, for evaluating the sample's quality across specific dimension using one of the following evaluators.
Alignment. Given 2t and the conditioning prompt c, we compute noisy latent CLIP scores as a
prediction of final sample alignment:
eCLIP CI.I Vi CLIIIee
(1)
CLIP is initialized from a standard pre-trained model trained on clean real images and corresponding captions from the WebLI dataset (Chen et al. (2023b). We replace the embedding layer of the vision encoder with a randomly initialized one matching the dimensionality
the diffusion encoder. We then fine-tune the model image-text pairs after encoding the images into diffusion latents and injecting random noise with a similar time schedule as for the diffusion model training. We further condition the vision encoder on timestep converting CLIP into conditiont encoder. We use the standard CLIP contrastive objective to map noisy latents to text descriptions.
Visual quality. Given Zt, we compute score corresponding to the likelihood of an image being real independently of via a noisy latent Discriminator trained to differentiate between real and generated images, similar to prior work (Kim et al. 2023 Na et al. 2024):
p(x)(()
DDisc
log 1 p(xx(t)
(2)
where p(az)(() is the time-conditional probability of image 2i be real timestep 1. We initialize the discriminator from the latent CLIP vision encoder and introduce classification head on top for predicting whether the images are synthetic or real. We train the discriminator on small of real
vs. generated images from the MSCOCO dataset (Lin et al. 2014), similar to Kim et al. (2023).
Reward (Human preference). Similarly reward modeling, further fine-tune the latent align- ment evaluator on pairs of generated images for the same prompt given human preference labels that reflect overall preference (aesthetics, alignment, artifacts). For converting pairwise comparisons to scores, we follow common approaches from L.LM alignment (Ouyang et al.. (022) for reward tuning and use the Bradley- Terry (BT) model (Bradley & Terry 1952). According to the BT model, CLIP is further optimized according to the following training objective:
p(ilc)
p(i jlc) p((]c) + p(ilc)
where p(ilc) and p(jlc) is CLIP similarity between the prompt and each image i,j in the compar- ison pair, with being the preferred one.
Text rendering. We consider capability-speciin evaluator for text rendering. a challenging pect in image generation. We fine-tune the alignment evaluator on generated images labeled with scores by an OCR model. We introduce multimodal head on top of the dual encoder and train the model to predict text rendering specific scores. We optimize the evaluator with an MSE objective:
1 > (eTR COCR)²
MSETR
where eTR, EOCR are the scores predicted by the latent evaluator and OCR model, respectively.
Numerical Reasoning. We consider another capability by fine-tuning the noisy latent CLIP on subset of WebL.-1100B images (Wang et al. 2025) filtered to contain countable entities. We fine-tune the model with the original contrastive objective on the capability-specífic dataset.
evaluator for numerical reasoning
3.3 DYNAMIC CFG SEARCH VIA ONLINE FEEDBACK
Dynamic CFG. Classifier-free guidance (CFG) (Ho & Salimans 2021 alleviates the need of classifier for generating samples with high fidelity and mode coverage. In CFG, a model trained be both conditional and unconditional, and the respective scores are combined during generation via the CFG scale which regulates the trade-off between fidelity, alignment and diversity:
6n Ex(2 c) ea ((e(( en (xx(0))
(5)
4

---

Published as a conference paper at ICLR 2026
where 0 is the parameters of the diffusion model, e is the condition applied to the diffusion model, ie., the prompt for text-to-image generation, and 9 is an empty sequence used for training the un- conditional variant of the diffusion model.
We propose to dynamically select the optimal CFG scale per timestep given feedback e from the online evaluators of Section 3.2 (see Figure T_ Formally, given set of CFG scales S
{&1, ; Sn at every step we select the scale
8 arg marssseesr
(6)
which maximises the timestep-conditioneed evaluator's score e, for the conditioning prompt c.
We optimize the final sample quality via a greedy search across timesteps, selecting the CFG scale that maximizes our latent evaluators' scores per step. Crucially, this search is performed without increasing the Number of Function Evaluations (NFEs). For each timestep t. we denoise once to obtain the conditional cg(2, (c) and unconditional (0())(0) predictions, and then cheaply test multiple CFG scales via Equation 5 Since our latent evaluators are lightweight and operate directly in the latent space, there no increase in computation during inference (around 1% increase in FLOPs in contrast to 400% increase if operating in the pixel-space, see details in Appendix (A.3). We further validate this design choice in Section 5.2 demonstrating that our greedy approach achieves performance comparable computationally intensive beam search strategies.
Adaptive evaluators' weighting. We aim to combine feedback from general and capability- specific evaluators. Intuitively, approach is founded on the principle that different properties emerge at different stages of generation. For example, coarse-grasd soarse-grained alignment is established early on, while text legibility and artifact removal are late-stage concerns. Prior work also notes that high initial guidance can degrade visual quality (Wang et al.) 2024). Given this sampling time- dependency, static linear weighting of evaluator scores is insufficient. We therefore employ dynamic weighting scheme that adjusts the influence of each evaluator e E E according to the current timestep, a strategy we show be critical for optimal performance Section 5.2
G2 = Σ Ceet * Ct2 where Ceet CEE
64 Ct+1
et+1
Intuitively, our dynamic weighting scheme amplifies an evaluator's influence at the precise moment its signal becomes meaningful. which we identify by detecting a significant change in its score across timesteps-ad sign that the generation has entered an information-rich phase for that property.
4 EXPERIMENTAL SETUP
Diffusion Models. We experiment with both open-source and SoTA proprictary model families. We use LDM (i.e., latent diffusion model), variant of the open-source StableDiffusion (Rombach) et al. 2022) text-to-ing -mage model, trained on web-scale scale image data. We use LDM (865M param- eters) for ablations and LDMaage (2B parameters) for main results. We also transfer our approach to Imagen 3 (Team et al. 2024) and test whether our improvements hold on near-perfect text-to-imaae generation. For each model family we train separate evaluators tuned on the respective latent spaces.
Prompt Sets. We use general purpose and specialized prompt sets for evaluating image gener- ation performance across different generation aspects. We use Gecko (Wiles et al. 2024) and GenAI-Bench (Li et al., 2024), which are diverse prompt sets containing fine-grained categories, for measuring overall preference in text-to-image generation. We use MS-COCO eval (Lin al.) 2014) for automatic evaluation on visual fidelity due to access to the ground-truth reference images, MARIO-eval (Chen et al. 2023a) for evaluating text rendering, and GeckoNum (Kajćć 2024) for testing numerical reasoning (i.e., counting).
Evaluation. For automatic evaluation, we use Gecko score (Wiles et al. 2024) for measuring fine-grained text alignment and FID (Heusel et al., 2017) on MS-COCO for measuring fidelity. For human evaluation, we run studies via side-by-side comparisons between model variants and report win rates over the baseline marking significance with 95% confidence intervals. For Gecko and GenAl- -Benhhhechhhee we ask raters to indicate the image that they overall prefer (with respect to both alignment and aesthetics), for MARIO-eval we ask them to choose the image with the best aligned rendered text, and for GeckoNum ask them indicate the image that more closely represents the correct count of objects/entities (see details in Appendix A.4).
5

---

Published as a conference paper at ICLR 2026
Table 1: Filtering performance. We evaluate the degree of prompt alignment via the Gecko score while filtering samples of poor alignment at different % during sampling. For filterin, we either use the latent CLIP evaluator or an off-the-shelf CLIP model operating the pixel space. In all cases, we select the best out of a batch of 4 when filtering. Computed on the Gecko prompt set.
Filter @ [Gecko Score]
Model
Evaluator
No filtering
25%
50%
75%
100%
latent-space CLIP
37.6
39.7
41.4
43.0
43.0
LDMamal1
pixel-space CLIP
37.6
43.4
44.6
44.7
45.1
latent-space CLIP
42.9
45.9
45.2
46.6
46.0
LDMrgge
pixel-space CLIP
42.9
47.1
48.9
48.4
48.6
Latent evaluators' training. Our analysis reveals that the reliability of feedback from our latent evaluators depends heavily on the noise level. While coarse attributes like overall visual structure and semantic alignment can be assessed early in generation, fine-grained details-such as minor artifacts or the legibility of rendered text-can only be evaluated accurately at lower noise levels. This motivates a time-weighted loss schedule for the human feedback and text rendering evaluators. We provide details on training and computational requirements in Appendix A.T
5 RESULTS
5.1 EVALUATION OF LATENT EVALUATORS
by answering two questions: 1. What is the information loss by directly assessing compressed latents instead of pixel- space images? 2. How early during denoising can we get signal for sample quality?
We evaluate the effectiveness of the latent evaluators described in Section
3.2
Similarly Karthik et al. 2003)) and Astolf et al. (2024), we perform filtering for evaluating the effectiveness of the evaluators. Instead of filtering samples after denoising, we evaluate potential paths during generation. We consider a large number B of initial seeds per prompt and aim at
subselecting the K best ones at timestep t. We explore filtering at different timesteps t corresponding to a different percentage of NFEs.
We report the Gecko score on evaluator at different sampling stages in Table [] We compare the performance of the latent evalua- tor against pixel-space equivalent. In this case, we first perform one-step denoising from 2; to zo and decoding of xo into pixels, which produces clean but blurry images that can be processed by an off-the-shelff encoder. We find that the information loss suffer by operating directly latents is consistent for different noise levels. A Although there is an expected performance drop when using la- tents, we still maintain information about sample quality while reducing the computational overhead allowing us to use the latent evaluators online during inference (see Appendix find that we correctly discard poorly aligned samples from as early as 25% of the denoising process. We observe a similar behavior for the visual quality evaluator (see Appendix A.5
when filtering images via the alignment (CLIP)
A.3). Importantly, we
5.2
DYNAMIC CFG SEARCH
LDM. We compare our dynamic CFG search against gradient- gradient-based guidance (Nichol al. 2022
Kim et al. 2023) and static CFG schedules (Kynkäänniemi et al., 2024 [aadat et al. 2023) on
L.Maage in Table 2 using the automatic metrics described in Section
Alignment (CLIP) guidance is indeed effective for improving alignment without any benefits in visual fidelity, whereas the visual quality (Discriminator) guidance only slightly improves align- ment, but not FID. When combining the gradients of the two models, we observe effect; while CLIP improves aligmment. discriminator guidance fails to boost fidelity. In contrast, our dynamic CFG search (last block of Table 2 demonstrates clear and controllable trade-off. Using only the alignment evaluator optimizes the Gecko score, while using only the visual quality evaluator opti- mizes FID. Our full approach leveraging adaptive weighting combine the evaluators, successfully

---

Published as a conference paper at ICLR 2026
Table Automatic evaluation on Marge We report alignment and visual fidelity performance via Gecko score and FID respectively for (1) gradient-based guidance that uses auxiliary models for correcting samples, (2) static CFG schedules derived from empirical observations, and (3) our dynamic CFG search when using latent alignment and/or visual quality (VQ) evaluators.
Method
Latent evaluator/
Gecko score f (Gecko prompts)
FID +
Static schedule
(MS COCO prompts)
Default CFG (fixed)
43.8
25.6
A Alignment (Nichol et al. VQ (Kim ct al. 2023)
2022
46.1
25.6
Gradient guidance
44.6
25.5
Ali Alignment VQ
45.3
25.5
Limited Guidance Interval
43.0
26.1
Kynkäänniemi et al. 2024)
Static CFG schedules
Annealing (Sadat al. Mean of Dynamic CFG
2023
47.0
28.9
46.5
26.8
Median of Dynamic CFG
45.8
26.0
Alignment
45.5
26.4
VQ
44.0
24.8
Dynamic CFG search
Alignment VQ (linear)
45.0
25.4
Alignment VQ (adaptive)
47.2
24.8
improves both dimensions at once. We find the adaptive weighting to be critical: using a static, time-independent weighting significantly hurts performance.
We first compare the dynamic CFG search against a constant value, limited-interval guid- ance (Kynkäänniemi et al., 2024) and an annealing schedule (Sadat et al. (2023) (third block of 2). While the annealing schedule improves alignment at the cost of visual fidelity, our dynamic schedule matches its alignment performance while simultancously improving fidelity. To determine if the gain comes from the schedule's general shape or its per-prompt adaptability, "mean schedule" by averaging our dynamic schedules over all prompts and apply it universally. We find that performance drops in this condition, which, while still competitive, highlights that the per-prompt adaptability of our approach is crucial component of our method's success.
create static
Imagen 3. We next assess how our method transfers to Imagen 3 via human evaluation as de- scribed in Section 4 We extend the suite of latent evaluators since we find the discriminator to be an insufficient visual quality predictor for Imagen 3 during early experimentation" As discussed in Section 3.22, instead use reward evaluator trained on human preference data alongside with two capability-specife evaluators: one for text rendering and one for numerical reasoning.
We report win rates of side-by-side comparisons in Table 3 across Gecko, GenAI-Bench, MARIO- eval and GeckoNum. Our dynamic CFG framework yields statistically significant improvements over the strong Imagen bascline. Consistent with our findings on LDM, using either the alignment or the reward evaluator is preferred over the baseline across all prompt sets. We further validate that combining the two evaluators with adaptive weighting achieves the best results across all prompt sets reaching up to 54.7% win rate on MARIO-eval for text rendering.
We demonstrate the flexibility of our framework by also deploying two specialized evaluators for text rendering and numerical reasoning, We test their effectiveness on specialized prompt sets tailored for measuring each capability separately. On these prompt sets, we find that both evaluators achieve the highest win rates against the baseline (55.5% on text rendering and 54.1 % numerical reasoning) when also combined adaptively with the general purpose evaluators (either alignment or reward).
We additionally report the performance of the heuristic CFG schedules (Kynkäänniemi et al. 2024 Sadat et al.) 2023) as applied LDM on Imagen The results are striking: the schedules that offered modest improvements on LDM fail on Imagen 3, degrading performance below the bascline in most cases. This failure underscores a fundamental weakness heuristic- based methods: they are brittle because they rely empirical rules derived from specific model architecture and training
We hypothesize that since Imagen generate very high quality photorealistic images, predicting small artifacts aesthetic improvements discriminator can be more challenging than L.DM.

---

Published as a conference paper at ICLR 2026
Table Human Preference on Imagen 3. Side-by-side human comparisons of the baseline Imagen 3 and Imagen 3 with our dynamic CFG search. We report win rates for the custom CFG schedules against the default and underline the wins that are significant with 95% confidence interval. We report results on Gecko and GenAl-Bench for overall preference, MARIO-eval for text rendering and GeckoNum for numerical reasoning.
Method
Latent Evaluator
Win Rate (%) f MARIO-
Gecko
GenAI- Bench
GeckoNum
eval
Limited Interval
27.9
33.1
19.6
46.6
Annealing
46.4
34.4
42.7
50.8
Alignment
50.9
53.2
52.3
51.1
Dynamic CFG
Reward
52.1
51.4
53,8
53.8
Alignment Reward
53.6
53.8
54.7
53.6
Capability-specife evaluators
Text rendering
53.1
Alignment
55.3
Reward
55.5
Dynamic CFG
Numerical
52.2
Alignment
53.2
Reward
54.1
regime. When exploring the interval-based guidance in particular, we find that this schedule fails completely for text rendering specific prompts. This agrees with intuition that text rendering benefits from higher guidance throughout, but also in the final sampling timesteps which the prompt independent schedules do not take into account. In contrast, both heuristic schedules perform best on prompts related to numerical reasoning indicating that lower guidance strength in the beginning of denoising favors diversity for producing entities and objects in variable numbers. Our method's strength lies in its model-agnostic, online adaptation. Instead of applying a pre-determined, "hard- coded" schedule, derived after cumbersome hyper-parameter search, our framework discovers the optimal guidance on-the-fly by reacting directly to the outputs of the target model. This is why our approach generalizes out-of-the-box from a weaker to a state-of-the-are model and consistently improves performance across different generation skills.
Dynamic CFG vs FK steering We finally compare our method against FK steering Singhal et al. (025) on the Gecko prompt set evaluate the trade-off between computational cost and gener- ation quality. While FK steering involves searching over multiple initial seeds sonis-sinnificatlly in- creasing NFEs and inference cost-our Dynamic CFG operates on fixed seed with only a ~ 1% computational overhead. We consider FK steering with & 4 particles for Imagen 3 and con- duct a side-by-side human evaluation against our method when using the general-purpose (align- ment and reward) latent evaluators. The analysis results in win rate of 50.7% for FK steering (95% CI [48.9%, 52.5%) against our approach. This confidence interval implies the two methods statistically indistinguishable in terms of overall human preference and the results demonstrate that Dynamic CFG achieves performance comparable to compute-intensive intensive inference-tine scaling strategies without the associated increase in NFEs or latency.
Greedy CFG Scale Selection vs. Global Optimization. While our method utilizes greedy se- lection strategy to determine the optimal CFG scale at each timestep, we acknowledge that this does not theoretically guarantee a global maximum for the final sample quality. However, the stability of our latent evaluators supports this approximation; as detailed in Section [..T. these evaluators robust predictors of final image alignment, maintaining consistent predictions even during early denoising stages (c.g., from 25% completion). This approach parallels greedy decoding strategies commonly used in LLM inference, which prioritize efficiency and provide strong approximate solu- tions without the exhaustive cost of global search.
To empirically validate the effectiveness of greedy approach against we implement a beam search strategy. At each denoising step t. instead of selecting a single optimal trajectory, this method maintains the top-k candidates. For each candidate i, generate potential
exhaustive search,
8

---

Published as a conference paper at ICLR 2026
Alggement
Aigoment gigmmnnt (adeptive)
Aligement
Gecko
Genti-Gench
NAND--wa
GeckoNuu
Timeesbeps
Timesteps
Tamestans
(a) Median values LDM the Gecko prompt set when using an alignment (CIIP) visual qual- ity (VQ) evaluator their com- bination with a fixed linear or adaptive weighting.
(b) Smoothed median normalized values in Imagen 3 for the Gecko prompt set when using align- ment (CLIP) reward (Human pref) evaluator or their combina- tion with adaptive weighting.
(e) Smoothed median normalized values in Imagen 3 for the differ- prompts when using the hest performing combination of eval- uators as shown in Table
Figure 2: Median of the dynamic CFG schedule different models and prompt sets.
Low Guidance in Dynamic CFG
High Guidance in Dynamic CFG
The qaic
THE QUICK BRWN FOX JUMPS JUMPEVR THE LAZY DOG.
(a) Prompt: "...pop art depicting Mona Lisa... blocks of bright pink and yellow checkered design. with touch of orange and white..."
Prompt: "The quick brown fox jumps the lazy dog, written in serif font."
(b) Prompt: "A photograph of a thin, white line drawn in the sand on heach at sunrise. The line is straight, clean and simple..."
(d) Prompt: "A peacock fans it's plumage while panda is walking and jellyfish is swimming in the ocean."
Figure 3: We rank images by Imagen 3 from lowest to highest guidance strength when using dynamic CFG for the Gecko prompt set. For each prompt, we present pair images for default (left) vs dynamic CFG (right). Validating hypothesis, creative or simple prompts get low guidance, whereas prompts including text rendering and compositionality get the highest guidance.
states wi-1 across range of CFG values and prune the retain only the top-k paths for the subsequent step.
The primary constraint of beam search is the computational penalty; specifically, beam width of scales the NFEs by a factor of k. We compare our standard greedy approach against beam search with k 2 (doubling the inference cost) on Imagen 3 using the Gecko prompt set. Utiliz- ing the latent CLIP evaluator for guidance and the Gecko metric for final quality assessment, we observe that beam search yields marginal improvement, achieving a score of 73.7% compared to 73.1% for our greedy method. Given that the 0.6% performance gain requires a 100% increase in NFEs, we conclude that greedy selection offers the superior trade-off between generation quality and computational efficiency.
5.3 DYNAMIC CFG SCHEDULE
LDM. Figure 2a visualizes the median CFG schedule on LDM The behavior of the individual evaluators confirms they working intended, defining the extremes of the alignment- fidelitty

---

Published as a conference paper at ICLR 2026
stereoscopic 3D cartoon of the simpsons
"The quick brown fox jumps over the lazy dog
(a) Artifact correction
(b) Text Rendering
Figure 4: Qualitative examples for Imagen 3 on the Gecko prompt set when using default sampling (left) vs our dynamic search (right).
tra The alignment evaluator consistently favors high CFG scales to maximize alignment, while the visual quality one pushes towards low scales (approaching unconditional generation) to maximize fidelity. Our full method, using adaptive weighting, successfully navigates this trade-off. It generates an arc-shaped schedule that avoids extreme CFG values at the beginning and end of sampling. This emergent shape aligns with empirical findings from prior work (Wang et al., 2024). In contrast, static weighting of the evaluators fails to find this balance and produces schedule largely dominated by the alignment signal.
Imagen 3. We present the smoothed normalized median of the dynamic CFG schedule for Imagen 3 in Figure 2b when using either of the alignment or reward evaluators or their combination. Simi- larly to LDM the alignment evaluator favors high guidance strength in the beginning of denoising, but the optimal median schedule derived by the combination of the two evaluators significantly dif- fers from the one discovered for LDM. This further validates that no empirical observations regard- ing CFG can generalize beyond specifie model family, highlighting the strength of our dynamic approach that can adapt to different models consistently providing improvements.
We also present the smoothed normalized CFG schedule for the best performing variant of our dy- namic CFG per prompt set in Figure 2c We find that the patterns in the CFG schedules agree with our empirical observations: in contrast to the general-purge purposs prompt sets, text rendering (MARIO- eval) on average requires higher guidance strength especially in the end of denoising, and numerical reasoning (GeckoNum) benefits from lower guidance strength in the beginning of generation which favors diversity and avoids "template-like" generations of objects and entities allowing the model to generalize to variable counts. We further rank the generated images for the Gecko prompt set, which contains diverse prompt categories, based the average selected CFG across timesteps when ing dynamic CFG. We present in Figure 3 two of the lowest ranking examples on the left (i.c., low guidance strength) and two of the highest ranking ones. The visualization further validates hypothesis that the degree of guidance is dependent on the requirements of the prompt. Indeed, creative or simple prompts benefit from low CFG values, whereas prompts that require strong align- ment, such as text rendering and compositionality, need much higher guidance strength. We present additional qualitative results in Appendix A.6
6 CONCLUSIONS
In this paper, we propose framework for dynamically selecting the optimal CFG scale during de- noising in text-to-image generation. We demonstrate that the optimal trade-off' between conditional and unconditional generation is not fixed, but rather a dynamic function of the prompts' content, the sampling timestep, and the diffusion model. We suggest a suite of latent evaluators for assessing both general purpose (alignment, visual quality) and specialized (text rendering, numerical reason- ing) properties of generation and demonstrate that we can successfully use them during diffusion inference minimal computational cost. Crucially, while our method relies on the quality of these evaluators, we find that even general-purpose models trained on noisy data generalize effectively to diverse tasks without specialized fine-tuning. Furthermore, performance can be maximized by curating domain-specifie training data, offering flexible pathway for future enhancements. Given such evaluators, our proposed dynamic CFG significantly boosts generation quality on both weaker (gLDM) and more powerful (Imagen) models, validating the generalization of the approach. Our approach can be extended to more specialized skills given appropriate evaluators and the framework can be expanded to perform inference-time search beyond the CFG schedule.
10

---

Published as a conference paper at ICLR 2026
Acknowledgements We thank Benigno Uria, Oliver Wang, and Zhisheng Xiao for their feedback throughout the project. We are grateful to Chris Dyer and Sayna Ebrahimi for their feedback on the manuscript and Jordi Pont- -uset for his help in human evaluation.
Ethics Statement The full details of the human evaluation study design presented in Section 5.2 and Table 3 including compensation rates, were reviewed by our institution' 's independent ethical review committee. All participants provided informed consent prior to completing tasks and were reimbursed for their time.
REFERENCES
Pietro Astolfi, Marlene Careil, Melissa Hall, Oscar Mañas, Matthew Muckley, Jakob Verbeek, Adri- ana Romero Soriano, and Michal Drozdzal. Consistency-diversstyyeellll pareto fronts of condi- tional image generative models. arXiv preprint wrXiv:2406.10425 2024.
Jason Becker, Chris Wendler, Peter Baylies, Robert West, and Christian Wressnegger. Controlling
latent diffusion using latent clip. arXiv preprint arXiv:2503.08455, 2025.
Ralph Allan Bradley and Milton E Terry. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39(3/4):324-345 1952.
Huiwen Chang, Han Zhang, Jarred Barber, AJ Maschinot, Jose Lezama, Lu Jiang, Ming-Hsuan Yang, Kevin Murphy, William T Freeman, Michael Rubinstein, et al. Muse: Text-to-image gen- eration via masked generative transformers. arXiv preprint arXiv:2301.00 00000400704 2023.
Jingye Chen, Yupan Huang, Tengchao Lv, Lei Cui, Qifeng Chen, and Furu Wei. Textdiffuser: Diffusion models as text painters. Advances in Neural Information Processing Systems, 36:9353- 9387, 2023a.
Xi Chen, Xiao Wang, Soravit Changpinyo, AJ Piergiovanni, Piotr Padlewski, Daniel Salz, Sebastian Goodman, Adam Grycner, Basil Mustafa, Lucas Beyer, et al. Pali: A jointly-scales multilingual language-image model. The Eleventh International Conference on Learning Representations, 2023b.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume (long and short papers), pp. 4171-4186, 2019.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. In NewrIPS 2021 Workshop on Deep Generative Models and Downstrean Applications, 2021.
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv:2207.12598 2022.
arXiv preprint
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6855 2020.
Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna, and Noah A Smith. Tifa: Accurate and interpretable text-to-image faithfulness evaluation with question an- swering. arXiv preprint axxiv:2303.11897 2023.
Ivana Kajić, Olivia Wiles, Isabela Albuquerque, Matthiss Bauer, Su Wang, Jordi Pont-Tuset, and Aida Nematzadeh. Evaluating numerical reasoning in -oxtato- models. Advances Neural Information Processing Systems, 37:4221 1--4224, 2024.
Shyamgopal Karthik, Karsten Roth, Massimiliano Mancini, and Zeynep Akata. If at first you don't t succeed, try, try again: Faithful diffusion-basse text-to-image generation by selection. arXiv preprint arXiv:2305. 13308, 2023.
11

---

Published as a conference paper at ICLR 2026
Dongjun Kim, Yeongmin Kim, Se Jung Kwon, Wanmo Kang, and Il-Chul Moon. Refining genera- tive process with discriminator guidance in score-based diffusion models. In Proceedings of the 40th International Conference on Machine Learning, pp. 16567-16598. 2023.
Sunwoo Kim, Minkyu Kim, and Dongmin Park. Test-time alignment of diffusion models without reward over-optimization. In The Thirteenth International Conference on Learning Representa- tions, 2025.
Tuomas Kynkäänniemi, Miika Aittala, Tero Karras, Samuli Laine, Timo Aila, and Jaakko Lehtinen. Applying guidance in limited interval improves sample and distribution quality in diffusion models. arXiv preprint SBti::20040077244 2024.
Baiqi Li, Zhiqiu Lin, Deepak Pathak, Jiayao Li, Yixin Fei, Kewen Wu, Tiffany Ling, Xide Xia, Pengchuan Zhang, Graham Neubig, et al. Genai-bench: Evaluating and improving compositional text-to-visual generation. CoRR, 2024.
Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects context. In Computer Vision-ECCV 2014: 13th Europeun Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp. 740-755. Springer, 2014.
Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan. Evaluating text-to-visual generation with image-to-text generation. arXiv preprint arXiv:2404.0220 2024.
Byeonghu Na, Yeongmin Kim, Minsang Park, Donghyeok Shin, Wanmo Kang, and Il-Chul Moon. Diffusion rejection sampling. arXiv preprint arXiv:2405. 77880, 2024.
Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob Megrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. In International Conference Machine Learning, pp- 16784-16804. PMLR, 2022.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to fol- low instructions with human feedback. Advances in neural information processing systems, 35: 27730-27744, 2022.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763. PmLR, 2021.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp.
2022.
Seyedmorteza Sadat, Jakob Buhmann, Derek Bradley, Otmar Hilliges, and Romann M Weber. Cads: Unleashing the diversity of diffusion models through condition-annealed sampling. arXiv preprint UrXiv:2310.17344 2023.
Raghav Singhal, Zachary Horvitz, Ryan Teehan, Mengye Ren, Zhou Yu, Kathleen McKeown, and Rajesh Ranganath. A general framework for inference-time scaling and steering of diffusion models. In Forty-second International Conference on Machine Learning. 2025.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In Interna- tional Conference on Learning Representations, 2023.
Imagen Team, Jason Baldridge, Jakob Bauer, Mukul Bhutani, Nicole Brichtova, Andrew Bunner, Lluis Castrejon, Kelvin Chan. Yichang Chen, Sander Dieleman, Yuqing Du, et al. Imagen 3. arXiv preprint arXiv:2408.07000 2024.
12

---

Published as a conference paper at ICLR 2026
Xi Wang, Nicolas Dufour, Nefeli Andreou, Marie-Paule Cani, Victoria Fernández Abrevaya, David Picard, and Vicky Kalogeiton. Analysis of classifier-fre guidance weight schedulers. arXiv preprint arXhv:2404.130400 2024.
Xiao Wang, Ibrahim Alabdulmohsin, Daniel Salz, Zhe Li, Keran Rong, and Xiaohua Zhai. Scaling pre-training one hundred billion data for vision language models. arXiv preprint arXiv:2502.07617, 2025.
Olivia Wiles, Chuhan Zhang, Isabela Albuquerque, Ivana Kajić, Su Wang, Emanuele Bugliarello, Yasumasa Onoe, Chris Knutsen, Cyrus Rashtchian, Jordi Pont-Tuset, al. Revisiting text- to-image evaluation with gecko: On metrics, prompts, and human ratings. arXiv preprint arXiv:2404. 16820, 2024.
Yilun Xu, Mingyang Deng, Xiang Cheng, Yonglong Tian, Ziming Liu, and Tommi Jaakkola. Restart sampling for improving generative processes. Advances in Neural Information Processing Sys- tems, 36:76806676833 76838, 2023.
Michal Yarom, Yonatan Bitton, Soravit Changpinyo. Roce Aharoni, Jonathan Herzig, Oran Lang, Eran Ofek, and Idan Szpektor. What you see is what you read? improving text-image alignment evaluation. Advances in Neural Information Processing Systems, 36, 2024.
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 11975-11986, 2023.
A APPENDIX
A.1 LATENT EVALUATORS
Training. We initialize the latent alignment (CLIP) evaluator with pre-trained CLIP model trained on the WebLI dataset. We use a pre-trained CLIP-ViT-B/16 (Radford et al. 2021: Zhai
al. 2023) model version with a ViT-B vision encoder and BERT-Base (Devlin al., 2019 text encoder. The dual encoder has in total 194M parameters.
we randomly initialize the embedding layer of the vision encoder in or- der to change the pixel-space embedding layer to a diffusion-specific latent-space one. Specifically, for LDM we convert ViT-B/16 to VIT-B/4 resulting in a 256 token sequence for an image with initial resolution of (512, 512) encoded into latents. Accordingly, Imagen 3. We then fine-tune the whole model on noisy diffusion latents encoded and corrupted from image-text papers of the WebLI dataset. We fine-tune the model for 90k steps using batch size of 512. We use a cosine learning rate schedule with linear warm up and no weight decay, Our base learning rate 5e -5, We train our model on 64 TPUv5e chips for 1.5 days.
As mentioned in Section 3.2
also change the embedding layer for
We initialize all other latent evaluators with the above latent alignment evaluator and continue fine- tuning the whole network for approximately 10k steps on the capability-specific data described in Section 3.2 and summarized in T
Table 4: Training data per latent evaluator.
Latent evaluator
Training data
Alignment evaluator
WebLI (Chen et al. 2023b)
Visual quality evaluator
Real generated images from MSCOCO (Lin et al. 2014)
Reward evaluator
Human preference data on generated images OCR scores on generated images
Text rendering
Numerical reasoning
100K re-captioned image-text pairs by Gemini 2.5 Pro for accurate descriptions of object counts
We observe that for the reward and text rendering evaluators, which measure fine-grained qualities
in image generation, a useful signal only emerges for timesteps < Emin + (ttma fmin). Conse- quently, during the initial high-noise phase of generation (t Imin + 3/2 (Imax (min)) apply
13

---

Published as a conference paper at ICLR 2026
near-zero weight to their corresponding loss. For the subsequent phase < Ein (fmax - -min). ) as the noise level decreases, we increase the loss weight. We experiment with schedules where this up-either
weight ramps up- either linearly exponentially-fromm from its initial low value, reaching maximum of at the final timestep (t min))
0.05
if Emsn + 3 (tmax tmin)
Wloss (t)
(8)
0.05 + 0.95
otherwise
where tmax the timestep corresponding to pure noise, tmin corresponds to clean data, a ((mmaz-fman) Imax+22mm and h is a hyper-parameter defining the sharpness of the curve
3
which we
A.2 DYNAMIC CFG SEARCH
CFG values. We find that the best default (fixed) value for both LDMsmall and LDMtarge is 7.5. For our dynamic CFG search, we are searching over the following set of CFG values:
[1,3, 7.5, 11, 15] for all denoising timesteps. For Imagen extend our search set of 24 discrete CFG values.
A.3 COMPUTE
We report FLOPs for different model functions (i.e., denoising, decoding, online evaluation) and for the full denoising process for the LDM model in Table
We overall use evaluators that are small and lightweight in order be computationally efficient in our online sampling setting. By operating in the latent space directly we use latent CLIP model which is 4 times more efficient than the pixel-space equivalent due cially, when using a latent evaluator, we do not require decoding the latents via the VAE at each denoising step. This reduces the computational cost from times more than the baseline for the pixel-space evaluator, which is prohibited, to only 1% of the overall computation required for sam- pling from LDMtarge-
compressed inputs. Cru-
Table 5: Comparison of FLOPS per model function.
Model
FLOPS x10
LDMamall denoising step
875
LDMtarge denoising step VAE-decode
2280
1489
I Latent alignment evaluator
5
Pixel-space alignment evaluator
22
LDMtarge: baseline sampling
115,489
LDMtarge: sampling with latent evaluator
116,739
LDMtarge: sampling with pixel-space evaluator
493,239
A.4 HUMAN EVALUATION
We recruited participants (N = 60) through an internal crowdsourcing pool. The full details of our study design, including compensation rates, were reviewed by our institution's 's independent ethical review committee. All participants provided informed consent prior to completing tasks and were reimbursed for their time. We collect and aggregate average two to three ratings per prompt- image pair, considering both the wins of each model and the ties in the ratings.
For the Gecko and GenAl-Bench prompt sets, display generated images by different model variants side-by-side for the same prompt and ask raters to indicate which one they overall prefer in terms of both aesthetics and prompt adherence (the options to indicate one images). For the MvRIO-eval prompt set. we again display the generated images bide-b asking the raters indicate the they prefer in terms of text rendering, i.e., which better
of the
14

---

Published as a conference paper at ICLR 2026
Table 6: Filtering performance. We report FID while filtering samples of poor visual quality at different % during sampling. For filtering, we use the visual quality evaluator and select the best out of batch of when filtering. Computed on the MS COCO prompt set.
Filter @ [FID 4]
Model
Noisy evaluator
Baseline
25%
50%
75%
100%
gLDMae latent Disc
29.2
27.6
27.4
27.0
26.8
visualizes the text requested by the prompt. Finally, for GeckoNum, we ask the raters to indicate the generated image of the two that better reflects the number of objects or entities described in the prompt.
A.5 ADDITIONAL EXPERIMENTAL RESULTS
Evaluation of Latent Evaluators. Additional to the results of Table when using the alignment evaluator, we report the filtering performance of the latent visual quality evaluator on LDM in terms of FID on the Gecko prompt Table 6 We validate that the latent visual quality can correctly predict bad samples from as early as 25% offering improvements over the baseline.
CFG Greedy Selection vs. Global Optimization. While our method utilizes greedy selection strategy to determine the optimal CFG scale at each timestep, we acknowledge that this does not theoretically guarantee global maximum for the final sample quality. However, the stability of our latent evaluators supports this approximation; detailed in Section 5.1] these evaluators are robust predictors final image alignment, maintaining consistent predictions even during early denoising stages (e.g.. from 25% completion). This approach parallels greedy decoding strategies commonly used in LLM inference, which prioritize efficiency and provide strong approximate solutions without the exhaustive cost of global search.
To empirically validate the effectiivness of our greedy approach against a more exhaustive search, implement& beam search strategy. At each denoising step t. instead of selecting single optimal trajectory, this method maintains the top-k candidates. For each candidate i, we generate potential next states æt_, across range of CFG values and prune the set retain only the top-k paths for the subsequent step.
The primary constraint of beam search is the computational penalty; specifically, beam width of scales the NFES by a factor of k. We compare our standard greedy approach against beam search with k 2 (doubling the inference cost) on Imagen 3 using the Gecko prompt set. Utiliz- ing the latent CLIP evaluator for guidance and the Gecko metric for final quality assessment, we observe that beam search yields marginal improvement, achieving a score of 73.7% compared to 73.1% for our greedy method. Given that the 0.6% performance gain requires a 100% increase in NFEs, we conclude that greedy selection offers the superior trade-off between generation quality and computational efficiency.
A.6 QUALITATIVE EXAMPLES
Qualitative Analysis on LDM. Figure provides qualitative comparison between the default CFG and our dynamic approach on LDM, showcasing the effects of each latent evaluator. As the ex- amples illustrate, the individual evaluators successfully target their respective domains but introduce trade-offs. Guiding with the discriminator alone enhances photorealism- for instance, improving the panda's fur texture Example 2-but does so at the expense of prompt alignment, causing the koala from the prompt to disappear. Conversely, using only the CLIP evaluator enforces stronger prompt adherence, correctly adding glasses to the tiger in Example 1, but often at the cost im- age quality and coherence, resulting "pasted-together"" artifact. Our full method with adaptive weighting successfully resolves this tension, synthesizing the strengths of both evaluators to produce images that are both photorealistic and faithful the prompt.
15

---

Published as a conference paper at ICLR 2026
(a) Default CFG.
(b) Dynamic CFG (Disc). (c) Dynamic CFG (CLIP). (d) Dynamic CFG (CLIP
Disc).
Prompt: "the tiger wears glasses and wears paisley tie "
(e) Default CFG.
(1) Dynamic CFG (Disc). (g) Dynamic CFG (CLIP). (h) Dynamic CFG (CLIP
Disc).
Prompt: "the panda waves to the koala bear" T
Figure 5: Qualitative examples for LDM when using different CFG schedules the Gecko prompt set. The images of the first row are generated for the prompt: "the tiger wears glasses and wears paisley tie" and the images of the second row generated for the prompt: "the panda waves the koala bear".
Qualitative Improvements on Imagen 3. Next, in Figure 6 we demonstrate our method's ability to improve upon the already powerful Imagen baseline. The qualitative improvements are most striking in areas where even sfate-of-the-ar models can falter. Our dynamic CFG approach consis- tently reduces subtle visual artifacts, improves overall text alignment and, most notably, produces significantly more coherent and legible rendered text than the default sampler. This highlights our method's value not only for enhancing general quality but also tool for targeted improvements on specific, challenging generation tasks.
A.7 LLM USE DISCLOSURE
LLM was used for polish writing of the paper and improving the phrasing certain sentences. No LLM was used to write extended parts of the paper from scratch, or for retrieval, discovery and research ideation.
16

---

Published as a conference paper at ICLR 2026
Arifacts (Gecko).
(a) Prompt: "An orange is being squashed under a glass bottle which is splintering into bits."
Text alignment (GenAl-Bench),
(b) Prompt: "There are two bananas in the basket, but no apples."
Text rendering (MARIO-eval).
SAFETY FIRS
SAFETY A FIRST
(c) Prompt: "In the factory, sign that reads "Safety Finst""
Numerical reasoning (GeckoNum).
(d) Prompt: "5 cookies."
Figure 6: Qualitative examples for Imagen on the Gecko prompt set when using different CFG schedules: default (left) ours dynamic (right). We observe improvements alignment, artifacts, text rendering, and numerical reasoning.
17


Reviews:
[{"review_id": "EOFrDRc0AQ", "reviewer": "Reviewer_67qG", "summary": "This paper introduces a dynamic Classifier-Free Guidance (CFG) scheduling framework for text-to-image diffusion models, replacing the traditional static, one-size-fits-all guidance scale with adaptive, prompt-specific schedules. The method uses latent-space evaluators (e.g., CLIP, discriminator, and human preference reward models) to provide real-time feedback on alignment, fidelity, and reasoning quality during the diffusion process. A greedy search dynamically selects the optimal CFG scale at each timestep, yielding schedules that adapt to both the prompt and diffusion stage. Experiments on StableDiffusion-like models and Imagen 3 show substantial gains in text alignment, visual quality, and human preference, proving that optimal CFG is inherently dynamic and prompt-dependent.", "questions": "It is unclear what $W$ represents in Equation (6); the authors should explicitly define it.\n\nThe relationship between the guidance scale $s$ and the evaluator score $e_t$  needs clarification. Is $e_t$ defined as an aggregate across multiple evaluators?\nIf $e_t$ is indeed defined as the average over different evaluator scores, this should be explicitly stated, along with the precise formulation.\n\nThe notation $E$ in Equation (7) is also ambiguous. The authors should clarify this as well.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 1, "presentation": 2, "contribution": 1, "strengths": "The authors identify a interesting research question: how to improve generated images across multiple aspects such as alignment, fidelity, et, by dynamically adapting the Classifier-Free Guidance (CFG) scale instead of relying on a static, fixed value.", "weaknesses": "1. The approach heavily depends on CLIP image encoder fine-tuning under noisy latent conditions, which may distort the original embedding space; however, the paper provides neither validation nor ablation studies to justify or quantify this effect.\n\n2. The experimental comparison is limited to conventional baselines. For example, [1] introduces a test-time optimization strategy designed for multi-reward settings. The authors may need to compare their method with this. In additions, considering that the method employs multiple specialized evaluators (e.g., human preference, counting) and substantial fine-tuning, it can also be compared against multi-objective RL-based diffusion approaches, which incur similar computational costs.\n\n3. The assumption that a greedy-selection-based dynamic CFG universally enhances performance across different aspects lacks both theoretical grounding and intuitive justification; additional analysis or formal reasoning is necessary.\n\n4. The overall writing quality could be improved for clarity, coherence, and readability.\n\n[1] Test-time Alignment of Diffusion Models without Reward Over-optimization, Kim et al., ICLR 2025", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kab5bMvcoB", "reviewer": "Reviewer_ik4p", "summary": "This paper proposes an approach, Dynamic CFG, for dynamically choosing the classifier-guidance scale at each individual timestep using a series of online evaluators. By choosing the optimal cfg-scale with respect to the alignment/visual quality/human preference score, Dynamic CFG improved overall generation quality with adaptive evaluators’  weight method.", "questions": "1. While the online evaluator is lightweight and the inference cost is low, and the current choice of five CFG-scale candidates proves effective, it remains unclear how much additional performance gain could be achieved by introducing more candidate scales.\n\n2. Transferability: For the latent evaluator, can it be transferred to other models trained in the same latent space?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "**Clear motivation and well-written:** The proposed method is clear motivated and the writing of this paper is great. Most of terms have been clearly explained. It is conformable to this paper. \n\n**Lightweight evaluator design:** The latent evaluator is lightweight and can be seamlessly integrated into existing diffusion models without significantly increasing computational cost.\n\n**Strong effectiveness:** The method demonstrates consistent improvements across multiple benchmarks, and it could enhance text alignment, visual quality, and human preference win rates on strong baselines like LDM and Imagen 3.", "weaknesses": "**High training cost for evaluators:** The method requires training multiple specialized latent evaluators with dedicated training datasets. This increases the overall implementation cost and limits the practicality of applying the approach to new tasks (e.g., when user wants to enhance the layout alignment).\n\n**Dependence on evaluator quality:** The performance of the proposed method heavily relies on the accuracy and generalization ability of the evaluators. One poorly trained evaluator may lead to unstable or suboptimal guidance schedules. The authors could add more discussion for this question to increase the transparency of the proposed method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ot5F1lgzK3", "reviewer": "Reviewer_3AEj", "summary": "* The paper introduces a framework for dynamic CFG in T2I diffusion models.\nUnlike conventional static CFG scales or fixed heuristic schedules, the proposed method adaptively adjusts the CFG scale at each sampling step using online feedback from multiple evaluators.\n* These evaluators include a CLIP-based alignment model, a discriminator for visual fidelity, a human preference reward model, and specialized evaluators for text rendering and numerical reasoning.\n* The framework employs a greedy search to select the optimal CFG scale per timestep, generating a prompt-dependent schedule.\n* Experiments on both SD models (LDM) and the Imagen 3 show consistent improvements across alignment, visual quality, text rendering, and counting tasks.", "questions": "* How does the method ensure temporal consistency of guidance across timesteps given the greedy selection strategy?\n* Could a differentiable or policy-gradient optimization approach yield smoother and potentially superior CFG schedules?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "* The idea of making CFG dynamic and feedback-driven is interesting.\nIt challenges the assumption that CFG scales should remain static during generation.\n* Operating in the latent space keeps the method lightweight, making it feasible for real-world use.\n* The framework transfers successfully from diffusion models to large-scale proprietary ones like Imagen 3, showing robustness and adaptability.\n* The authors assess their method across diverse benchmarks combining quantitative metrics and human preference studies.\n* Visualizations of dynamic schedules clearly show how guidance strength varies with prompt complexity, enhancing transparency and interpretability of the approach.", "weaknesses": "* I believe per-step greedy search does not guarantee global optimality.\nThe method assumes independence between timesteps, which may overlook temporal correlations in the diffusion trajectory.\n* The comparisons may be a bit limited.\nIncluding stronger recent baselines, like RL-based or multi-step optimization methods (FK steering or diffusion RL), would make the results more convincing.\n* In my understanding, each evaluator must be re-trained for the specific diffusion model, which weakens the claim of being fully model-agnostic.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7OqLbLgBfw", "reviewer": "Reviewer_Es1L", "summary": "The paper introduces a new method to **dynamically regulate** the classifier-free guidance (**CFG**) scale in text-to-image diffusion models. At each denoising time step a guidance scale is greedily selected among a predefined set based on the feedback scores coming from different **evaluators**. These evaluators take in input a time step $t$ and a latent $x_t$ and evaluate: (a) the alignment to the text prompt, (b) the visual quality of the output, (c) the alignment w.r.t. human preferences, and optionally the quality of (d) text rendering or (e) numerical reasoning. The method is evaluated by experimenting different base models (LDM and Imagen 3) and different text prompt datasets (Gecko, GenAI-Bench, MARIO, GeckoNum). Both quantitative and qualitative results overall prove the effectiveness of the method and of each evaluator.", "questions": "Following the weakness listed above, I would like to ask these questions:\n1. Lines 216–220 could be misread as computing all CFG outputs in parallel and assembling them afterward. Could this part be explained better?\n2. Section 5.1 is named *Evaluation of latent evaluators*, but in practice only evaluates the performances of the latent CLIP evaluator and not of the others. Would it be possible to provide a justification of this choice?\n3. Some minor issues:\n    - Equation 6 refers to $W$ which, if I am not wrong, is never defined before. May this be $S$?\n    - In Equation 7 the alpha coefficients could potentially assume negative values. Should there be an absolute value around the difference between $e_{t}$ and $e_{t+1}$?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. **Interesting use of the CFG scale.** I find the idea of using CFG scale to improve the alignment and the quality of text-to-image generative models very interesting and not too explored in the research community. I think the paper would definitely provide value to the ICLR community and inspire similar research directions. \n2. **Interesting experiments.** In addition to the main contribution, I find the experiments reported in Figure 2 very interesting as they show how different CFG scales influences different metrics at different denoising time steps.\n3. **Clear and well-structured presentation.** The paper is well structured and easy to follow.", "weaknesses": "1. **Ambiguity in algorithm description.** Lines 216–220 could be misread as computing all CFG outputs in parallel and assembling them afterward. Clarifying that the selection at each step influences subsequent sampling would be helpful.\n2. **Partial evaluation of latent evaluators.** Section 5.1 reports detailed analysis only for the CLIP evaluator; others (e.g. discriminator) are not explicitly benchmarked.\n3. **Model-agnostic claim.** I do not necessarily agree with the claim that the method is *model-agnostic* (line 428). Similarly to how baselines which rely on heuristics have to run hyper-parameter search to find the optimal CFG scale for a specific model, the proposed method also has to train the evaluators on the noisy latents coming from each specific model.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)