Analysis of Classifier-Free Guidance Weight Schedulers
Authors: Xi WANG
Nicolas Dufour
Nefeli Andreou
Marie-Paule CANI
Victoria Fernandez Abrevaya
David Picard
Vicky Kalogeiton
Venue: Accepted by TMLR ()

Abstract: Classifier-Free Guidance (CFG) enhances the quality and condition adherence of text-to-image diffusion models. It operates by combining the conditional and unconditional predictions using a fixed weight. However, recent works vary the weights throughout the diffusion process, reporting superior results but without providing any rationale or analysis. By conducting comprehensive experiments, this paper provides insights into CFG weight schedulers. Our findings suggest that simple, monotonically increasing weight schedulers consistently lead to improved performances, requiring merely a single line of code. In addition, more complex parametrized schedulers can be optimized for further improvement, but do not generalize across different models and tasks.

Published in Transactions on Machine Learning Research (12/2024)
Analysis of Classifier-Free Guidance Weight Schedulers
Xi Wang', Nicolas Dufour Nefeli And Marie-Paule Cani', A Abrevaya', David Picard Vicky Kalogeiton"
Victoria Fernández
LIX, École Pobytechnique, CNRS, IPP
forsnname lostenmme@pllltchhiiieeee edu
LIGM, École des Ponts, Unáv Gustave Biffel, CNRS
forstname, lastaeeeettttteeeeeeaae. fr nefeliandroou Soutlook. com victoria, abreveya Stuchiggennnnnnnnnn..umm mpp..de
University of Cyprus
Max Planck Institute for Intelligent Systems, Tübingen
Work done during internship at LIX, prior to joining Amazon.
Denotes equal supervision
Reviewed on OpenReview: httpss//ppeeeevvveeerr///oooooop
Abstract
Classilier-Free Guidance (CFG) enhances the quality and condition adherence of text-to- image diffusion models. It operates by combining the conditional and unconditional pre- dictions using a fixed weight. However, recent works vary the weights throughout the dif- fusion process, reporting superior results but without providing any rationale or analysis. By conducting comprehensive experiments, this paper provides insights into CFG weight schedulers. Our findings suggest that simple, monotonically increasing weight schedulers consistently lead to improved performances, requiring merely a single line of code. In addi- tion, more complex parametrized schedulers can be optimized for further improvement, but do not generalize across different models and tasks.
1 Introduction
Diffusion models have demonstrated prominent generative capabilitics in various domains e.g. images et al. 2020), videos (Luo et al., 2023), acoustic signals (Kang et al. 2023b), or 3D avatars (Chen et al.
HHo
2023 Conditional generation with diffusion (e.g. text-conditioned image generation) has been explored in
numerous works (Saharia et al. 2022 Ruiz et al.] 2023 Balaji et al. 2022), and is achieved in its simplest
form by adding an extra condition input to the model (Nichol & Dhariwal 2021]. To increase the influence of
the condition on the generation process, Classifier Guidance (Dariial & Nichol 2021) proposes to linearly combine the gradients of a separately trained image classifier with those of diffusion model. Alternatively, Classifier-Free Guidance (CFG) (Ho & Salimans 2021) simultaneously trains conditional and uncondit ional models, and exploits Bayesian implicit classifier to condition the generation without an external classifier.
In both cases, weighting parameter controls the importance of the generative and guidance terms and is directly applied at all timesteps. Varying w is a trade-off between fidelity and condition reliance, as an increase in condition reliance often results in a decline in both fidelity and diversity. In some recent literature, the concept of dynamic guidance instead of constant one has been mentioned: MUSE (Chang et al. 2023) observed that a linearly increasing guidance weight could enhance performance and potentially increase diversity. This approach has been adopted subsoquent works, such as Stable Video Diffusion (Blattmann) et al.) )023), and further mentioned in Gao et al. (2023) through an exhaustive search for a parameterized cosine- based curve (pes4) that performs very well on specific pair of model and task. Intriguingly, despite the recent appearance of this topic in the literature, none of the referenced studies has conducted any empirical experiments or analyses to substantiate the use of a guidance weight scheduler. For instance, the concept of linear guidance is briefly mentioned in MUSE (Chang el al., 2023), around Eq. 1: 'we reduce the hit to diversity by linearly increasing the guidance scale t [...] allowing early tokens to be sampled more freely". Similarly, the pcs4 approach Gao et al.] (2023) is only briefly discussed the appendix, without any detailed ablation or comparison static guidance basclines. Thus, to the best of our knowledge, a comprehensive guide to dynamic guidance weight schedulers does not exist at the moment.

---

Published in Transactions on Machine Learning Research (12/2024)
In this paper, we bridge this gap by delving into the behavior of guidance and systematically examining its influence on the generation, discussing the mechanism behind dynamic schedulers and the rationale for their enhancement. We explore various heuristic dynamic schedulers and present comprehensive benchmark of both houristic and parameterized dynamic schedulers across different tasks, focusing on fidelity, diversity, and lextual adherence. Our analysis supported by quantitative, and qualitative results and user studies.
Low static guidance:
Wi 2.8
for in range(1, T):
eps modelix, T-t, c)
eps u mxdellx, 1-t, 8) eps her]]"eps__ c wdeps_ x denoisetx, 895, T-5)
but details textures
High guidance:
14.0
for in range(1, 1):
eps modelix. 1-t, modelix.
her1)epps_ c xfeps_ denoiselx, 895, T-t)
Sharp inages, but lack of details and solid colors
Dynanic guidance:
=lll 14.8
for in range (1, T):
eps modelix. T-t, c)
eps modellx; T-t, 9)
clarp-linear scheduler
naxt1, w012*t/T)
[er])"ops__ c x
/ Sharp images with nany details and textures, without extra cost.
body. dressed tiking, waapees some tiking stip. bottle coloring, plaw hypor- typer
Figure 1: Classifier-Fres Guidance introduces trade-off between detailed but fuzzy images (low guidance, top) and sharp but simplistic images (high guidance, middle). Using guidance scheduler (bottom) is simple yet very effective in improving this tradc-off.
Our findings are the following: First, we show that too much guidance at the beginning of the denoising process is harmful and that monotonically increasing guidance schodulers are performing the best. Sccond, we show that simple linearly increasing scheduler always improves the results over the basic static guidance, while costing no additional computational cost, requiring no additional tuning, and being extremely simple to implement. Third, parameterized scheduler, like clamping linear scheduler below carefully chosen threshold (Figure D can significantly further improve the results, but the choice of the optimal parameter does not generalize across models and tasks and has thus to be carefully tuned for the target model and task. All our findings are guides to CFG schedulers that will benefit and improve all works relying on CFG.
2 Related Work
Gencrative and Diffusion Models. Before the advent of diffusion models, several generative models were developed to create new data that mimics a given dataset, either unconditionally with conditional guidance. Notable achievements include Variational AutoEncoders (VAFs) (Kingma & Welling 2014 and
2

---

Published in Transactions on Machine Learning Research (12/2024)
baseline
linear
cosine
Static
Increasing
inviinear
sine
A-shape
V-shape
Decreasing
Prompt: Storttrooee drinking coffee Zarie bar, with Eiffel Tower background.
Figure 2: Examples of all heuristics on SDXL. Increasing ones (linear and cosine) enhance fidclity, textual adherence and diversity.
Generative Adversarial Networks (GANs) Goodfellow et al. 2014). which have recorded significant progress in various generative tasks (Brock et al.
2018 Kang et al. 2023a Dufour et al. 2022: Donahue et al.,
2018). Recently, diffusion models have demonstrated remarkable capacity to produce high-quality and
diverse samples. They have achieved state-of-the-art results in several generation tasks, notably in image
synthesis (Song et al., 2020; Ho et al. 2020), text-to-image applications (Dhariwal & Nichol 2021 Rombach
ct al. 2022 Podell et al.] 2023 Pernias et al.) 2023) and text-to-motion (Chen et al. 2023).
Guidance in Diffusion and Text-to-Image. Making generative models controllable and capable of
producing uscr-aligned outputs requires making the generation conditional on a given input. Conditioned diffusion models have been vastly explored (Saharia el al. 2022: Ruiz et al., 2023 Balaji et al. 2022). The
condition is achieved in its simplest form by adding extra input, typically with residual connections (Nichol & Dhariwal 2021). To reinforce the model's fidelity to specific conditions, two main approaches prevail: Classilier Guidance (CG) (Dhariwal & Nichol 2021,, which involves training an image classifier externally, and Classifier-Free Guidance (CFG) (Ho Salimans 2021), , that relies on an implicit classifier through joint training of conditional and unconditional models (using dropout on the condition).
Particularly, CFG has catalyzed advancements in text-conditional generation, domain where training noisy text classifier is less convenient and performs worse. This approach breathed new life into the text-to-image application, initially proposed in several works such as (Reed et al., 2016 Mansimov et al.
2015). Numerous works (Rombach et al.) 2022 Ramesh et al.] 2022 Nichol et al. 2022 Avrahami et al, 2022 have leveraged texi-to-image generation with CFG diffusion models conditioned on text encoders like CLIP (Radford et al.] (021), showcasing significant progress the field, c.g. the Latent Diffusion Model (Dhariwal & Nichol 2021) and Stable Diffusson (Rombach et al., 2022) employ VAE latent space diffusion with CFG with CLIP encoder. SDXL, an enhanced version, leverages larger model and an additional text encoder for high-resolution synthesis.
Improvements on Diffusion Guidance. Noticed that in Classifier Guidance (CG), the classifier's gradi- ent tends to vanish towards the early and final stages due to overconfidence, Zheng et al. (022) leverages the entropy of the output distribution as an indication of vanishing gradient and rescales the gradient accord- ingly. To prevent such adversarial behaviours, Dinh et al. (2023b) explored using multiple class conditions, guiding the image generation from noise state towards an average of image classes before focusing on the
3

---

Published in Transactions on Machine Learning Research (12/2024)
Static
Linear
Cosine
Static
Linear
Casine
DкоMр:: flover sitting in a vase.
prompt: A corrider of pisk colored stose arches,
prompt: flying the sky weather.
:rompt: figurine
key
Figure 3:
Qualitative results of fidelity for different guidance schedulers compared with static baseline. linear and cosine schedulers show better image details (flower petal, figurine engraving); more natural colour (pink corridor), and better textual adherence (bad weather for the two birds image, key-chain of the figurine).
desired class with empirical scheduler. Subsequently, Dinh et al. 2023a) identified and quantified gradient conflicts emerging from the guidance and suggested gradient projection solution.
In Classifier-Fres Guidance (CFG). Li et al. (2023) used CFG to recover a zero-shot classifier by sampling across timesteps and averaging the guidance magnitude for different labels, with the lowest magnitude corre- sponding to the most probable label. However, they observed a discrepancy in performance across timesteps with early stages yielding lower accuracy than intermediate ones. Chang el. al. (2023) observed that a linear increase in guidance scale enhances diversity. Similarly, Gao al. (2023) developed a parameterized power- cosine-like curve, optimizing specific parameter for their dataset and method. However, these lincar and power-cosine schedulers have been suggested as improvements analysis or testing. To this end, we provide an extensive study of dynamic guidance for both heuristic and parametrized schedulers across several tasks. Concurrently, Kynkäänniemi et al. (2024) proposes empirically removing the initial and final timesteps of the classifier-fi guidance (CFG) for improved generation. Simi-
constant static guidance without rigorous
larly, Zhang et al. [2024]; Castillo et al. (2023) observes that the conditional and unconditional responses of some models may converge similar behaviours at certain timesteps, particularly towards the ending stage.
3 Background on Guidance
Following DDPM Ho et al. 2020 diffusion consists in training a network eg to denoise a noisy input to recover the original data at different noise levels. More formally, the goal to recover 20, the original image from (f) / (t)c,,, where (0, is a noise scheduler of the timestop and applied to a standard Gaussian noise e~~N(0, 1). In practice, Ho et al. (2020) find that predicting the noise ep instead of 2o yielded better performance leading to the training loss: Lstmple=B. (0
(llee(z)) - <ll
based the target image distribution Pdata with U uniform distributions.
Once the network is trained, we can sample from Pasta by setling xr=e ~ NO 1) (with -())=0), and gradually denoising to reach the data point PorPPaata with different types of samplers e.g., DDPM Ho et al. 2020) or DDIM (Song et al. (020).. To leverage a condition e and instead sample from p(x,(c), Dhariwal Nichol 2021 h propose Classifier Guidance (CG) that uses pretrained classifier p(c|æ:), forming:
Va, logp(1a/c)= logg(o:)+ (onr:/)))_ log logp(c(xx)
(1)
according to Bayes rule. This leads to the following Classifier Guidance equation, with scalar w>0 con- trolling the amount of guidance lowards the condition
C0 c) co(x)) + (w 1) logp(c)zz)
(2)
4

---

Published in Transactions on Machine Learning Research (12/2024)
baseline
linear
cosine
SD1.5
prompt: : brown teddy bear lying on sand with hat.
SDXL
prompt: an as astraunaut walking in the jungle, in the night, photorealistic.
Figure 4: Qualitative results of diversity of different guidance schedulers compared with static baseline. Heuristic schedulers show better diversity: more composition and richer background types for the teddy bear example, well as the gesture, lighting, colour and compositions in the astronaut image.
However, this requires training a noise-dependent classifier externally, which can be cumbersome and im- practical for novel classes or more complex conditions e.g. textual prompts. For this reason, with an implicit classifier from Bayes rule Vz, log log p(c[z,)=VV logp(z,, v) log p(zz), Ho & Salimans (2021) propose to train diffusion network on the joint distribution of data and condition by replacing co (a(x)) with co co c) Liuplee- By dropping the condition during training, they employ a single network for both Va, logp(z;; c) and Vz, log)( This gives the Classéfier-Fre. Guidance (CFG), also controlled by wt
(0( = cz), + w (co(#e, (g(x)))
(3)
We can reformulate the above two equations into two terms: a generation term cę(x)) logp(r;) and a guidance term Va, p(cl() The guidance term can be derived either from a pre-trainod classifier or an implicit one, with w balancing between generation and guidance.
4 Towards dynamic guidance: Should guidance be constant?
Our initial experiments show that removing guidance at certain timesteps can improve performance. This is line with concurrent work (Kynkäānnieni et al. (024)). To further investigate this, we conducted a negative perturbation analysis experiment determine the impact of the guidance across all timesteps.
Negative Perturbation Analysis. This analysis is the CIFAR-10 dataset: 60,000 images dataset with a 32 x 32 resolution, distributed across 10 classes. We choose the original DDPM method (Ho et al. 2020) denoising on pixel space as the backbone and class-conditioning guidance.
To investigate the importance of guidance across different timestep intervals, we first employ static guidance of w 1.15, then independently set the guidance to zero across different 50-timestep intervals (20 intervals total spanning all timesteps), and compute the FID for each of these piece-wise zeroed guidance schedulers of 50,000 generated images. If we mathematically model the removal method, it can approximate family
5

---

Published in Transactions on Machine Learning Research (12/2024)
of parameterized gate/inverse gate functions with two parameters defining the starting point and size of the kernel d: s(t) = - (H(t () - H[ (s + d))), where H is Heaviside step function.
The results are illustrated in Figure 6b For example, the second data point on the left of the curve represents the FID performance when guidance is removed only in the interval = [50, 100] while maintaining static at others. We observe multiple phenomena from the results: (1) non-constant guidance curve can outperform static guidance in terms of FID; (2) zeroing the guidance at earlier stages improves the FID performance; (3) zeroing the guidance at later stages significantly degrades it.
However, this removal scheme is not practical for real usage: (i) grid searching two parameters requires generating a prohibitively costly number of images; (ii) as shown in Section 6 parameterized methods often fail to generalize; (iii) further investigation, detailed in Appendix Section B demonstrates that instead of
completely removing CFG from some timesteps, keeping with lower values increases the performance.
(i) Conflitt Ratio (Penestags)
(0) Aog. Cuaine Smilucit
[88] Aug. Coaflet Magaltude
0.04
0.02
Percottae
SD [A]]
-1115
linour
0.2-
933 Timosdep
T Timestoge
Timostops
Figure 5: Visualization of Conflicted Terms from SD1.5 Rombach et al. (2022) shows that static guidance presents conflicts, while a guidance scheduler reduces the conflict between generation and guidance terms.
Conflicted terms. Our assumption is that the guidance and generation terms (see Eq. 3 may be adversarial during inference. Following (Dinh et al. 2023ab, Figure quantifies the conflict by measuring (a) the ratio of negative cosine similarity; (b) average cosine similarity (i.e. directional conflict, -1 and 1 for maximum and - where minimum conflicts); and (c) conflict magnitude (Din et al. 2023a), defined as: (E1, E2) Te: 3+1eal2
is each term at each timestep, with resulting and O indicates zero and maximum conflict. We evaluate 1000 generation from COCO prompts (Lin et al. 2014) with SD1.5 and show in Figure 5 that SD (orange) exhibits ~ 50% conflict ratio along the generation with high magnitude conflict (scc Figure 5 (right)) with curves eloser to zero). When the guidance is lowered at the beginning (e-g. linearly increasing as shown in Section less conflict for both magnitude and directional metrics is shown in all subfigures blue curves.
Dynamic guidance. Having obscrved that removing the guidance at certain timesteps improves the per- formance over using a static weight w for CFG like in Ho & Salimans) (2021); Dhariwal & Nichol (2021) and reducing the guidance at beginning linearly can reduce the conflict, we ask the question of whether we can replace static guidance with other options. Therefore, we investigate dynamic guidance scheduler that evolves throughout the generation process, which is also in line with some empirical schemes mentioned
recent. literature (Blattmann al. 2023 Chang al.) 2023
Donahue al., 2018). In that case, the CFG
Equation 3 is rewritten as follows:
(a( c) eo ex c) ((( (co( ea (xx))
To identify an effective dynamic schoduler w(t), we analyse two types of function subsoquent scctions: parameter-free heuristic schedulers in Section and
parameterized ones in Section D
5 Dynamic Guidance: Heuristic Schedulers
We first use six simple heuristic schedulers as dynamic guidance w(t), split into three groups depending on the shape of their curve: (a) increasing functions (linear, cosine); (a) decreasing functions (inverse linear,
6

---

Published in Transactions on Machine Learning Research (12/2024)
Heursites Weights va Timesteps
Heuristics FID vs IS
Negutive on CIEMR-10
Trwee
Aaaape
Taser ieliaca osiux
Ashlape V-slape
arrlacar uniise
Voskeo
regaiin pertulatioon
2.0
3.2-
Weigles
1.5
3.51
1.0
200
600
0.0
900
1000
9.6
9.7
9.5
Timestope
Inceptionn
Timmstep
(a) Experiment on Different Heuristics
(b) Negative Perturbation
Figure 6: Preliminary Analysis on CIFAR-10 (a) Various heuristic curves with their FID vs. IS performances. (b) Negative perturbation by setting the guidance scale across distinct intervals while preserving static guidance to the rest. By eliminating the weight. at the initial stage (e.g., T 800), the lowered FID shows an enhancement, whereas removing guidance at higher timesteps leads to worse FID.
sine): (c) non-monotonic functions (lincar V-shape, linear A-shape), defined as:
linear: wt) t/T,
invlinear: w(t) t/T,
cosine: w(t) cos (Tt/ rt) + 1,
sine: w(t) = sin (st/ rtt/TT - x/22) + 1,
V-shape: w(t) invlinear(t) ift < 1/2,
A-shape: w(t) linear(t) (t) ift < T/2,
linear(t) else,
invlinear(() edse.
To allow for a direct comparison between the effect of these schedulers and the static guidance w, we normalize each scheduler by the area under the curve. This ensures that the same amount of total guidance is applied over the entire denoising process, and allows users to rescale the scheduler to obtain a behavior similar to that of increasing w in static guidance. More formally, this corresponds to the following constraint: Io dt) = wf. For example, this normalization leads to the corresponding normalized linear scheduler (t) = 2(1 t/T).. We show in Figure 6a (left) the different normalized curves of the schedulers.
5.1 class-conditional image generation with heuristic schedulers
Heuristic Schedulers Analysis. We first study the 6 previously defined heuristic schedulers w()) on the CIFAR-10-DDPM setting for class-conditional synthesis same as the Section g To assess the performance, we use the Frechet Inception Distance (FID) and Inception Score (IS) metrics, over 50, 000 inference from 50-step DDIM (Song et al. 2020 In this experiment, total weight: [1.1, 1.15, 1.2, 1.25, 1.3, 1.35), to study its influence over the image quality class adherence trade-off. We show the results in Figure 6a. right panel and observe that both increasing schedulers (linear and cosine) significantly improve over the static baseline, whereas decreasing schedulers (invlinear and sine) are significantly worse than the static. The V-shape and A-shape schedulers perform respectively better and worse than the static baseline, but only marginally.
evaluate the impact of range of different guidance
Preliminary Conclusion. Combining with the observation from Section 4 that removing the beginning stage improves the performance, they point to the same conclusion: monotonically increasing guidance schedulers achieve improved performances, revealing that the static CFG primarily may overshoot the guidance in the initial stages. In the remainder of this work, we only consider monotonically increasing schedulers, as we consider these findings sufficient to avoid examining all other schedulers on other tasks. (more details in Appx. and Figure 2 shows a qualitative results in SDXL)
Experiments on ImageNet. On ImageNet, we explore the linear and cosine schedulers that performed best. CIFAR-10. In Figure 7 observe that the linear and cosine schedulers lead to significant

---

Published in Transactions on Machine Learning Research (12/2024)
improvement over the bascline, especially at higher guidance weights, enabling a better FID/Inception Score trade-ofT. More experiments in Appx. lead to similar conclusion.
5.2 Text-to-image generation with heuristic schedulers
We study the linear and the cosine scheduler on texl-lo-image generation. The results for all proposed heuristics are in Appx. Tables 12 and [4 where observe a similar trend as before: heuristic functions with incrcasing shape report the largest gains on both SD1.5 and SDXL.
Dataset and Metrics. We use text-lo-image models pre-trained on LAION (Schuhmann el al. 2022 which contains 5B high-quality images with paired textual descriptions. For evaluation, we use the COCO Lin
et al. 2014) val set with 30, 000 text-image paired data.
We use three metrics: (i) Fréchet inception distance (FID) for the fidelity of generated images; (ii) CLIP- Score (CS) (Radford et al. 2021 to assess the alignments between the images and their corresponding text prompts; (iii) Diversity (Div) to measure the model's capacity to yield varied content. For this, we compute the standard deviation of image embeddings via Dino-v2 (Oquab et al. 2023) from multiple generations of the same prompt (more details for Diversity in Appendix).
We compute FID and CS for a sample set of 10, 000 images against the COCO dataset in a zero-shot Tashion (Rombach et al. 2022 Saharia et al.j (022).. For diversily, we resort to two text description subsets from COCO: 1000 longest captions and shortest captions each (-L and -S in Figure (7)) to represent varying descriptiveness levels; longer captions provide more specific conditions than shorter ones, presumably leading to less diversity. We produce 10 images for each prompt using varied sampling noise.
2022 which uses the CLIP (Radford et al. 2021) text encoder to transform text inputs to embeddings. We use the public checkpoint of SD v1.5 and employ DDIM sampler with 50 steps. (2) SDXL (Podell et al. 2023 which is a larger, advanced version of SD (Rombach et al. (022). generating images with resolutions up to 1024 pixels. It leverages LDM (Dhariwal& & Nichol, 2021) with larger U-Net architectures, an additional text-encoder (OpenCLIP ViT-bigG), and other conditioning enhancements. We use the SDXL-base-1, base-bas--1(" (SDXL) version without refiner, sampling with DPM-Solver+++
Model. We experiment with two models: (1) Stable Diffusion (SD) (Rombach et al.
(Tuu et al.
2022b of 25 steps.
7c for SD and SDXL respectively (more tables in Appx. Section We expect an optimal balance of a high CS and a low FID (i.e., the right-down corner).
Results. We show the FID vs. CS curves in Figure 7a
Analysis on SD (Figure 7a). For FID vs CS, the bascline (Rombach et al. 2022) yields inferior results compared to the linear and cosine heuristics with linear recording lower FID. The baseline regresses FID fast when CS is high, but generates the best FID when CS is low, i.e., low condition level. This, however, is usually not used for real applications, e.g., the recommended w value is 7.5 for SD1.5, highlighted by the dotted line in Figure 7a with the black solid arrow representing the gain of hcuristic schedulers on FID and CS respectively. For Div vs CS, heuristic schedulers outperform the baseline (Rombach et al. 2022) on both short (S) and long (T) captions at different guidance scales. Also, cosine shows superiority across the majority of the CS range. Overall, heuristic schedulers achieve improved performances in FID and Diversity, recording 217((17%) gain on FID and 0.004(16%) gain (of max CS-min CS of baseline) on CS over w=7.5 default guidance in SD. Note, this gain is achieved without hyperparameter tuning or retraining.
Analysis on SDXL (Figure 7ch. In FID, both the linear and cosine schedulers achieve better FID-CS than the baseline (Podell et al. 2023). In Diversity, linear is slightly lower than cosine, and they are both better than static baseline. Additionally, unlike the baseline (blue curves) where higher guidance typically results in compromised FID, hcuristic schodulers counter this.
User study. We present users with pair of mosaics of generated images and ask them to vote for the best in terms of realism, diversity and text-image alignment. Each pair compares static baseline generations against cosine and linear schedulers. Figure F the results. We observe that over 60% of users consider scheduuler-generateed images more realistic and better aligned with the text prompt, while approximately
8

---

Published in Transactions on Machine Learning Research (12/2024)
0.250
0.275
Toalen
Stakler Bonbar
Diveritty
Prefeennc Rate
FID
Divesely
14-
LaoelEne
linest
cosine
1.0
skpumsent
w=7.5 SD1.5
0.255
0.260
0.265
0.270
0.275
0.260
0.20
0.27
0.25
CLIP-Soore
CLIP-Score
linear
casino
(a) SDL5: FID and Div-CS
(b) SD1.5: Study
30
0.26
0.28
SDXL FID vs. CS
SDXL Div vs. CS
CIN 256
25
L.1
linear
cosine
Diversiit
8.0
+8
celie
24
Wed for SDXL
0,2630
0,2675 0,2700 0.2725
0,2500 02775 0.2600 0.2625 0.2650
0.27
0.26
2000
2200
3000
CLIP-Som
CLIP-Scam
Inception Spow
(c) SDXL: FID-CS
(d) CIN-256:IID-IS
Figure Class-condiiioned and text-to-image generation results of monotonicall--iherrssng heuristic schedulers (linear and cosine). (a) FID and Div vs. CS for SD1.5 (Rombach et al.) (022). We highlight the gain of FID and CS compared with the default w=7.5 with black arrows, diversity on the right shows that the heuristic guidance performs better than static bascline guidance; (b) our user study also reveals that images generated with schedulers are consistently preferred than the baseline in realism, diversity and text alignment; (c) results for SDXL (Podell et al. 2023) on FID and Div vs. CS with similar setup to (a); (d) CIN-256 LDM (Dhariwal & Nichol 2021 are assessed with FID vs. IS. Heuristic schedulers outperform the baseline static guidance on fidelity and diversity across multiple models.
Parameter
free: frre:
Linear (cfg=5)
Parameterized; pcs (s=0.1)
Figure 8: Failure cases of parameter-free and parameterized approaches: monotonically increasing guidance may mute the guidance at the beginning (especially when overall guidance is low), causing structural errors; and incorrectly chosen paramcters can lead to fuzzy details and low saturation problems.
80% find guidance schedulers results more diverse. This corroborates our hypothesis that static weighting is perceptually inferior to dynamic weighting. More details in Appx.
Qualitative results. Figure 3 depicts the fidelity of various sets of text-to-image generations from SD and SDXL. Heuristic schedulers (linear and cosine) enhance the image fidelity: better details in petals and leaves of the flower images, as well as the texture of bird features. In the arches example, we observe more natural colour shading as well as more detailed figurines with reflective effects. Figure showcases the diversity of outputs in terms of composition, color palette, art style and image quality by refining shades and enriching textures. Notably, the teddy bear shows various compositions and better-coloured results than the baseline,

---

Published in Transactions on Machine Learning Research (12/2024)
baseline
linzar
damm linear (c-2)
clamp lincar (c-4)
20omed
baseling
linear
clamp linnar (c=2)
clmpp lincar (c-4)
prompt: dog eating ice-cream with sunglass.
IHC
promet:
Big Ben made of French fries.
prompt: a squirrel wearing a crown on stage.
Figure 9: Qualitative comparison of baseline, linear and clamp-linear on SDXL. Both dynamic schedulers are better than the baseline, clamp-linear with c=4 outperforms all with better details and higher fidelity.
which collapsed into similar compositions. Similarly, in the astronaut example, the bascline generates similar images while heuristic schedulers reach more diverse character gestures, lighting and compositions.
5.3 Findings with heuristic schedulers
In summary, we make the following observations: monotonically increasing heuristic schedulers (e.g., linear and cosine) (a) improve generation performances (IS/CS vs. FID) over static baseline on different models; (b) improve image fidelity (texture, details), diversity (composition, style) and quality (lighting, gestures). We note that this gain is achieved without hyperparameter tuning, retraining extra computational cost.
Failure cases. For the failure cases involving monotonically increasing guidance, we observe that under- shooting the guidance scale during the initial stages can compromise the structural integrity of the generated outputs. This often results anatomical and geometric errors, such as the appearance of third leg humans, fifth leg quadruped animals, or incorrect spatial relationships, as illustrated in Figure
6 Dynamic Guidance: Parametrized Schedulers
We investigate two paramctorized schodulers with a tunable parameter maximize performance: a power- cosine curve family (introduced in MDT (Gao et al. 2023 and two clamping families (linear and cosine).
The parameterized family of powered-cosine curves (pcs) and clamping (clamp) is defined by the controllable parameter s and e respectively:
1 (m=t)²
W.
(pes)
(5)
2
W/ max(c, wx)
(clamp)
(6)
In our work, we use clamp-linear but this family can be extended to other schedulers (more in sup. mat.). Our motivation lies in our observation that excessive muting of guidance weights the initial stages can compromise the structural integrity of prominent features. This contributes bad FID at lower values of w in Figure 7a suggesting trade-off wetween model guidance and image quality. However, reducing guidance
10

---

Published in Transactions on Machine Learning Research (12/2024)
FID vs IS
FID VS IS
Statl)
cheppize (em1.))
10 (mmL)
3.2
Binz desplize 12M))
chmpltee (em1.))
po: (112)
cleep (oel
po. (4m44
F10
9.53
9.50
9.655
9.70
9.75
99.01
9.85
9.50
150
175
200
225
250
273
300
325
Inception Seate
Inception Sture
(a)
(b) CIN-256- CLN-256-LDM
Figure 10: Class-conditonned generation results of parameterized clamp-linear and pes on (a) CIFAR- 10-DDPM and (b) CIN-256-LDM. Optimising parameters improves performances but these parameters do not generalize across models and datasets.
intensity early the diffusion process also the origin of enhanced performances, as shown Section This family represents a trade-off between diversity and fidelity while giving users precise control.
ansted Lit) [omostat 7.3)
154
20
clasp- liseaz (c-33
dump-lacur (c-4)
dlmp-litoe (c-1)
linsoz
dhanp-lijeu (om4)
Insar
camp-laaar (c=B)
clan--linae (c=2)
18-
ching-llteer (owl)
60-
clmpp (o=2)
clun--lituor (e-3)
dung-lio (ow2)
class-linae linase (c=4)
16-
14
20
250
500
750
1000
0.2510
0.280
0,271
0.270
0.280
012233
0:270
012275 CLIP-Soom
0.250
112285
Timestep
CLIP-Score
(a) (t): clamp- clamp-linear linear
(b) SD1.5: clamp- linear
(c) SDXL: clamp-linear
ui() (oonstant 2-7.0)
50
cocustant
lanelia:
ps ()m=2)
beseliat
(am2) (s=dl1))
(3-0.33
22.5
fam1)
30-
pox (em1))
pot (e-1)
pas (pm44)
wwd fir SDXL
(xm2)
20-
pox (emd))
17.5
RTD
30
10-
15.0-
750
1000
0.20
0:27
Tamestep
CL.IP-Soxe
CLIP-Seom
(d) w(t): pes
(e) SD1.5: pes
(I) SDXL: pes
Figure 11: Text-to-image performance for two parameterized schedulers: clamp-linear and pcs. For clamp-linear, (a) shows the guidance curves for different parameters and (b,c) displays the FID vs. CS for SD1.5 and SDXL, respectively. For pes, (d) shows the guidance curves and (c,f) depicts the FID vs. CS. Optimal parameters for either clamp or pes outperform the static baseline for both SDL.5 and SDXL.
6.1 caas-condiiional image generation with parametrized schedulers
We experiment with two parameterized schedulers: clamp-linear and pes on CIFARIO-DDPM (Figure 10ab and MmageNet(CINN2555 results over bascline and heuristic schedulers. The optimal parameters are c=1.01 for clamp-lincar and s=4 for pes on CIFARIO-DDPM. c=1.1 for clamp-linear and s=2 for pes on CIN-256. Overall, parameterized schedulers improve performances; however, the optimal parameters do not apply across datasets and models.
(Figure [[b].. We observe that, for both families, tuning parameters improves
11

---

Published in Transactions on Machine Learning Research (12/2024)
6.2 Text-to-iman generation with parametrized schedulers
clamp family
family
baseline
linear
clamp-linear [c=2] clamp-linear (c:4)
(s=44
pcs (s=2)
pcs (s=
pes (s=0.1)
Prompt: mug with mathematical equations put wooden table.
Prompt:
black car running on the road with a lot trees on the side.
Prompt: green alien riding borse.
Drompt:
storm glass water.
Figure 12: Qualitative results for parametrized schedulers clamp-linear and pes on SDXL Podell et al. 2023 Overall, c= for clamp-linear gives the most visually pleasing results.
We experiment with two parameterized schedulers: clamp-linear (clamp-cosine in sup. mat.) and pes, with their guidance curves in Figures lla l1d respectively.
For SD1.5 Rombach et al. 2022), the FID vs. CS results are depicted in Figures 11b and 11c The pes family
struggles achieve low FID, except when s = 1. Conversely, the clamp family exhibits optimal performance around 2=2, achieving the best FID and CI.IP-soore balance while outperforming all pes values.
2023 the FID CS results depicted in Figures 11c and 11f The pes shows For SDXL (Podell et al.
the best performance at 0.1. Clamp- linear achieves optimum at (FID 18.2), largely improving FID across the entire CS range compared to the baseline (FID) 24.9, about 30% gain) and the linear scheduler.
The optimal parameters of clamp-linear (resp. pos) are not the same for both models, i.e. c=2 for SDL.5 and c=4 for SDXL (resp. s=1 and s=0.1 for pcs). This reveals the lack of generalizability of this family.
Qualitative results. The results of Figure g further underscore the significance of choosing the right clamping parameter. This choice markedly enhances generation performance, as evidenced by improved fidelity (dog and squirrel image), textual comprehension (Fries Big Ben), and details (sunglasses).
Figure 12 compares two parameterized familics: (i) clamp and pes (Gao ct al. 2023 where the clamp performs best at e = 4 and the pes at 8 = 1. We observe that the clamp-linear details (e.g., mug, alien), realism (e.g., car, storm), and more textured backgrounds (e.g., mug, car). Although for pes leads the best results for class-conditioned generation, we see that the pes text-to-image task tends over-simplify content, produce fuzzy images (c.g., mug) and deconstructed composition. This highlights our argument that optimal parameters do not generalize across datasets tasks.
demonstrates better
12

---

Published in Transactions on Machine Learning Research (12/2024)
Rectified flow model. Recent advancements in Rectified Flow (RF) (Liu et al. 2022 improve generative models with straighter and shorter generation trajectories. In this section, we show the performance of guid- ance schedulers on RF-based methods e.g. Stable Diffusion 3 (SD3). Our experiments involved calculating the FID and CS on the COCO datasct, similar to all previous experiments. The results, shown in Figure 13a reveal three key findings: (1) dynamically increasing schedulers can further enhance the generation capabil- ities of RF methods, c-g. SD3 (~ FID (gain more than 20%) at the same CS); (2) applying clamping leads additional improvements; and (3) the optimal hyperparametters (c 0.5) does not generalize to other models (SD1.5, SDXI), which is in line with our central hypothesis. The qualitative results presented in Figure 13b and in appendix Figure 24 also demonstrate that the guidance scheduler and clamping methods enhance the details (castle, ship), tone (night and sea), and composition (fishes) of the generation.
Text-to-text Image Translation To show the generalization of schedulers other than text- or label- conditioned tasks, we experiment with image-to-image translation on SD1.5 (Rombach al., 2022 with details in Appendix Results show improved FID-CS balance for linear schedulers, with no clamping being optimal. This differs from the SD1.5 T2T task, showing that the optimal parameter does not generalize.
baseline
linear
lincar (c-0.5)
linear (c-
desp-ltaz
rystical perched cliff illumisated Bull
20
18-
0:320 CMMP-Socce
2eEeE: An underwater world with vibrant cocal cesta, achoola of #iab, and sunken pizate ship.
(a) SD3: clamp-linear
(b) SD3: Qualitative Results
Figure 13: Text-to-image performance and qualitative results on SD3. We show that (a) both lincar and clamp-linear guidance schedulers enhance the balance between FID and CLIP score (CS), and (b) the generated images exhibit improved detail and higher fidelity.
6.3 Findings with parametrized schedulers
Our observations are: (a) tuning the parametrized functions improves the performance for both generation tasks, (b) tuning clamps seems easier than pes family, as its performance shows fewer variations, and (c) the optimal parameters for one method does not generalize across different settings. Thus, specialized tuning is required for cach model and task, lcading to extensive grid searches and increased computational load.
Failure cases. The main risk with parameterized functions arises from ill-chosen parameters. As shown in Figures S 2 20 poorly chosen parameters, particularly for pcs family, often lead to overshooting at the later stages, resulting in fuzzy detail, broken structure and unnatural colour in the generated images.
7 Conclusion
We analyzod dynamic schedulers for the weight paramcter Classificr-Frec Guidance by systematically comparing heuristic and parameterized schedulers. We experiment on two tasks (class-conditioned generation and text-to-image generation), several models (DDPM, SD1.5 and SDXL. and various datasets.
13

---

Published in Transactions on Machine Learning Research (12/2024)
Discussion. Our findings are: (1) simple monotonically increasing scheduler systematically improves the performance compared to constant static guidance, at no extra computational cost and with no hyper- parameter search. (2) parameterized schedulers with tuned parameters per task, model and dataset, improve the results. They, however, do not generalize well to other models and datasets as there is no universal parameter that suils all lasks.
For practitioners who target state-of-the-arr performances, we recommend searching or optimizing for the best clamping parameter. For those not willing to manually tune parameters per case, we suggest using heuristics, specifically linear cosine.
8 Acknowledgment
This work was supported by ANR APATE ANR-22-CE30-0016, ANR TOSAT ANR-200-IADJ-0000 HilParis grant and fellowship, and by the European Union's Horizon 2020 research and innovation programme under the Marie Curie grant agreement No 860768 (CLIPE project). It was granted access to the High- Performrfee Computing (HPC) resources of IDRIS under the allocations 2024-AD011014300RJ made by GENCI.
References
Omri Avrahami, Dani Lischinski, and Ohad Fried. Blended diffusion for text-driven editing of natural images. In IEEE Conf. Comput. Vis. Paltern Recog., 2022.
Yogesh Balaji, Seungjun Nah, Xun Huang, Arash Vahdat, Jiaming Song, Karsten Kreis, Miika Aittala, Timo Aila, Samuli Laine, Bryan Catanzaro, et al. ediffi: Text-lo-image diffusion models with ensemble of expert denoisers. ar preprint arXiv:2211.08 1..1124, 2022.
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint ax11v2331..15127, 2023.
Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale gan training for high fidelity natural image synthesis. In Int. Conf. Learn. Represent., 2018.
Angela Castillo, Jonas Kohler, Juan C Pérez, Juan Pablo Pérez, Albert Pumarola, Bernard Ghanem, Pablo Arbeláez, and Ali Thabet. Adaptive guidance: Training-free acceleration of conditional diffussio models. arXiv preprint Xix::22312 12487, 2023.
Huiwen Chang, Han Zhang, Jarred Barber, AJ Maschinot, José Lezama, Lu Jiang, Ming-Hsuan Hsuan Yang, Kevin Murphy, William T Freeman, Michael Rubinstein, et al. Muse: Text-to-image generation via. masked generative transformers. In Int. Conf. Machine Learning, 2023.
Xin Chen, Biao Jiang, Wen Liu, Zilong Huang, Bin Fu, Tao Chen, and Gang Yu. Executing your commands via motion diffusion latent space. Tn IEEE Conf. Comput. Vis. Pallern Recog., 2023.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Adv. Neural Inform. Process. Syst., 2021.
Anh-Dung Dinh, Daochang Liu, and Chang Xu. Pixelasparam: gradient view on diffusion sampling with guidance. In Int. Conf. Machine Learning. PMLR, 2023a.
Anh-Dung Dinh, Daochang Liu, and Chang Xu. Rethinking conditional diffusion sampling with progressive guidance. In Adv. Neural Inform. Process. Syst., 2023b.
Chris Donahue, Julian McAuley, and Miller Puckette. Adversarial audio synthesis. In Int. Conf. Learn. Represent., 2018.
Nicolas Dufour, David Picard, and Vicky Kalogeiton. Scam! transferring humans between images with semantic cross attention modulation. In Fur. Conf. Comput. Vis. Springer, 2022.
14

---

Published in Transactions on Machine Learning Research (12/2024)
Shanghua Gao, Pan Zhou, Ming-Ming Cheng, and Shuicheng Yan. Masked diffusion transformer is strong image synthesizer. In Tnl. Conf. Comput. Vis., 2023.
Tan Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Ade. Neural Inform. Process. Syst., 2014.
Jonathan Ho and Tim Salimans. Classifie- frese diffusion guidance. In NewrIPS- W on Deep Generative Models and Downstream Applications, 2021.
Jonathan Ho, Ajay Jain, and Picter Abbeel. Denoising diffusion probabilistic models. Adv. Neural Inform. Process. Syst., 2020.
Nandakishore Kambhatla and Todd K Leen. Dimension reduction by local principal component analysis. Neural computation, 1997.
Minguk Kang, Jun Zhu, Richard Zhang, Jaesik Park, Eli Shechtman, Sylvain Paris, and Taesung Park. Scaling up gans for text-to-image synthesis. In IEEE Conf. Comput. Vis. Pattern Recog., 2023a.
Minki Kang, Dongchan Min, and Sung Ju Hwang. Grad-stylespeech: Any-speaker adaptive text-to-speeds synthesis with diffusion models. In IEEE Int. Conf. on Acoustics, Speech and Signal Processing, 2023b.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. stal, 2014.
Tuomas Kynkäänniemi, Miika Aittala, Tero Karras, Samuli Laine, Timo Aila, and Jaakko Lehtinen. Applying guidance in limited interval improves sample and distribution quality in diffusion models. arXiv preprint arXiv:2404.077 07724, 2024.
Alexander C Li, Mihir Prabhudesai, Shivam Duggal, Ellis Brown, and Deepak Pathak. Your diffusion model is secretly a zero-shot classifier. In Int. Conf. Comput. Vis., 2023.
Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Mierosoft coco: Common objects in context. Tn Eur. Conf. on Comput. Vis. Springer, 2014.
Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint orXiv:2209.03000 2022.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Adv. Neural Inform. Process. Syst., 2022a.
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. +pm-solver++: Fast solver for guided sampling of diffusion probabilistic models. arXiv preprint arxix:2211. 1.010.011999 01095. 2022b.
Zhengxiong Luo, Dayou Chen, Yingya Zhang, Yan Huang, Liang Wang, Yujun Shen, Deli Zhao, Jingren Zhou, and Tieniu Tan. Videofusion: Decomposed diffusion models for high-quality video generation. In IEEE Conf. Comput. Vis. Pattern Recog., 2023.
Elman Mansimov, Emilio Parisotto, Jimmy Lei Ba, and Ruslan Salakhutdinov. Generating images from captions with attention. arXiv preprint arXiv:1511.02798, 2015.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In Int. Conf. on Machine Learning, 2021.
Alexander Quinn Nichol, Prafulla Dhariwal, Adilya Ramesh, Pranav Shyam, Pamela Mishkin, Bob Mogrew, 1lya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text- guided diffusion models. In Int. Conf. on Machine Learning. PMLR, 2022.
Maxime Oquab, Timothóe Darcet, Théo Moutakanni, Huy Vo, Mare Szafranice, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaacldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. Xiv preprint ar Xxix22344.07193 2023.
15

---

Published in Transactions on Machine Learning Research (12/2024)
Pablo Pernias, Dominic Rampas, and Mare Aubreville. Wuerstchen: Efficient pretraining of text-to-image models. arxiv preprint arXin:2306.006: 00037, 2023.
Dustin Podell, Zion English, Kyle Lacey, Andreas Blautmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. anXiv preprint ar Xrx:22307700190 2023.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya. Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In Inl. Conf. on Machine Learning. PMLR, 2021.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Pranav Shyam, Pamela Mishkin, Bob McGrew, and Ilya Sutskever. Hierarchical text-conditional image generation with clip latents. arXiv preprint WxXiv2204.06122 2022.
Scott Reed, Zeynep Akata, Xinchen Yan, Lajanugen Logeswaran, Bernt Schiele, and Honglak Lee. Generative adversarial text to image synthesis. In Int. Conf. on Machine Learning. PMLR, 2016.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In IEEE Conf. Comput. Vis. Pattern Recog., 2022.
Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfrr Aberman. Dream- booth: Fine tuning tmxt-to-image diffusion models for subject-driven generation. In JEEE Conf. Comput. Vis. Pallern Recog., 2023.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Bureu Karagol Ayan, Tim Salimans, et al. Photorealistic text- to-image diffusion models with deep language understanding. Adv. Neural Inform. Process. Syst., 2022.
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. Adv. Neural Inform. Process. Syst., 2022.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In Int. Conf. Learn. Represent., 2020.
Wentian Zhang, Haozhe Liu, Jinheng Xie, Francesco Faccio, Mike Zheng Shou, and Jürgen Schmidhu- ber. Cross-attention makes inference cumbersome in text-to-image diffusion models. orXiv preprint urXiv:2404.0274 2024.
Guangeong Zheng, Shengming Li, Hui Wang, Taiping Yao, Yang Chen, Shouhong Ding, and Xi Li. Entropy- driven sampling and training scheme for conditional diffusion generation. In Eur. Conf. on Comput. Vis. Springer, 2022.
Appendix
In this appendix, we provide additional content covering: (A) an extra experiment about the task of image- to-image translation; (B) an ablation study to demonstrate the necessity of CFG at all time intervals; (C) a toy example to cxplain the mcchanism and rationale of the dynamic weighted schoduler; (D) an additional comparison of parameterized function-based dynamic schedulers; (E) more qualitative results; (F) ablation experiments on different aspects of dynamic weighting schedulers; (G) a list of tables of all results demonstrated;; (H) detailed design of user study. Following is the table of contents
(A) Image-to-image Translation Task
(B) The necessity of CFG at all time interval
A toy example of fidelity vs condition adherence
16

---

Published in Transactions on Machine Learning Research (12/2024)
(D)
Comparison of Parameterized Schedulers
(E)
Qualitative Results
(F)
Ablation on Robustness and Generalization
(G)
Detailed Table Experiments
(H)
User Study
A Image-to-image Translation Task
In addition to the image generation task, we also evaluated the image-to-inage translation task to demon- strate the generalization capabilities of the dynamic guidance scheduler across multiple conditioning scenar- ios. The experimental setup closcly follows the one outlined the main manuscript (Section 6 using the SD1.5 backbone model [Rombach et al. 2022 and images and their correspondent prompts from COCO datasct (Lin et al. 2014) achieve image-to-image translation task. As shown in Figure 14a the lincar dynamic guidance scheduler significantly improves the FID vs. CLIP-Score trade-off (~ FID at the same CLIP-Score) in image-to taaage-o--maag translation tasks. However, the optimal parameter for the parameterized scheduler was found to involve clamping at guidance level O, which differs from the optimal parameters identified in the generation task (c = 2). This further supports our primary claim that generalized param- eterized scheduler does not. exist across different lasks. The qualitative results in Figure 14b also showed better structure (car), details (bird background) and prompt understanding (graffiti in the bin).
baseline
linear
clamp-linear (c=2)
18
Ieadias
chapplace (u-0)
17
ckesp-tee (m02)
y-8
15-
144
12-
11
0.208
0.270
0.272
0,27a
0.276
CLIP
(a) Image-to-image: clamp-linear
(b) Image-to-image: Qualitative Results
Figure 14: Imago-to-image performance and qualitative results on SD1.5. We show that (a) both linear and clamp-linear guidance schedulers enhance the balance between FID and CLIP score (CS) of the image-to-ine -maage translation task, and (b) the generated images exhibit improved detail and higher fidelity.
B
The necessity of CFG at all time interval
Recent concurrent work Kynkäänniení et al. 20244: Zhang et al. (2024): Castillo et al. (2023) has suggested that partially removing the classifier-Iree guidance (CFG) (beginning or ending) could enhance generation performance or achieve the acceleration with minimal performance influence. For instance, Kynkaänniemi et al. (2024) proposes removing the initial and final timestops of CFG, retaining only a middle interval for the guidance process. In this section, we conduct two ablation studies on SD1.5 to confirm that the CFG be required across all time intervals.
17

---

Published in Transactions on Machine Learning Research (12/2024)
clapp-lieee wit) (constact um
15.0-
damp- (cm)
Fomore-lie Lave-LLaw wl() (constaal 7
15.0
menoovellierr (x=1)
liusuu
clany-linne (cm3)
damp- (m-3)
amnoellieerr (m1)
bascline
annooe-iiee (x-1)
12.5-
dhmp- (c=3)
12.5
(x=1)
dampp-lmer [c=1]
Treen (x=
clamp--uou (c-1)
semove-liwar (x=1)
clamp lncar (c-2)
10.0-
10.0
16
7.5-
7.5
14-
5.0
5.0
2.5
2.5
12-
0.0
0.0
500
1000
300
1000
0.253
0:260
0.265
0.270
0.275
0.260
Tamnlep
Timnste
CLIP-Seore
Figure 15:
Comparison between guidance removal (labelled as z) and clamping (labelled as c) on SD1.5. We see that CFG removal scheme shows improvement compared to the static guidance baseline (black) but is worse than the linear guidance scheduler (blue solid) and clamping schemes (red dotted).
The necessity of CFG at the beginning stage As demonstrated in Figure Gb we explore the impact of negative perturbation and ablation of all heuristic functions in Figure 12 Generally, employing lower guidance level in the initial stages can enhance performance compared to static guidance. To analyse the effectiveness of guidance removal (setting to zero), static guidance, the linear scheduler, and the clamping scheme, we conducted experiments on SD1.5. Guidance is removed at the same timestep as the clamping transition point: rather than clamping guidance to a hyperparameter constant, we reduce completely to zero (Figure 15 left two panels). The results, depicted in Figure 15 right panel, show that while guidance removal at the beginning stage (red dotted curve) indeed improves performance compared the static baseline (black solid curve), both the linear scheduler (blue solid curve) and clamping schemes (green dotted lines) achieve better balances of FID vs. CLIP-Score (CS).
The necessity of CFG at the ending stage Zhang et al. 2024); Castillo et al. 2023) suggest that removing the final stage of guidance could accelerate generation by directly replacing the CFG with condi- tional or unconditional outputs. However, as shown Figure (b(b), our analysis indicates that removing this stage can reduce performance for specific tasks. Despite this, the possibility of safely removing the ending stage guidance does not contradict argument that enhancing the end could improve per- formance. To further confirm this, conducted an ablation experiment comparing the elfects of removing versus boosting the final guidance intervals. In this experiment, 10%, 20%, and 30% of the ending guidance were either removed or increased by a factor of 1.5. The results, presented in Table 1 reveal: (i) removing or boosting the ending guidance has a marginal impact on the CLIP-Score; (ii) elimination of guidance can lead to a regression in performance; and (iii) boosting the guidance can significantly enhance FID, with gains of 0.54 and 0.8 in FID when boosting the final 30% of guidance by 1.5x.
In conclusion, based the results from two previous ablation studies, we confirm that an adequate level of guidance is necessary at all intervals of the generation process. While removing parts of the guidance can accelerate the process, it results in underperformance when compared to our analyzed hcuristic monotonically incrcasing guidance scheduler, such as lincar, and also when compared to well-tuned parameterized functions, such as the clamping method.
C A toy example of fidelity vs condition adherence
Knowing the equation of CFG can be written as combination between a generation term and a guidance term, with the second term controlled by guidance weight w:
Ca( c) = to (xx, c) + w (eo( (((xr,,))e((x))))
To better understand the problems in diffusion guidance, we present a toy example, where we first train a diffusion model on synthetic dataset of 50, 000 images (32 32) from two distinct Gaussian distributions:
18

---

Published in Transactions on Machine Learning Research (12/2024)
Table 1: Impact of removing/ boosting CFG at the end with SDL.5.
Guidance scale
w-8
w=11
Method
Clip-Score
FID
Clip-Score
FID
static
0.2775
16.78
0.2792
19.03
remove final 10%
0.2777
17.61
0.2792
19.94
remove final 20%
0.2777
18.33
0.2792
20.68
remove final 30%
0.2777
19.18
0.2792
21.65
boost (1.5x) final 10%
0.2773
16.75
0.2789
18.39
boost (1.5x) final 20%
0.2772
16.51
0.2787
18.96
boost (1.5x) final 30%
0.2772
16.24
0.2790
18.23
PCA of lwo Gaussians (GT)
PCA of conditonnl generated results
Repeated Trajectories
Random Noze
Bimestopn0 Derker Sample (Phd Lubel)
Denkor
Bagitar Samples
Bamples
Samples of two Gaussians
guidence scale
guidannee scale
guidance scale 100
Shaky Motions
Figure 16: Two-Gaussians Example. We employ DDPM with CFG to fit two Gaussian distributions, a bright one (red) and darker one (blue). The middle panel showcases samples of generation trajectories at different guidance scales W, using PCA visualization. Increasing guidance scale w raises two issues: repeated trajectory: when w=50 the generation diverges from its expected direction before converging again, and shaky molion: when 1=100 trajectories wander aimlessly.
one sampled with low values intensity (dark noisy images in the bottom-left of Figure 16 and the other with high-intensstics (bright noisy images). The top-left part Figure 16 shows the PCA KKmbhatla
Loen (997)-visualise distribution of the two sets, and the bottom-left part shows some moound-rruth, images. To fit. these two labelled distributions, we employ DDPM (Ho et al. 2020 with CFG (Ho & Salimans 2021 conditioned on intensity labels.
Upon completion of the training, we can adjust the guidance scale w to balance between the sample fidelity and condition adherence, illustrated in the right part of Figure [6 The first. row depicts the variations generated distributions on different w (from 0 to 100), visualized by the same PCA parameters. The second row shows the entire diffusion trajectory for sampled data points (same seeds across different w): progressing from a random sample (i.e., standard Gaussian) when T to the generated data (blue or red in Figure 16) when i O.
Emerging issues and explainable factors. As w increases, the two generated distributions diverge due to guidance term in Eq. shifting the generation towards different labels at a fidelity cost. (see Figure 16 first row).
As shown in Figure 16 (socond row), two issues arise: (i) repeated trajectories that diverge from the expected convergence path before redirecting to it; and (ii) shaky motions that wander along the trajectory.
19

---

Published in Transactions on Machine Learning Research (12/2024)
FID va CIFAR
lockee (datl)
ooep (oal) Camplamu
Auqumeee Jraf 31
321
4.
disgeaatte 1.30
3.1
3.1
5.0
5.0
2.-
2.3
28-
28-
2.7-
9.5
9.7
9.8
99
9.6
9.7
0.5
99
9.6
9.7
9.8
9.0
Inropiion
losplion Score
FID va IS CIN-256
3-
Scadan (4Mt))
9-
lundan (cida] 3-103) giu Qeiil)
Cood)
6-
5-
4-
4-
3-
3-
2.0
260
300
200
250
310
150
210
250
300
Inception
lnception
Inception
Figure 17: Classicon image generation results of two parameterized families (clamp- lincar, clamp-cosinc and pcs) on CIFAR-10 and CIN-256. Optimising parameters of guidance results in performance gains, however, these parameters do not generalize across models and datasets.
These two issues can be attributed to two factors: (1) incorrect classification prediction, and (2) the conflicts between guidance and generation terms in Eq. 7 For the former, optimal guidance requires a flawless classifier, whether explicit for CG or implicit for CFG. In reality, discerning between two noisy data is challenging and incorrect classification may steer the generation in the wrong direction, generating shaky trajectories. A similar observation is reported in Zheng al. (2022); Dinh et al. (2023b) for CG and in Li et al. (2023) for CFG. For the latter, due to the strong incentive of the classifier to increase the distance with respect to the other classes, trajectories often show a U-turn before gravitating to convergence (repeated trajectory in Figure 16). We argue that this anomaly is due to the conflict between guidance and generation
terms in Eq.
In conclusion, along the generation, the guidance can steer suboptimally (especially when - T),, and even impede generation. We argue that these erratic behaviours contribute to the performance dichotomy between fidelity and condition adherence (Ho & Salimans, 2021 Dhariwal & Nichol 2021
Comparison of Parameterized Schedulers
D.1 Parameterized Comparison on Clas-Conditioned Generation
For CIFAR-10-DDPM, we show in Figure 17 upper panels (see all data in Table 5
the comparison of two parameterized functions families: (i) clamp family on linear and cosine and ii pcs family mentioned in
6
7
Gao et al. (202)).
The ImageNet-222 256 and Latent Diffusion Model (LDM) results are presented in Figure 17 lower pancls and
(data in Table 0 10 11
20

---

Published in Transactions on Machine Learning Research (12/2024)
20
laselline
clacap-liusor (c=8)
1.25
sodtdd wef (constian w
linexr
clazup- liuesr (c=1)
18
clump-linear (e=1)
w-7.5 801.5
1.20
10
16
clamp-lizou (pe2)
Divernity
1.15
1.10
5-
1.05
12-
1.00
250
500
750
1000
0.250
0.255
0:270
0.275
0.250
0.26
0.27
0.28
Timeslep
CLIP-Soure
CLIP-Scure
(a) clamp-linear
15-
staled wit (oooslam w.7..5
20-
baseline [38]
clamp-cosize cosine (c=3)
1.25
clamp-cosiee (o-i4)
15
clamp-oosine (c=1)
w-7.5 7.5 SD1.5
1:20
10-
damp-rosine (c=22
Diverslo
7.15
5-
11-
1.10
1.55
12-
1.00
750
1000
0.255
0.265
0.275
0.26
Timestop
CLIP-Soore
CLIP--coore
(b) glamp-cosing
1.3
40
baseliin
pes (x=2)
woued (i (cone wm7.
22.5
(jm1)
pes (s=0.1)
30
20.0
(s=4)
7.5.5 SD1.5
Diversioy
1.2
20
17.5
15.0
1.1%
10
12.5
1.0
750
0.245
0:250
0.255
0.260
0.265
0.270
0.275
0.290
0.26
Timestep
CLIP Scoze
CLIP-Score
(c) pes Family
Figure 18: Text-to-image generation FID and diversity of all two parameterized families (clamp with clamp-linear, clamp-cosine and pcs) on SD1.5 (left to right): (a) parameterized scheduler curves; (b) FID vs. CS of SD1.5 and (c) FID vs. Div. of SD1.5. We show that in terms of diversity, the clamp lamily still achieves more diverse results than the baseline, though reduces along the clamping parameter, the beginning stage of the diffusion is muted.
The conclusion of these parts is follows: (i) optimising both groups of parameterized function helps improve the performance of FID-CS; (ii) the optimal parameters for different models are very different and fail to generalize across models and datasets.
D.2 Parameterized Comparison on Text-to-image Generation
We then show the FID CS and Diversity vs. CS performance of the parameterized method in Figure 18 The conclusion is coherent with the main paper: all paramcterized functions can enhance performance on both FID and diversity, provided that the parameters are well-selected. Moreover, for the clamp family, it appears that the clamp parameter also adjusts the degree of diversity of the generated images; lowering the clamp parameter increases the diversity. We recommend that users tune this parameter according the specific model and task. For SDXL, the clamp-cosine is shown in Figure 19 and also reaches similar conclusion.
21

---

Published in Transactions on Machine Learning Research (12/2024)
FID vs CS clamp linsar
FID vs. CS clamp
FID CS pos lamily
por 6-1))
65
dlep laaer (ow4)
45-
Shusp on (c=d)
65
pos (1--4)
dlemp Iaser
strmp (e-4)
clorpp (mw0)
(lmnp (r-d)
por (pmL)))
9--8 SIXI
S0N3.
2- for 5000
30-
30-
30-
25
20
20-
20
01265
0,270
0,277
0,2511
01265
11.355
0.270
0.275
0.2601
11285
0:24
0.25
0.25
0.27
0.28
CT Cl.IP-Soom
CLIP-Soom
Figure 19: Text-to-image generation results of two parameterized families (clamp-linear, clamp- cosine and pcs) on SDXL. Both clamps reach their best FID-CS c vs s 0.1 for pes, which differ from the optimal parameters for SD1.5.
E Qualitative Results
More Results of Parameterized Functions on SDXL In Figure 20 we show more examples of different parameterized functions. It appears that carefully selecting the parameter = 4), especially for the clamp- linear method, achieves improvement in image quality in terms of composition (e.g., template), detail (e.g.. cat), and realism (e.g., dog statue). However, for SDXL, this method shows only marginal improvements with the pes family, which tends to produce images with incorrect structures and compositions, lcading to fuzzy images.
clamp family
pcs family
beseline
linear
clamp-linear (c-2) clamp-linear (c-4) pcs (s-4)
pcs (s-2)
pcs (s-1)
pcs (s-0.1)
Prompt: A cat eating street food in front of the school.
Pxompt: A buddhist temple on the noon.
Prompt: & dog statue inside en art gallery.
Figure 20: Qualitative comparison clamp vs. pes family, we see clearly that clamping at = 4 gives the best visual qualitative results.
Stable Diffusion v1.5. Figure 21 shows qualitative results using increasing shaped methods: linear, cosine compared against the bascline. It shows clcarly that the increasingly shaped hcuristic guidance generates more diversity and the baseline suffers from a collapsing problem, i.c., different sampling of the same prompt seems only to generate similar results. In some figures, e.g., Figure 21 with example of
22

---

Published in Transactions on Machine Learning Research (12/2024)
Table 2: Ablation on sampling steps DDIM. Experiment on CIN-256 and Latent Diffusion Model
baseline
(static)
linear
cosine
steps
FID4
1St
FID4
1St
FID4
ISt
50
3.393
220.6
3.090
225.0
2.985
252.4
100
3.216
229.8
2.817
225.2
2.818
255.3
200
3.222
229.5
2.791
223.2
2.801
254.3
the mailbox, we can see that the baseline ignores graffiti and increasing heuristic guidance methods can correctly retrieve this information and illustrate it in the generated images. We also see in M&M's that heuristic guidance methods show more diversity in terms of colour and materials. with much richer variance and image composition. However some negative examples can also be found in Figure 2T in particular, the foot of horses in the prompt: person riding a horse while the sun sets. We posit the reason for these artefacts is due to the overmuting of the initial stage and overshooting the final stage during the generation, which can be rectified by the clamping method.
SDXL. The SDXL (Podell et al. 2023) shows better diversity and image quality comparing to its precedent. Whereas some repetitive concepts are still present in the generated results: see Figure 22 that first row 'A single horse leaning against a wooden fence' the baseline method generate only brown horses whereas all heuristic methods give a variety of horse colours. A similar repetitive concept can also be found in the *A person stands on water skies in the water' with the color of the character. For the spatial combination diversity, please refer to the example in Figure [3] 'A cobble stone courtyard surrounded by buildings and clock tower." where see that heuristic methods yield more view angle and spatial composition. Similar bchaviour can be found in the example of 'bowl shelf" in Figure 22 and "teddy bear" in Figure 22
SD3. In SD3 (see Figure 24 we demonstrate that the schedulers enhance the details (e.g., books, street, lake surface), improve tone and chromatic performance (e.g., street and sunsct), and lead to a better under- standing of the prompt (e.g,, giant clock).
F Ablation on Robustness and Generalization
Different DDIM steps. DDIM sampler allows for accelerated sampling (e.g., 50 steps as opposed to 1000) with only a marginal compromise in generation performance. In this ablation study, we evaluate the effectiveness of our dynamic weighting schedulers across different sampling steps. We use the CIN256-LDM codebase, with the same configuration as our prior experiments of class-conditioned generation. We conduct tests with 50, 100, and 200 steps, for baseline and two heuristics (lincar and cosine), all operating at their optimal guidance scale in Tab [ The results, FID vs. IS for each sampling step, are presented in Tab. 2 We observe that the performance of dynamic weighting schedulers remains stable across different timesteps.
Different Solvers. To validate the generalizability of our proposed method beyond the DDIM (Song et al. 2020) sampler used in the experiment Section, we further evaluated its performance using the more advanced M-M-Solver (Lu et al. 2022a) sampler (3rd order). This sampler is capable of facilitating diffusson generation with fewer steps and enhanced efficiency compared to DDIM. The experiment setup similar the text-to-image generation approach using Stable Diffusion (Rombach et al. 2022) v1.5. The results of this experiment are reported in Table and visually illustrated in Figure 25
As depicted in Figure 25 our proposed methods continue to outperform the baseline (static guidance) approach. Substantial improvements are seen in both FID and CLIP-Score metrics, compared to baseline (w=7.5) for example. Notably, these gains become more pronounced as the guidance weight increases, a trend that remains consistent with all other experiments observed across the paper.
Diversity Diversity plays a pivotal role textual-based generation tasks. Given similar text-image match- ing levels (usually indicated by CLIP-Score). higher diversity gives users more choices of generated content. Most, applications require higher diversity to prevent the undesirable phenomenon of content collapsing,
23

---

Published in Transactions on Machine Learning Research (12/2024)
baseline
linear
cosine
prompt: à box to drop mail into covered with graffiti.
prompt: Different colors KAN's in bowl on the table
Yellow, blue, and green stacked each other.
prompt: Person riding horge while the sun sets.
prompt: apple, orange, yogurt, peanuts are sitting in plastic bowl.
Figure 21: Qualitative SD1.5
24

---

Published in Transactions on Machine Learning Research (12/2024)
baseline
linear
cosine
prompt: A single horse leaning against
wooden fence.
prompt: Yellow, blue, green suitcases stacked on each other.
prompt: person stands in the
prompt:
There is stuffed bear sitting on book shelf
prompt: & dog in tuxedo is sitting at a table on which piece cake sitting.
Figure 22: Qualitative SDXL (1)
25

---

Published in Transactions on Machine Learning Research (12/2024)
baseline
linear
cosine
prompt: & cobble stone courtyard surrounded by buildings and
prompt: & man is skateboarding across city street.
prompt: À shelf with bowls lined up on it.
prompt: double decker bus moving past tall building.
prompt:
with tiles
Figure 23: Qualitative SDXL (2)
26

---

Published in Transactions on Machine Learning Research (12/2024)
baseline
linear
linear (c=0.5)
linear (c-1)
prompt: cozy library filled with ancient books, warm lighting, crackling fireplace.
prompt: A ayberpunk street scene at night, filled with glowing signs, street vendors, and bustling crowds.
prompt: surreal dreamscape with floating islands, giant clocks, river of stars,
prompt: serene lake sunset with mountains in the distance and small wooden boat floating the water.
Figure 24: Qualitative SD3
27

---

Published in Transactions on Machine Learning Research (12/2024)
FID vs CS
26
baseline
24
cusine
22
linear
w.7.5 for SDL.5, Equal ClipSeure Line
20-
W=7.5 for SD1.5, Equal FID line
18-
0.255
0.200
0.265
0.270
0.275
0,250
0,285
CLIP-Score
Figure 25: FID vs. CLIP-Score generated by SD1.5 Rombach et al. 2022
with DPM-Solver [Lu et al.
2022a
Table 3: Experiment of FID and CLIP-Score generated by Stable Diffusion v1.5 with DPM-Solver Lu ct al. 2022a)
w
3
13
15
20
clip-score
0.2287 0.2692
0..7746
0.2767 0.2782 0.2791
0.2797 0.2802 0.2805
baseline( staatic)
FID
28.188 10.843 13.696 16.232 17.933 19.136 19.930 20.538 21.709
clip-score
0.2287 0.2646 0.2713 0.2743 0.2762 0.2774 0.2785 0.2792 0.2813
linear
FID
28.188 13.032 11.826 12.181 12.830 13.461 13.984 14.541 15.943
clip-score
0.2287 0.2643 0.2712 0.2741 0.2762 0.2778 0.2789 0.2797 0.2812
cosine
FID
28.188 12.587 11.810
12.400
13.197 13.968
14.717 15.366 16.901
28

---

Published in Transactions on Machine Learning Research (12/2024)
where multiple samplings of the same prompt yield nearly identical very similar results. We utilize the standard deviation within the image embedding space as measure of diversity. This metric can be derived using models such as Dino-v2 Oquab et al. (2023) or CLIP Radford et al. (2021). Figure 26 provides a side- by-side comparison of diversitics computed using both Dino-v2 and CLIP, numerical results are also reported in Table. 16 is evident. that Dino-v2 yields more discriminative results compared the CLIP embedding. While both embeddings exhibit similar trends, we notice that CLIP occasionally produces narrower gap between long captions (-L) and short captions (-S). In some instances, as depicted in Figure 26 CLIP even reverses the order, observation not apparent with the Dino-v2 model. In both cases, our methods are consistently out purprrormingg the baseline on both metrics.
Dino-v2(Qquab
CLIP score vs CLIP-based Diversity for SD
CLIP score vs Dinov2-buaed Diversity for SD
0.32
1.2
0.30
0.28
0.26
Leadian bealkc)L
nae8
10-
Ireeds
0.255
0.200
02000
0.270
0200
CLIP so0ne
CLIP scoec
Figure 26: Experiment on Stable Diffusion on two types of diversity. Zero-shot COCO 10k CLIP- Score Diversity computed by CLIP and Dino-v2 respectively.
G Detailed Table of Experiments
In this section, we show detailed tables of all experiments relevant the paper:
CIFAR-10-DDPM. results of different shapes of heuristics (Table 4 results parameterized methods (Table Table Table
CIN (ImageNet) 256-LDM: results of different shapes of heuristics (Table and results of paramcterizod methods (Table Table Table 11
Stable Diffusion 1.5: results of different shapes of heuristics in Table 12 and results of parame- terized methods in Table 13
Stable Diffusion XL: results of different shapes of heuristics in Table and results of parame-
terized methods in Table 1
Table 4: Experiment of different Heuristics CIFAR-10 DDPM. We evaluate the FID and IS results for the baseline, all heuristic methods (green for increasing, red for decreasing and purple for non- linear) of 50K images. Best FID and IS are highlighted. We see clearly that the increasing shapes outperform all the othess.
Guudance Scale
haseline (staticc
FIB
18
F11
Tnear
13
F1L
13
F10
InVilnss
F10
kIn
13
FID
A-chapa
15
FTD
Viahape
1.10
2.508
5.3.5
2.533
5.595
2.875
5.608
3.0153
5.351
3.058
9.350
3.0117
9.615
3.005
9.350
1:15
2.5-67
9.854
2.853
5.8688
2.8224
9.670
35.050
5.6:28
3.086
9.612
3.040
9.635
2.084
9.336
1000
2.071
9. 0.000
2:854
0.7.9
2.611
9.726
3.106
0.643
3.149
0.645
3.119
9.735
2.828
0.644
235
3.025
9. 9.733
2.587
9.788
2.550
9.793
3.182
9.675
3.261
9.600
3.251
9.746
2.800
9.677
30
9.
2.968
9.833
2.933
9.838
3.311
9.659
3.389
9.654
3-407
9.774
2,851
3.480
3.543
2.955
H User Study
In this section, we elaborate on the specifics of our user study setup corresponding to Figure 3. (b) in our main manuscript.
29

---

Published in Transactions on Machine Learning Research (12/2024)
Table 5: Experiment of clamp-linear on CIFAR-10 DDPM. We evaluate the FID and IS results for the baseline, parameterized method as clamp-linear of 50K images FID. Best FID and IS are highlighted, the optimal parameter seems at c = 1.1.
Guidance Scale
bascline (static)
FID
15
FID
linear
13
linear (c=1
FID
(c=1.05)
15
linear
FID
I.I)
15
linear
FID
1.15)
IS
1.10
2.966
2.893
9.595
2.862
9.622
2.856
9.635
2.870
1.15
2.917
9.645
2.853
9.606
2.816
9.693
2.793
9.696
2.832
9.693
1.20
2.971
9.690
2.834
9.729
2.822
9.737
2.820
9.755
2.834
9.750
1.25
3.025
9.733
2.897
9.799
2.863
9.809
2.863
9.809
2.863
9.809
1.30
3.111
9.764
2.968
9.833
2.920
9.870
2.922
9.863
2.929
9.867
3.025
9.910
Table 6: Experiment of clamp-cosine on CIFAR-10 DDPM. We evaluate the FID and IS results for the baseline, parameterized method as clamping on the cosine increasing heuristic plammp-cosine) of 50K images. Best FID and IS are highlighted. It sees the optimising clamping parameter helps to improve the FID-IS performance, the optimal parameter seems at c = 1.05.
Guidance Scale
baseline (static)
[c= 1.05)
(c=
1.1)
1.15)
FID
18
FID
18
F1D
18
F(D
18
F1D
1$
2.839
1.15
2.947
9.645
2.821
9.670
2.781
9.712
2.794
9.710
2.917
9.689
1.20
2.971
9.690
2.813
9.726
2.771
9.781
2.786
9.774
2.901
9.753
1.25
3.025
9.733
2,850
9.794
2.810
9.828
2.819
9.823
2.913
9.821
1.30
3.111
9.764
2.933
9.838
2,880
9.884
2.888
9.885
2.976
9.865
3.233
9.787
3.026
9.882
3.052
Table 7: Experiment of pes family on CIFAR-10 DDPM. We evaluate the FID and IS results for the baseline, parameterized pcs method of 50K image FID. Best FID and IS are highlighted. It sees the optimising clamping parameter helps to improve the FID-IS performance, the optimal parameter seems at
Guidance Seale
baseline (static)
(s=4) 4)
pcs
(s=2)
pes (s=])
(s=0.1)
FID
IS
FID
18
FID
1S
F(D)
18
F1D
IS
2.966
9.564
1.15
2.947
9.645
2.818
9.663
2.886
9.670
2.824
9.670
2.983
9.657
1.20
2.971
9.690
2.748
9.726
2.844
9.729
2.813
9.726
3.010
9.706
1.25
3.025
9.733
2.714
9.782
2.839
9.782
2.850
9.791
3.065
9.733
1.30
3.111
9.764
2.700
9.834
858
9.847
2.933
9.838
3.157
9.770
3.233
9.787
2.711
9.882
3.276
9.786
Table 8: Experiment of different Heuristics on CIN-256-DDM, We evaluate the FID and IS results for the baseline, all heuristic methods (green for increasing, red for decreasing and purple for non-linear) of 50K images FID. Best FID and IS are highlighted. We see clearly that the increasing shapes outperform all the others.
guidance
FID
baselin
15
FID
Liueue
15
FID
EG2
15
FJD
In-lineer
15
FID
15
FID
A-shape
13
FID
V-alape
1.41
8:117
183.2
8.1346
178.3
1.331.
175-4
3.3123
180.7
1.305
180-22
3.6444
207
6:118
140.2
1.6
3.500
225.0 0
3.000
220.6
3.053
216.2
3.074
222.7
4.170
221.7
3. 3.6994
250.0
4.450
176.
1.8
8.800
260 &
257.5
2.985
252.4 4
7.707
257.3
0.887
254.6
1.822
284.0
3.763
200.
5.072
201
288.9
3.459
8.085
284.2
281-2
3.906
310.8
315.1
4.258
9.164
340.2
4.208
830.6
831.2
Table 9: Experiment of clamp-linear family on CIN-256-LDM, We evaluate the FID and IS results for the baseline, parameterized clamp-lincar on 50K images FID. Best FID and IS are highlighted. It sees the optimising parameter helps to improve the FID-IS performance, the optimal parameter seems at 1.005.
base line
linear
linear (c= (055)
linear (c= L.)
linear (c= [.3)
guidance
FID
FID
FID
FID
FID
IS
4.12
178.3
184.6
1.6
3.39
225.0
3.09
220.6
3.08
219.6
3.13
219.2
3.14
222.7
3.94
257.5
257.2
259.0
2.0
5.07
291.4
3.86
288.9
3.88
287.0
3.86
288.7
3.92
289.6
6.40
315.8
4.89
315.1
4.91
312.4
4.87
313.8
4.92
314.9
2.4
8.95
335.9
6.03
336.5
6.00
334.3
5.97
336.8
6.01
337.2
30

---

Published in Transactions on Machine Learning Research (12/2024)
Table 10: Experiment of clamp-cosine family on CIN-256-LDM. We evaluate the FID and IS results for the baseline, parameterized method of clamp-cosine method on 50K images. Best FID and IS are high- lighted. It sees the optimising parameter helps to improve the FID-IS performance, the optimal parameter scems at c = 1.005.
guidance
baseline
cosine
cosine
1.055)
cosine
[c= 1.1)
cosine
1.3)
FID
15
F1D
18
FID
15
FID
18
FID
IS
176.0
177.1
1.6
3.39
224.96
3.08
216.2
3.06
217.0
3.08
217.1
3.09
224.6
1.8
3.94
260.85
2.98
252.4
2.91
251.8
3.01
233.2
3.13
258.4
2.0
5.07
291.37
3.46
283.3
3.47
282.5
3.48
284.1
3.67
288.2
2.2
6.40
315.84
4.26
310.1
4.27
307.9
4.28
310.5
4.49
313.1
334.1
Table 11: Experiment of pcs family CIN-256-LDM, We evaluate the FID and IS results for the baseline, parameterized method of the pes family of 50K images. Best FID and IS are highlighted. It sees the optimising parameter helps to improve the FID-IS performance, the optimal parameter seems at s = 2 for FID. Interestingly, the pcs family presents a worse IS metric, than baseline and clamp-linear/osse /cosine methods.
guidance
baseline
pcs (==1)
pcs (s=2)
pes
pcs (s=0.1)
FID
18
FID
18
FID
18
FID
18
FID
1S
1.4
4.12
181.21
6.94
111.98
6.10
150.19
1.31
175.10
1.09
181. $51.00
1.6
3.39
224.96
3.69
162.99
4.27
180.52
3.08
216.21
3.43
225.31
1.8
3.94
260.85
4.80
179.71
3.29
208.86
2.98
252.37
3.96
264.03
2.0
3.07
291.37
4.18
195.75
2.88
234.09
3.46
283.32
5.08
294.77
2.2
6.40
315.84
3.73
210.60
2.81
257.22
4.26
310.14
6.44
319.97
335.86
Table 12: Different Heuristic Modes of SD1.5, we show FID vs. CLIP-score of 10K images. we highlight different range of clip-score by low (~ 0.272), mid (~ 0.277) and high (~ 0.280) by pink, orange and blue colors. We see that increasing modes demonstrate the best performance at high w, whereas decreasing modes regress on the performance. non-linear modes, especially A-shape also demonstrate improved performance to bascline but worse than incrcasing shapes.
8
10
12
14
bascline
clip-score
0.2593
0.2719
0.2757
0.2775
0.2790
0.2796
0.2803
FID
16.777
18.419
19.528
20.462
clip-score
0.2565
0.2697
0.2741
0.2763
0.2780
0.2788
0.2799
linear
FID
14.649
11,260
12.056
13.147
14.179
15.032
15.663
elip-score
0.2553
0.2686
0.2728
0.2751
0.2770
0.2782
0.2793
cos
FID
15.725
11.846
12,009
12.796
13.629
14.282
15, 15.008 008
clip-score
0.261
0.272
0.2754
0.2773
0.2780
0.2787
0.2793
sin
FID
10.619
14.618
18.323
20.829
22.380
23.534
24.661
invlinear
clip-score
0.2608
0.2723
0.2757
0.2773
0.2781
0.2789
0.2793
FID
10.649
14.192
17.810
20.206
21.877
22.962
24.128
A-shape
clp--score
0.2003
0.2719
0.2766
0.2774
0.2785
0.2794
2802
FID
11.940
12.106
14.183
16. 16,100
17.530
18. 663
19. 19.723
V-shap
cip-seore
0.2500
0.2700
0.2747
0.2704
0.2773
0.2783
0.2780
FID
11.790
12.107
15.912
18,220
19.799
20.992
21.905 905
31

---

Published in Transactions on Machine Learning Research (12/2024)
Table 13: Different parameterized functions of SD1.5, we show FID vs. CLIP-score of 10K images. highlight different range of clip-score by low (~ 0.272), mid (~ 0.277) and high (~ 0.280) by pink, orange and blue colors. We see that for the pcs family the optimal parameter is at 1, whereas for clamp-linear and clamp-cosine methods, they are at = 2.
clip-score
0.2593
0.27
0.2757
0.2775
0.2790
0.2796
0.2803
baseline
FID
11.745
11.887
14.639
16.777
18.419
19.528
20.462
pcs (s=4)
clip-score
0:2453
0,2582
0.2706
FID
23.875
19.734
19.167
19.627
20.513
22.022
23.585
pcs (s=2)
clip-score
0.2591
0.2641
0.2691
0.2720
0.2740
0.2754
0.2766
FID
18.026
14.414
13.503
13.652
14.175
14,806
15.480
prs (s=1)
clip-score
0.2553
0.2686
0.2728
0,2751
0.2770
0,2782
0,2793
15.725
12.009
12.796
13.629
14.282
pes (s=0.1)
clip-scure
0,2507
0.2642
0.2755
0.2772
0.2785
0.2796
0. 0.2800
FID 19.532
14.414
14.770
16.001
18.312
19.349
20.271
linear (c=1)
clip-scure
0.2613
FID
11.4448
11.011
12.130
13.211
14.219
12.
15.888
linear (c=2)
clip-score
0.2679
0.2717
0.2751
0.2769
0.2783
0.2793
0.2800
FID 10.7382
14.166
11.916
16.041
linear (c=3)
clip-score
0.2719
0.2732
0.2756
0.2771
0.2783
0.2798
0.2800
12.1284
12.328
13.019
13.916
14.701
16.109
16.420
clip-score
0.2742
0.2746
0.2761
0.2775
0.2786
0.2794
0.2802
linear (cm4)
FID
13.768
13.813
14.213
14.765
15.311
15.834
16.422
cos (c=1)
clip-score
0,2618
0,2703
0.2744
0.2762
0.2777
0.2787
0.2795
FID
11,386
10.986
11.732
12.608
13.460
14.288
14,978
008 (c=2)
clip-score
0,2682
0.2722
0.2751
0,2769
0,2780
0.2789
O 0,2800
10.816
11.309
clipsscure
0,2719
0.2736
0.2757
0.2772
0.2792
0.2792
0,2800
cos (c=3)
FID
12.121
12.363
12.956
13.631
14.263
14.869
15.385
cos (c=4)
clip-scure
0.2742
FID
13.734
13.827
14.222
14.690
15.090
15.560
15.916
Table 14: Different Heuristic Modes of SDXL, we show FID vs. CLIP-score of 10K images. we highlight different range of clip-score by low (~ 0.2770), mid (~ 0.280) and high (~ 0.2830) by pink, orange and blue colors. We see that increasing modes demonstrate the best performance at high w, whereas decreasing modes regress on the performance. non-linear modes, especially A-shape demonstrate improved performance against bascline but regress fast when the w is high.
5
7
11
13
15
20
baseline
clip-score FID
0.2248
0.2712
0.2767
0.2791
0.2806
U.2817
0.2826
0.2832
0.2836
25.7080
27.4628
28.0538
Inear
elip-score
0.2248
0.2653
0.2732
0.2773
0.2798
0.28
0.2821
0.2828
0.2840
FID
59.2480
29.0917
25 25.0276
24.4500
24.6705
25.1286
25.5488
25.8457
26.5993
clip-scnre
0.2248
0.2621
0.2708
0.2751
0.2776
0.2794
0.2803
0.2817
0,2830
FID 59.2480
$2.8264
27.0004
25.5468
25.4331
25.5244
25.7375
25.8758
26. 26.8427
invlinear
elip-score
0.22188
0.2739
0.2783
0.2800
0.2814
0.2826
0.2823
0.2807
0.2730
FID
59.2480
23.8196
25. 13.4335
26. 1458
27.8969
29.6194
31.8970
35.2600
47. 84.64
dip-scooee
0.2248
0.2741
0.2756
0.2803
0.2816
0.2823
0.2816
0.2794
0.2713
sin
FID
59.2480
23.9147
.137
28.1756
30.5314
36.3029
6676
A-shape
clipscooee
0,2248
0,2721
0.2782
0.2809
0,2826
U. 01.8311
0.2837
0,2846
0,2849
FID 59.2480
22.3927
24.0785
25. 25.6845
26.7019
27.5095
28.8
3870
34.9896
Vishape
clip-soore
0,2248
0.2747
0.2770
0.2785
0.2795
0.2786
0.2736
FID
59.2480
21.6560
22.7042
23.6659
24.0550
25.4073
26. 26.2993
27.6580
35.2935
32

---

Published in Transactions on Machine Learning Research (12/2024)
Table 15: Different parameterized results in SDXL, we show FID vs. CLIP-Score of pes family and clamp family of 10K images: pcs family records best performance at s = 0.1, clamp-linear and clamp-cosine strategies all record best performance at c = 4.
1
3
7
9
11
13
15
20
haseline
clip-score
0.2248
0.2712
0.2767
0.2791
0.2806
0.2817
0.2826
0.2832
0.2836
FID
69.2480
24.3634
24.9296
25.7080
26. 1654
27.2308
27.4628
28. 05.0038
29.6868
pes (s
clip-score
0.2248
0.2336
0.2396
0.2440
0.2470
0.2494
0.2513
0.2527
0.2549
FID
59.2480
55.2402
52.0731
50,3335
48,9980
48.4516
48.0146
47.70 47.7025
48.9481
clipsscore
0.2248
0,2480
0.2681
0.2638
0.2673
0,2704
0.2722
0.2738
0.2765
pes (* = 2)
FID
59 69.0
35.2002
28.7500
24.8120
22.8518
21.7098
22. 1061
23 23.0833
23.5282
pes (+ = 1)
clip-score
0.2248
0.2621
0.2708
0.2761
0.2776
0.2794
0.2803
0.2817
0,2830
FID
59 2480
32, 82.4
27 0000
25,54688
25.4331
25.5214
25. 35.7375
25 8758
26. 86187
pes (x 0.1)
clip-score
0.2218
0.2710
0.2769
0.2798
0.2812
U.2823
0.2830
0.2836
0.2814
FID
59.2480
18.5 18.5894
8975
19.8658
20.5433
21.1257
21. 1.6248
21.9118
23.7671
linear = 2)
clip-score
0.2218
0.2717
0.2752
0.2781
0.2798
0,2810
0.2822
0,2830
0,2810
FID
59.2480
24.3084
23.8361
24.0241
24.4806
24.6759
24.9336
25.6498
26, 26.6398
linear (c = 4)
clip-score
0.2218
0,2773
0,2778
0,2792
0,2805
0,2818
0.2827
0.2831
0,2815
FID
59 2480
18.2321
18.2517
18.2678
18.3675
18,5902
18. 1336
19.1395
19.9400
lineer (c = 6)
clip-score
0.2248
0.2798
0.2799
0.2803
0.2811
0.2819
0.2825
0.2832
0.2846
FID
59. 2480
19.3309
19.3295
19.2716
19.280
19.2955
19.4298
19.5635
1577
clip-scue clip-score
0.2248
0.2720
0.2748
0.2775
0.2799
0.2806
0.2816
0,2822
0,2836
cosine 2)
59.2480
24.2768
23.9367
23.8442
24.1493
24.3516
24.6917
25.0779
25.8126
clipescore
0,2248
0.2773
0,2780
0,2793
0,2806
0,2816
0.2825
0,2832
0,2843
cosine (e = 4]
FID
59 24.0
18 23.2
18 2336
18. 18.2761
18.2361
18.3372
18. 18.5678
18 8925
19. 19.6065
clip-score FID
0.2248
0.2798
0,2799
0.2803
0.2813
0.2821
0.2826
0.2830
0,2843
cosine (e 6)
59.2480
19.29-13
19.2701
19.2261
19.2656
19.2711
19.27413
19.2670
19. 19.7335
Experiment on SD1.5 with Diversity measures of 10K images, comparison between the baseline and two increasing heuristic shapes, linear and cosine.
Table 16:
w
2
4
6
8
10
12
14
20
25
clip-score
0.2593
0.2719
0.2737
0.2776
0.2790
0.2796
0.2803
0.2813
0.2817
FID
11.745
11.887
14.639
16.777
18.419
19.528
20.462
22.463
23.810
baseline
Div-CLIP-L
0.315
0.289
0.275
0.267
0.260
0.257
0.254
0.250
0.251
Div-Dinev2-L.
1.188
1.083
1.033
1.007
0.987
0.976
0.967
0.951
0.948
Div-CLIP-S
0.317
0.288
0.273
0.263
0.256
0.252
0.249
0.246
0.246
Div-Dinov2-8
1.241
1,131
1.082
1.051
1.031
1.019
1,006
0.992
0.986
clip-score
0,2565
0.2697
0.2741
0.2763
0.2780
0,2788
0.2799
0.2817
U.2826
FID
14.649
11.260
12.056
13.147
14.179
15.032
15.663
17.478
18.718
Kneer
Div-Dinov2-L
DIv-CILIPL
0.320
1,209
0.300
1.119
0.239
1.076
0.255
1.048
0.275
1,030
0.271
1.016
0.288
1.006
0.262
0.986
0.259
0.979
Div-CLIP-S
0.321
0,302
0.291
0.282
0.277
0.271
0.270
0.263
0.261
Dinuvin
1.262
1.172
1.129
1.099
1.082
1.060
1.057
1.038
1.027
clip-score
0.2553
0.2686
0.2728
0.2751
0.2770
0.2782
0.2793
0.2812
0.2821
FID
15.725
L1.846
12.009
12.796
13.629
14.282
15.058
16.901
18.448
Div-CLIP-L
0.322
0.304
0.293
0.287
0.282
0.278
0.275
0.268
0.265
cos
Div-Dinov2-L
1.215
1.134
1.092
1.051
1.039
1,030
1.008
1.001
Div-CL
0.326
0.307
0.296
0.290
0.285
0.282
0.278
0.272
0.269
Div-Dinov2-8
1.266
1.186
1.145
1.120
1.104
1.093
1.081
1.063
1.054
Table 17: Experiment on SDXL with Diversity., we present FID vs. CLIP-Score (CS) for SDXL of 10K images, and we see the similar trending to Table 16 that the heuristic methods outperform the baseline, both on FID and Diversity.
3
11
13
15
20
clip-score
0.2712
0.2767
0.2791
0.2799
0. 1,2806
0.2817
0.2826
0 0,2832
0.2836
baseline
FID
24.36
24.93
25.71
26.06
26.17
27.23
27.46
28.05
29.60
Div-Dinov2-L
0.951
0.896
0.857
0.850
0.841
0.831
0.827
0.829
0.853
Div-Dinvv2-S
1.052
0.985
0.950
0.940
0.931
0.920
0.916
0.912
0.927
clip-score
0.2653
0.2732
0.2773
0.2789
0.2798
0.2810
0.2821
0.2828
0.2840
Bnear
FID
29.09
25.03
24.15
21.52
21.67
25.13
25.55
25.85
26. 60
L
0.999
0.949
0.916
0.904
0.897
0.881
0.873
0.863
Div-Dinov2-8
1.123
1.061
1.030
1.018
1.007
0.989
0.980
0.973
0.956
clip-score
0.2621
0.2708
0.2751
0.2764
0.2776
0.2794
0.2803
0.2817
0.2830
FID
32.83
27.00
23.55
25.41
25.43
25.52
25.74
25.88
26.84
cosine
Div-Dinov2-L
1.017
0.969
0.941
0.932
0.922
0.908
0.899
0.893
0.879
Div-Dinov2.8
1.143
1.095
1.066
1.056
1.045
1.031
1.020
1.008
0.994
33

---

Published in Transactions on Machine Learning Research (12/2024)
For the evaluation, each participant presented with total of 10 image sets. Each set comprised 9 images. Within each set, three pairwise comparisons were made: linear vs. baseline, and cosine vs. baseline. Throughout the study, two distinct image sets (20 images for each method) were utilized. We carried out two tests for results generated with stable diffusion v1.5 and each image are generated to make sure that their CLIP-Score are similar.
Subsequently, participants were prompted with three questions for each comparison:
Which set of images is more realistic or visually appealing?
2. Which set of images is more diverse!
Which set of images aligns better with the provided teat description?
In total, we recorded 54 participants with each participant responding to 90 questions. We analyzed the results by examining responses to each question individually, summarizing the collective feedback.
34


Reviews:
[{"review_id": "14qbSGFbV4", "reviewer": "Reviewer_5x2e", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n1. This paper includes a comprehensive analysis of CFG weight schedulers' results to fill the gap in the existing literature, which is helpful for both understanding each scheduler's behavior and further improving the generation performance. \n2. The study includes experiments across multiple tasks, models, datasets, and a real user study.\n3. The paper explores both complex parameterized methods and simple heuristic approaches and is filled with enough details for experimental setup and results.\n\nWeakness:\n1. The paper lacks technical contribution, while focusing on experimental analysis. \n2. While the paper provides empirical observations for different CFG schedulers, it is heavily focused on empirical results and doesn't offer a deep theoretical explanation for why these scheduling strategies work. Therefore, the results and observations may not generalize well on other different models and tasks, potentially limiting their broad applicability.\n3. Since this paper focuses on empirical observations, then including more negative cases where the simple linear scheduler fails to generalize might be interesting and potentially lead to valuable observations in the future.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper systematically studies the classifier-free guidance (CFG) for improving the performance of diffusion models. It provides a comprehensive empirical analysis of CFG through various experiments and a real user subjective study. Based on the observations, they suggest using a simple monotonically increasing hyperparameter-free weight scheduler, that could improve the generation performance with only 1 line of code implementation. In addition, they observed a lack of generalization for other tasks using parameterized schedulers with tuned parameters for one task.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Including theoretical analysis of the training behavior using different schedulers would be beneficial to this work.\n2.  While the proposed simple scheduler improved the performance, it would be interesting to include more discussion about when the proposed simple weight scheduler would fail to work, and provide more guidance on the selection of the scheduler based on different factors such as the model architectures.", "broader_impact_concerns": "There might be potential copyright and ownership issues. For example, Pikachu (shown in this paper Figure 7) is a trademarked character owned by Nintendo and The Pokémon Company. Using AI to generate images of Pikachu without proper authorization could potentially infringe on these copyrights and trademarks.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "yZ7eaQF6wN", "reviewer": "Reviewer_wiZb", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The empirical analysis is thorough and comprehensive.\n- The paper is clearly written and the figures are generally clear.\n- The proposed solution is straightforward and effective.\n\nWeaknesses:\n- While the empirical evidences are strong, I do find a general lack of insights into why dynamic scheduler (or monotonically increasing scheduler) works better.\n- It is unclear if the same conclusion holds for models not following DDPM schedule (e.g., rectified flow), which are becoming increasingly popular recently.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper conducts an in-depth analysis of guidance weights used in Classifier-Free Guidance. The authors systematically explore the role of dynamic weight schedulers, presenting empirical evidence that monotonically increasing schedulers (such as linear and cosine) outperform static guidance in fidelity and diversity. The paper also explores more complex, parameterized guidance weight schedulers, which allow for further performance improvements. These contributions provide valuable insights and practical recommendations for improving text-to-image generation performance.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Figure 5 caption seems wrong (a and b should be swapped).\n2. A theoretical analysis of the findings would be helpful.\n3. I would like to see if the conclusion of the paper holds for rectified flow models (e.g., SD3).", "broader_impact_concerns": "I have no broader impact concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "6HqqNVN3S2", "reviewer": "Reviewer_d2J5", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths ###\n\n- The paper is generally well-written, with lots of illustrations.\n- Diffusion models are very popular right now, both in terms of research and practical usage.\n- The experiments are sound and fairly thorough.\n- The paper ends by giving practical recommendation, which is useful.\n\n### Weaknesses ###\n\n- The paper is limited to text-to-image diffusion models, which is only a portion of the diffusion models out there. For example, adding some experiments for the case where the conditioning is different would greatly strengthen the work.\n- I am not sure of the significance of the work for the research community. From the results, I think the practical value is clear, but as the authors state, previous papers have mentioned the value of increasing guidance weight schedulers.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper performs an empirical analysis of classifier-free guidance weight schedulers, focusing on the text-to-image setting. It considers different types of heuristic schedulers and both qualitative and quantitative evaluation. The main conclusion is that in general monotonically increasing schedulers improve performance, but more complex parameterized schedulers perform well only in specific settings and do not generalize.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Critical:\n- Include experiments on diffusion models with alternative conditioning, such as image-to-image (https://huggingface.co/docs/diffusers/main/en/using-diffusers/img2img)", "broader_impact_concerns": "No broader impact concerns are stated, but I think that it's fine because the paper only investigates the inference behavior of existing models.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper offers a comprehensive empirical evaluation of CFG weight schedulers, demonstrating their consistent performance improvements with simple, monotonically increasing methods. Reviewers generally agree on its clarity, practical relevance, and thorough experimentation. The authors addressed most concerns, adding theoretical insights and broadening experiments, including rectified flow models and image-to-image tasks. The work is empirically strong and practically useful, and all the reviewers are leaning towards acceptance. Thus, I recommend acceptance.
Decision: Accept (poster)