TFG: Unified Training-Free Guidance for Diffusion Models

Authors: haotian  ye
haowei  lin
jiaqi  han
minkai  xu
sheng  liu
yitao  liang
jianzhu  ma
james  zou
stefano  ermon
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 04:33:27

Abstract: Given an unconditional diffusion model and a predictor for a target property of interest (e.g., a classifier), the goal of training-free guidance is to generate samples with desirable target properties without additional training. Existing methods, though effective in various individual applications, often lack theoretical grounding and rigorous testing on extensive benchmarks. As a result, they could even fail on simple tasks, and applying them to a new problem becomes unavoidably difficult. This paper introduces a novel algorithmic framework encompassing existing methods as special cases, unifying the study of training-free guidance into the analysis of an algorithm-agnostic design space. Via theoretical and empirical investigation, we propose an efficient and effective hyper-parameter searching strategy that can be readily applied to any downstream task. We systematically benchmark across 7 diffusion models on 16 tasks with 40 targets, and improve performance by 8.5% on average. Our framework and benchmark offer a solid foundation for conditional generation in a training-free manner.

Main paper: 
 introduction: 
 Recent advancements in generative models, particularly diffusion models [61, 21, 62, 66] , have demonstrated remarkable effectiveness across vision [65, 48, 52] , small molecules [74, 73, 24] , proteins [1, 72] , audio [35, 29] , 3D objects [40, 41] , and many more. Diffusion models estimate the gradient of log density (i.e., Stein score, [67] ) of the data distribution [65] via denoising learning objectives, and can generate new samples via an iterative denoising process. With impressive scalability to billions of data [58] , future diffusion models have the potential to serve as foundational generative models across a wide range of applications. Consequently, the problem of conditional generation based on these models, i.e., tailoring outputs to satisfy user-defined criteria such as labels, attributes, energies, and spatial-temporal information, is becoming increasingly important [63, 2] .Conditional generation methods like classifier-based guidance [66, 7] and classifier-free guidance [23] typically require training a specialized model for each conditioning signal (e.g., a noise-conditional classifier or a text-conditional denoiser). This resource-intensive and time-consuming process greatly limits their applicability. In contrast, training-free guidance aims to generate samples that align with certain targets specified through an off-the-shelf differentiable target predictor without involving any additional training. Here, a target predictor can be any classifier, loss function, probability function, or energy function used to score the quality of the generated samples.In classifier-based guidance [66, 7] , where a noise-conditional classifier is specifically trained to predict the target property on both clean and noisy samples, incorporating guidance in the diffusion Figure 1 : (a) Illustration of the unified search space of our proposed TFG, where the height (color) stands for performance. Existing algorithms search along sub-manifolds, while TFG results in improved guidance thanks to its extended search space. (b) The label accuracy (higher the better) and Fréchet inception distance (FID, lower the better) of different methods for the label guidance task on CIFAR10 [30] , averaged across ten labels. Ours (TFG-4) performs much closer to training-based methods. (c∼h) TFG generated samples across various tasks in vision, audio, and geometry domains.process is straightforward since the gradient of the classifier is an unbiased driving term. Trainingfree guidance, however, is fundamentally more difficult. The primary challenge lies in leveraging a target predictor trained solely on clean samples to offer guidance on noisy samples. Although various approaches have been proposed [18, 63, 6, 2, 78] and are effective for some individual tasks, theoretical grounding and comprehensive benchmarks are still missing. Indeed, existing methods fail to produce satisfactory samples for label guidance even on simple datasets such as CIFAR10 (Figure 1 ). Moreover, the lack of quantitative comparisons between these methods makes it difficult for practitioners to identify an appropriate algorithm for a new application scenario. This paper proposes a novel and general algorithmic framework for (and also named as) Training Free Guidance (TFG). We show that existing approaches are special cases of the TFG as they correspond to particular hyper-parameter subspace in our unified space. In other words, TFG naturally simplifies and reduces the study of training-free guidance, as well as the comparisons between existing methods, into the analysis of hyper-parameter choices in our unified design space. Within our framework, we analyze the underlying theoretical motivation of each hyper-parameter and conduct comprehensive experiments to identify their influence. Our systematic study offers novel insights into the principles behind training-free guidance, allowing for a transparent and efficient survey of the problem.Based on the framework, we propose a hyper-parameter searching strategy for general downstream tasks. We comprehensively benchmark TFG and existing algorithms across 16 tasks (ranging from images to molecules) and 40 targets. TFG achieves superior performance across all datasets, outperforming existing methods by 8.5% on average. In particular, it excels in generating userrequired samples in various scenarios, regardless of the complexity of targets and datasets.In summary, we (1) propose TFG that unifies existing algorithms into a design space, (2) theoretically and empirically analyze the space to propose an effective space-searching strategy for general problems, and (3) benchmark all methods on numerous qualitatively different tasks to present the superiority of TFG and the guideline for future research in training-free conditional generation algorithms. This advancement demonstrates the efficacy of TFG and establishes a robust and comprehensive benchmark for future research in training-free conditional generation algorithms.Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?Answer: [Yes] Justification: We have both theoretically and empirically justified our contributions.• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
background: 
 Generative diffusion model. A generative diffusion model is a neural network that can be used to sample from an unconditional distribution p 0 (x) with the support on any continuous sample space X [21, 62, 64, 27] . For instance, X could be [-1, 1] d×d×3 representing the RGB colors of d × d images [4, 22] , or R 3d representing the 3D coordinates of molecules with d atoms [24, 74, 73] . Given a data x 0 sampled from p 0 (x), a time step t ∈ [T ] ≜ {1, • • • , T }, a corresponding noisy datapoint is constructed as x t = √ ᾱt x 0 + √ 1 -ᾱt ϵ where ϵ ∼ N (0, I) and {ᾱ t } T t=1 is a set of pre-defined monotonically decreasing parameters used to control the noise level. Following [21] , we further define α t = ᾱt /ᾱ t-1 for t > 1 and α 1 = ᾱ1 . The diffusion model ϵ θ : X × [T ] → X parameterized by θ is trained to predict the noise ϵ that was added on x t with p.d.f p t (x t ) = x0 p 0 (x 0 )p t|0 (x t |x 0 )dx 0 2 . In theory, this corresponds to learning the score of p t (x) [65] , i.e., arg minEQUATIONFor sampling, we start from x T ∼ N (0, I) and gradually sample x t-1 ∼ p t-1|t (x t-1 |x t ). This conditional probability is not directly computable, and in practice, DDIM [62] samples x t-1 viaEQUATIONwhere {σ t } T t=1 are DDIM parameters, ϵ ∼ N (0, I), andEQUATIONis the predicted sample given x t . According to the Tweedie's formula [11, 51] , x 0|t equals to the conditional expectation E[x 0 |x t ] under perfect optimization of ϵ θ in Equation (1) . It has been theoretically established that the above sampling process results in x 0 ∼ p(x) under certain assumptions.Target predictor. For a user required target c, we use a predictor f c (x) : X → R + ∪ {0} 3 to represent how well a sample x is aligned with the target (higher the better). Here f c (x) can be a conditional probability p 0 (c|x) for a label c [62, 14] , a Boltzmann distribution exp -ec(x) for any pre-defined energy function e c [31, 63, 38] , the similarity of two features [47] , or even their combinations. The goal is to samples from the conditional distribution p 0 (x|c) ≜ p 0 (x)f c (x)x p 0 ( x)f c ( x)d x .(4) 2 In this paper, we use p(x) to represent the probability density function (p.d.f.), and pt(x), p t|s (x| x) to represent the probability at time step t and the conditional probability of x at time step t given x at time step s. 3 Here c can has any mathematical form. We assume in this paper that fc(x) has finite two norm, i.e.x∈X [f 2 c (x)] < +∞, such that the probabilistic explanation is well-defined.Training-based guidance for diffusion models. [66] proposes to train a time-dependent classifier to fitf c (x t , t) ≜ E x0∼p 0|t (•|xt) f c (x 0). This can be regarded as a predictor over noisy samples. SinceEQUATIONif we denote the trained classifier as f (x t ) (that implicitly depends on c and model parameters), we can replace ϵ θ (x t , t) in Equation ( 3) by ϵ θ (x t , t) -√ 1 -ᾱt ∇ xt log f (x t ) upon sampling to obtain unbiased sample x 0 ∼ p 0 (x 0 |c). On the other hand, [23] proposes the classifier-free diffusion guidance approach. Instead of training a time-dependent predictor f , it encodes conditions c directly into the diffusion model as ϵ θ (x, c, t) and trains this condition-aware diffusion model with samplecondition pairs. Both methods have been proven effective when training resources are available.This paper in contrast focuses on conditional generation in a training-free manner: given a diffusion model ϵ θ (x, t) and an off-the-shelf target predictor f (x) (we omit the subscript c below), we aim to generate samples from p 0 (x|c) without any additional training. Unlike training-based methods that can accurately estimate f (x t , t), training-free guidance is significantly more difficult since it involves guiding a noisy data x t using f (x) defined over the clean data space.Most existing methods take advantage of the predicted sample x 0|t defined in Equation ( 3) and use the gradient of f (x) for guidance. We review and summarize five existing approaches below, and provide a schematic and a copy of pseudo-code in Appendix B for the sake of reference. Due to the variety in underlying intuitions and implementations, coupled with a lack of quantitative comparisons among these methods, it is challenging to discern which operations are crucial and which are superfluous, a problem we address in Section 3.DPS [6] was initially proposed to solve general noisy inverse problems for image generation: for a given condition y and a transformation operator A, we aim to generate image x such that ∥A(x)-y∥ 2 is small. For instance, in super-resolution task [71] , the operator A is a down-sampling operator, and y is a low-resolution image. DPS replaces ∇ log f (x t , t) in Equation ( 5) by ∇ xt log f (m(x t )). As suggested in [63] , this corresponds to a point estimation of the conditional density p 0|t (x 0 |x t ).LGD [63] replaces the point estimation in DPS and proposes to estimate f (x t , t) with a Gaussian kernel E x∼N (x 0|t ,σ 2 t I) f (x, t), where the expectation is computed using Monte-Carlo sampling [59] . FreeDoM [78] generalizes DPS by introducing a "recurrent strategy" (called "time-travel strategy" [39, 10, 70] ) that iteratively denoises x t-1 from x t and adds noise to x t-1 to regenerate x t back and forth. This strategy empirically enhances the strength of the guidance at the cost of additional computation. FreeDoM also points out the importance of altering guidance strength at different time steps t, but a comprehensive study on which schedule is better is not provided.MPGD [18] is proposed for manifold preserving tasks, e.g., the target predictor is supposed to generate samples on a given manifold. It computes the gradient of log f (x 0|t ) to x 0|t instead of x t , i.e., ∇ x 0|t log f (x 0|t ) to avoid the back-propagation through the diffusion model ϵ θ that is highly inefficient. This strategy is effective in manifold-preserving problems, but whether it can be generalized to general training-free problems is unclear. In addition to the computation difference, theoretical understanding on the difference between gradients to x 0|t and x t is missing. 4UGD [2] builds on FreeDoM, with the difference that it additionally solves a backward optimization problem ∆ 0 = arg max ∆ f (x 0|t + ∆) and guides x 0|t and x t simultaneously. UGD also implements the "recurrent strategy" to further improve generation quality.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes] Justification: See Sec. 6.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
tfg: a unified framework for training-free guidance: 
 Despite the array of algorithms available and their reported successes in various applications, we conduct a case study on CIFAR10 [30] to illustrate the challenging nature of training-free guidance and the insufficiency of existing methods. Specifically, for each of the ten labels, we use the pretrained diffusion model and classifiers from [7, 9] to generate 2048 samples, where the hyper-parameters are selected via a grid search for the fairness of comparison. We compute the FID and the label accuracy evaluated by another classifier [20] and present results in Figure 1 . Even in such a relatively simple setting, all training-free approaches significantly underperform training-based guidance, with a significant portion of generated images being highly unnatural (when guidance is strong) or irrelevant to the label (when guidance is weak). These findings reveal the fundamental challenges and highlight the necessity of a comprehensive study. Unfortunately, comparisons and analyses of existing approaches are missing or primarily qualitative, limiting deeper investigation in this field.This sections introduces our unified framework for training-free guidance (TFG, Algorithm 1) and formally defines its design space in Definition 3.1. We demonstrate the advantage of TFG by drawing connections between TFG and other algorithms to show that existing algorithms are encompassed as special cases. Based on this, all comparisons and studies of training-free algorithms automatically become the study within the hyper-parameter space of our framework. This allows us to analyze the techniques theoretically and empirically, and choose an appropriate hyper-parameter for a specific downstream task efficiently and effectively, as shown in Section 4. Definition 3.1. Given a denoising step T , the hyper-parameter space (design space) of Algorithm 1 is defined as1: Input: Unconditional diffusion model ϵ θ , target predictor f , guidance strength ρ, µ, γ, number of steps T, Nrecur, Niter 2: xT ∼ N (0, I) 3: for t = T, • • • , 1 do 4: Define function f (x) = E δ∼N (0,I) f (x + γ√ 1 -ᾱtδ) 5: for r = 1EQUATIONWe use H TFG to represent the complete hyper-parameter space and H TFG (N recur = N 0 ) to represent the subspace constrained on N recur = N 0 . Definition 3.1 defines the hyper-parameter space spanned by TFG, where one hyper-parameter in H TFG is an instantiation of the framework. Intuitively, N recur controls the recurrence of the algorithm, N iter controls the iterating when computing ∆ 0 (Line 8), γ controls the extent we smooth the original guidance function f (Line 4), and ρ, µ control the strength of two types of guidance (Lines 7 and 8). A comprehensive explanation of the effect of each hyper-parameter can be found in Section 3.2.Below is the major theorem showing that all algorithms presented in Section 2.1 correspond to special cases of TFG, thus unifying them into our framework and obviating the need for separate analyses. Theorem 3.2. The hyper-parameter space of• MPGD [18] H MPGD is equivalent to H TFG (N recur = N iter = 1, ρ = 0, γ = 0).• LGD [63] H LGD is equivalent to H TFG (N recur = 1, N iter = 0, µ = 0).• UGD [2] H UGD is equivalent to H TFG (γ = 0).• DPS [6] H DPS is equivalent to H TFG (N recur = 1, N iter = 0, µ = 0, γ = 0).• FreeDoM [78] H FreeDoM is equivalent to H TFG (N iter = 0, µ = 0, γ = 0).The complete analysis and proof of Theorem 3.2 is postponed to Appendix C. It implies that existing algorithms are limited in expressivity, covering only a subset of H TFG . In contrast, TFG covers the entire space and is guaranteed to perform better. In addition, TFG streamlines nuances between existing methods, allowing for a unified way to compare and study different techniques. Consequently, the versatile framework that TFG provides can simplify its adaptation to various applications.We now present a concrete analysis of TFG and its design space H in detail. Similar to standard classifier-based guidance, TFG guides x t at each denoising step t. To provide appropriate and informative guidance, TFG essentially leverages four techniques for guidance: Mean Guidance (Line 8) controlled by N iter , µ, Variance Guidance (Line 7) controlled by ρ, Implicit Dynamic (Line 4) controlled by γ, and Recurrence (Line 5) controlled by N recur .Mean Guidance computes the gradient of f (x) to x 0|t and is the most straightforward approach. However, this method can yield inaccurate guidance. To show this, notice that under perfect optimization we havex 0|t = E[x 0 |x t ],and when p 0 (E[x 0 |x t ]) is close to zero, the predictor has rarely been trained on data from the region close to x 0|t , making the gradient unstable and noisy. To mitigate this, one can iteratively add gradients of f (x) to x 0|t , encouraging x 0|t to escape low-probability regions.Variance Guidance provides an alternative approach for improving the gradient estimation. The reason why we dub it variance guidance might be ambiguous, as the only difference is that the gradient is taken with respect to x t (Line 7) instead of x 0|t (Line 8). The lemma below demonstrates that this essentially corresponds to a covariance re-scaled guidance. Lemma 3.3. If the model is optimized perfectly, i.e., ϵ θ (x, t) = -√ 1 -ᾱt ∇ log p t (x), we haveEQUATIONwhere Implicit Dynamic transforms the predictor f into its convolution via a Gaussian kernel N (0, γ(1 -ᾱt )I). This operation is initially introduced by LGD [63] to estimate p 0|t (x 0 |x t ). However, it is unclear why the form is preselected as a Gaussian distribution. We argue that this technique is effective because it creates an implicit dynamic on x 0|t . Specifically, starting from x 0|t , it iteratively adds noise to x 0|t , evaluates gradient, and moves x 0|t based on the gradient. The repeating process converges to the density proportional to f (x) when N iter goes to infinity, driving x 0|t to high-density regions. This explanation is justified by Table 1 : the performance remains nearly unchanged as we gradually decrease the number of Monte-Carlo samples in estimating the expectation (Line 4) down to 1, implying that the preciseness of estimation is not essential, but adding noises is.Σ 0|t ≜ x p 0|t (x|x t )(x -E[x 0 |x t ])(x -E[x 0 |x t ]) ⊤ dxRecurrence helps strengthen the guidance by iterating the previous three techniques to obtain x t-1 and resample x t back and forth. This can be understood as an Ornstein-Uhlenbeck process [42] on x t-1 where Line 6∼9 corresponds to the drift term and x t-1 → x t (Line 10) the white noise term. Intuitively, it finds a trade-off between the error inherited from previous steps (the more you recur, the 8) of ρ and µ on CIFAR10 and ImageNet, under different choices of the rest hyper-parameters in H TFG . We set ρ = 0, γ = 0 when studying structures of µ, and similarly for ρ. Results are averaged across all labels. The comparative relationship between structures remains unchanged when the rest of the parameters vary.less previous error stays) and the accumulated error in this step (the more you recur, the more error in the current guidance you suffer). Empirically, we also find that the generation quality improves and then deteriorates as we increase N recur .Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer: [Yes]• It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
design space of tfg: analysis and searching strategy: 
 Admittedly, a more extensive design space only yields a better performance if an effective and robust hyper-parameter searching strategy can be applied. For example, arbitrarily complex neural networks are guaranteed to have better optimal performance than simple linear models, but finding the correct model parameters is significantly more difficult. This section dives into this core problem by comprehensively analyzing the hyper-parameter space structure of H TFG , and further proposing a general searching algorithm applicable for any general downstream tasks.The hyper-parameters of H TFG can be categorized into two parts: time-dependent vectors ρ, µ, and time-independent sacalars N recur , N iter , γ. While a grid search can potentially result in the best performance, performing such an extensive search in H TFG is highly impractical, especially considering the vector parameters ρ, µ. Fortunately, below we demonstrate that, if we decompose ρ into ρ • s ρ (t) (same for µ) where ρ is a scalar and s ρ (t) is a "structure" (a non-negative function) such that t s ρ (t) = T , then some structures are consistently better than others regardless of the other hyper-parameters. This allows us to pre-locate an appropriate structure for the given task and efficiently optimize the rest of the scalar hyper-parameters. Our analysis is conducted on the label guidance task on CIFAR-10 [30] and ImageNet [55] , with experimental settings identical to Section 3.Structure analysis. Motivated by the default structure selected in UGD and LGD, we consider three structures for both s ρ (t) and s µ (t) asEQUATIONThese structures are selected to be qualitatively different, while each is justified to be reasonable under certain conditions [18, 78, 2] . We leave the study of more structures to future works. The rest of the parameters are grid-searched for the comprehensiveness of the analysis. For s ρ (t), we set N recur = {1, 2, 4} and ρ = {0.25, 0.5, 1.0, 2.0, 4.0}; and for s µ (t), we set N iter = {1, 2, 4} and μ = {0.25, 0.5, 1.0, 2.0, 4.0}. We run label guidance for each configuration and each of the ten labels on CIFAR10 (four labels on ImageNet, due to computation constraints).As presented in Figure 2 , the relationship between different structures remains unchanged when the rest of the parameters vary. For instance, on both datasets, the Validity-FID performance curves consistently move top-left (implying a better performance) when we switch from "decrease" structure (red lines) to "constant" structure (yellow lines) to "increase" structure (blue lines) for both ρ, µ and different values of N recur and N iter . This invariant relationship is essential as it allows for an efficient hyper-parameters search in H TFG by first determining appropriate structures for s ρ (t), s µ (t) under a simple subspace, and then selecting the rest scalar parameters. Computation cost analysis. Among the scalars parameters, N recur and N iter directly influence the total computational cost, while ρ, μ, γ do not. With a certain range, performance increases when the value of N recur , N iter increase 5 , and the tradeoff between generation quality and computation time is presented in Figure 3 : recurrence leads to a N recur times cost with clear performance gain; iteration (on x 0|t ) results in less increase of computation time, and its effect plateaus. In practice, users can determine their values based on computation resources, but an upper bound of 4 suffices to unlock a near-optimal performance.Searching strategy. The above analysis successfully simplify the task-specific hyper-parameter search problem without significant performance sacrifice. It remains to be decided the scalar values ρ, μ, γ. Here we propose a strategy based on beam search to effectively and efficiently select their values. Specifically, our searching strategy starts with an initial set T = {(ρ init , μinit , γinit )}, where these initial values are small enough to approximate TFG as an unconditional generation. At each searching step, for each tuple in T , we separately double the values of ρ, μ, and γ to generate up to 3|T | new configurations. We conduct a small-sized generation trial for each new configuration and update T to be the top K configurations with the highest evaluation results that are determined by user requirements (e.g., accuracy, FID, or a combination). This iterative process is repeated until T stabilizes or the maximum number of search steps is reached. Notice that this process is conducted with a much smaller sample size, and consequently, the computation time is highly controllable. 
benchmarking: 
 This section comprehensively benchmarks training-free guidance under the TFG framework and the design searching strategy in Section 4. We consider 7 datasets, 16 different tasks, and 40 individual targets with a total experimental cost of more than 2,000 A100 GPU hours. For comparison, we also run experiments for each of the existing methods (where the design searching is conducted in the corresponding subspace). All methods, tasks, search strategies, and evaluations are unified in our codebase, with details specified in Appendices D and E.Diffusion models. (1) CIFAR10-DDPM [48] is a U-Net [54] model trained on CIFAR10 [30] images.(2) ImageNet-DDPM [7] is an larger U-Net model trained on ImageNet-1k [55] images. (3) Cat-DDPM is trained on Cat [12] images. ( 4) CelebA-DDPM is trained on CelebA-HQ dataset [26] that consists millions of human facial images. ( 5) Molecule-EDM [24] is an equivariant diffusion Other settings. We consistently set the time step T = 100 and the DDIM parameter η = 1. We consider N recur = 1, N iter = 4 and use a single sample for Implicit Dynamic (Line 4) throughout all experiments and methods for fair comparison. For TFG, the structures of ρ and µ are set to "increase" and the scalars ρ, μ, γ are determined via our searching strategy. We follow the setting in original papers if they specify their hyper-parameters. blueFor specific tricks in the code that are not mentioned in papers, we choose to align with original papers. Otherwise, values are determined via searching with 1/8 of the sample size and a maximum search step of 6. For fairness of comparison, we use accuracy as the metric during the search and compare different algorithms on the metric, but we report both accuracy and FID.We compare all six methods in Table 3 . TFG outperforms existing algorithms in 13 over 14 settings, achieving an average guidance validity improvement of 7.4% compared to the best existing algorithm. Notice that we do not compare with the best algorithm in terms of generation fidelity because obtaining high realness samples is not our objective in training-free guidance, and an unconditional model suffices to generate high realness samples (with extremely low validity). Interestingly, different methods achieve the second best performance on different tasks, suggesting the variance of these methods, while TFG is consistent thanks to the unification.We want to highlight that despite the superior performance of TFG, the key intention of our experiments is not restrained to comparing TFG with existing methods, but more importantly to systematically benchmark under the training-free guidance setting to see how much we have achieved in various tasks with different difficulties. Below we go through each task separately and conduct relevant ablation studies to provide a more fine-grained analysis.Fine-grained label guidance. In addition to the standard label guidance, we for the first time study the out-of-distribution fine-grained label guidance under the training-free setting, a problem where no existing training-based methods are available. We consider the bird-species guidance using an EfficienNet trained to classify 525 fine-grained bird species. This problem remains highly difficult for leading text-to-image generative models such as DALLE. Under recurrence, TFG can generate at most 2.24% of accurate birds, compared with the unconditional generation rate of 0. Recurrence on label guidance. We go back to the failure case we study in Section 3, i.e., the standard label guidance problem on CIFAR10 where the training-based method offers an 85% accuracy, while the accuracy of TFG without recurrence accuracy is 52% only. As presented in Table 4 , increasing N recur significantly closes the gap from 33% to 8%. Similar improvement is observed in other datasets as well.Table 5 : The accuracy of multi-label guidance on CelebA, where labels 0 and 1 correspond to female and male (gender), nonblonde and blonde (hair color), and young and old (age). The accuracy is lower for minority groups, indicating an implicit bias in the generation process. Despite this, it is still much higher than unconditional generation. Molecule property guidance. To our knowledge, we are the first to study training-free guidance for molecule generation. We interestingly find in Table 3 that TFG is effective in guiding molecules towards desirable properties, yielding the highest guidance validity on 5 out of 6 targets with 5.64% MAE improvement over existing methods, verifying the generality of our approach as a unified framework in completely unseen domains. Notice that, unlike images, molecules with better validity usually have lower generation fidelity, a finding reflected in previous work [3] .Audio Guidance. We extend our investigation to the audio modality, where TFG achieves significant relative improvements over existing methods. Given that the audio domain is rarely explored in training-free guidance literature, our benchmarks will contribute to future research in this area. 
discussions and limitations: 
 Recently, training-free guidance for diffusion models has gained increasing attention and has been adopted in various applications. TFG is based on an extensive literature review over ten algorithmic papers for different purposes, including images, audio, molecules, and motions [34, 8, 43, 32, 19, 13, 16, 15, 45, 38, 68] . While we incorporate several key algorithms into our framework, we acknowledge that encompassing all approaches is impossible, as it would make the unification bloated and less practical. We seek to find a balance point by unifying most representative algorithms while keeping the techniques clear and easily studied.An often discussed problem is why we care about training-free guidance, given the ever-growing community of language-based generative models such as the image generator of GPT4. In practice, there are countless conditional generation tasks where the conditions are hard to accurately convey to or represent by language encoders. For instance, it can fail to under a complex property of a molecule or generate CelebA-style faces. We give an illustrative analysis in Appendix A. In this section, we argue that training-free guidance using off-the-shelf models is a crucial and timely research problem deserving more attention and effort. We begin by providing an illustrative analysis that highlights the limitations of current strong text-to-image generative models, underscoring the necessity for training-free guidance. Furthermore, we assert that training-free guidance remains a significant challenge, with previous literature underestimating its complexity. Given its necessity and inherent difficulties, we call for increased focus from the research community on this problem and offer our benchmarks and code base to help accelerate progress in this area.A.1 Failure case of image generation with GPT4Figure 4 : Prompting GPT4 to generate property guided molecules. It is hard for the image generator to understand the target and generate faithful samples. In this dialog, GPT4 claims to generate a benzene molecule but the sample is apparently not a benzene. There are also many invalid carbon atoms with more than 4 bonds and the polarizability target is not achieved.It's hard for GPT4 to understand targets. In Figure 4 , we ask GPT-4 to generate a molecule with polarizability α = 3, which is a task we use to evaluate training-free guidance (refer to Figure 16 for visualization). We found that the GPT-4 generated molecule is apparently invalid and unrealistic: the generated molecule contains many carbon atoms with more than 4 bonds (the maximum allowed number is 4); and the generated molecule is apparently not a benzene which is claimed by the text outputs. From this case we may understand that it is hard to follow diverse user-defined instructions for the foundational generative models, where the user-defined targets may be subtle, fine-grained, combinatorial, and open-ended.To this end, training-free guidance offers two key advantages: (1) It allows for greater specificity in target requirements by enabling the use of a differentiable objective function, making the generation process more steerable; (2) The objective function is plug-and-play, facilitating the addition of new targets and tasks to a pre-trained generative model. Since there is no need to retrain the diffusion or prediction models, this approach makes the generative process lightweight and applicable to various downstream tasks.It's hard for GPT4 to capture the targeted distribution. Another important metrit for trainingfree guidance is the flexibility of choosing diffusion models. For the same target, we can switch from different diffusion models to change the unconditional background distribution. For example, it is hard for GPT4 to generate CelebA-like samples though it "knows CelebA dataset very well": Despite the array of algorithms available and their reported successes in various applications, we conduct a case study on CIFAR10 [30] to illustrate the challenging nature of training-free guidance and the insufficiency of existing methods. Specifically, we compare the training-based approach and training-free approach for the label guidance task on CIFAR10, with the diffusion model pretrained by [7] , the training-based time-dependent classifier f (x, t) by [7] , and the training-free standard label classifier f (x) pretrained only on clean CIFAR10 by [9] . and a "fake" training-free classifier defined as f (x, t)| t=0 . The first serves as the oracle benchmark, while the second corresponds to the standard training-free guidance. The third setting, as considered in LGD [63] , uses a "fake" training-free classifier since its parameters are shared across different time steps t during training, resulting in an implicit regularization that is not available for practical predictors. This setting serves as a comparison to help identify the difficulty of training-free guidance.Quantitative and qualitative results are shown in Figure 6 . All training-free approaches significantly under-perform training-based guidance, with a significant portion of generated images being highly unnatural (when guidance is strong) or irrelevant to the label (when guidance is weak). A more clear illustration on this can be found in Figure 7 . In terms of "fake" classifiers, it leads to a remarkable difference from real training-free classifiers even under identical experimental settings. It generates much less messy images due to the implicit regularization from the training process, where noisy images are also "seen" (although t is fixed to 0 upon guidance). Unfortunately, such types of predictors are inaccessible in practice (otherwise, we can use classifier-based guidance directly).From the comparison, we know that the key challenge of training-free guidance is the lack of a "smoothing" classifier that can produce faithful guidance in the "unseen" noisy image space.The observation largely uncovers the essential difficulties of training-free guidance, and motivates us to systematically study techniques that can improve generation quality. Unfortunately, comparisons between existing techniques are ambiguous since different methods are tested on distinct and primarily qualitative applications, which in turn hinders the in-depth study in this field. To this end, we resolve to revisit this complicated scenario and design a clear and comprehensive framework for training-free guidance.We have presented the pseudo-code of TFG in Algorithm 1. Below, we provide a copy of the DPS (Algorithm 2), MPGD (Algorithm 3), FreeDoM (Algorithm 4), UGD (Algorithm 5), and LGD (Algorithm 6). Notice that LGD does not provide a pseudo-code, and we present their algorithm following their paper as a modification of DPS. We do not change the original algorithms' notations for reference. Please see the proof in Appendix C for the equivalence analysis. We provide a schematic of existing algorithms in Figure 8 .Algorithm 2 DPS -Gaussian Require: N , y, {ζi} N i=1 , {σi} N i=1 xN ∼ N (0, I) for i = N -1 to 0 do ŝ ← s θ (xi, i) x0 ← 1 √ ᾱi (xi + (1 -ᾱi)ŝ) z ∼ N (0, I) x ′ i-1 ← √ α i (1-ᾱi-1 ) 1-ᾱi xi + √ ᾱi-1 β i 1-ᾱi x0 + σiz xi-1 ← x ′ i-1 -ζi∇x i ∥y -A(x0)∥ 2 2end for return x0Algorithm 3 MPGD for pixel diffusion models1: xT ∼ N (0, I) 2: for t = T, . . . , 1 do 3: ϵt ∼ N (0, I) 4: x 0|t = 1 √ ᾱt (xt - √ 1 -ᾱtϵθ(xt, t)) 5:if requires manifold projection then 6:x 0|t = gM(x 0|t , L(x 0|t ; y), ct) 7: else 8:x 0|t = x 0|t -ct∇x 0|t L(x 0|t ; y) 9:end if 10:xt-1 = √ ᾱt-1x0|t 11:+ 1 -ᾱt-1 -σ 2 t ϵ θ (xt, t) + σtϵt 12: end for 13: return x0Require: condition c, unconditional score estimator s(•, t), time-independent distance measuring function D θ (c, •), pre-defined parameters βt, ᾱt, learning rate ρt, and the repeat times of time travel of each step {r1, • • • , r T }.x T ∼ N (0, I) for t = T, ..., 1 do for i = rt, ..., 1 do ϵ1 ∼ N (0, I) if t > 1, else ϵ1 = 0. xt-1 = (1 + 1 2 βt)xt + βts(xt, t) + √ βtϵ1 x 0|t = 1 √ ᾱt (xt + (1 -ᾱt)s(xt, t)) gt = ∇x t D θ (c, x 0|t (xt))) xt-1 = xt-1 -ρtgt if i > 1 then ϵ2 ∼ N (0, I) xt = √ 1 -βtxt-1 + √ βtϵ2Require: N , y, {ζi} N i=1 , {σi} N i=1 , n xN ∼ N (0, I) for i = N -1 to 0 do ŝ ← s θ (xi, i) x0 ← 1 √ ᾱi (xi + (1 -ᾱi)ŝ) z ∼ N (0, I) xi-1 ← √ α i (1-ᾱi-1 ) 1-ᾱi xi + √ ᾱi-1 β i 1-ᾱi x0 + σiz xi-1 ← xi-1 -ζi∇x i log 1 n n j=1 exp(-ℓy(x (j) i ) ▷ x (j) i sampled i.i.d. from N (x0,We prove Theorem 3.2 and Lemma 3.3 below.Proof. For each algorithm, we prove the equivalence of design space separately below. Notice that when γ = 0, f degrades back to f . MPGD (Algorithm 3). Below we demonstrate that any hyper-parameter {c t } T t=1 in Algorithm 3 is equivalent to the TFG with f (x 0|t ) = exp{-L(x 0|t ; y)} and hyper-parameterN recur = 1, N iter = 1, γ = 0, ρ = 0, µ = (c 1 , • • • , c T ) ⊤ .To show this, notice that since both ρ and γ are zero, Line 4 and Line 7 take no effect. When using the identical sampling function (Line 9), TFG generates x t-1 usingx t-1 = Sample(x t , x 0|t , t) + c t √ ᾱt-1 ∇ x 0|t log f (x 0|t ) = √ ᾱt-1 x 0|t + 1 -ᾱt-1 -σ 2 t x t - √ ᾱt x 0|t √ 1 -ᾱt + σ t ϵ t + c t √ ᾱt-1 ∇ x 0|t log f (x 0|t ) = √ ᾱt-1 (x 0|t -c t ∇ x 0|t L(x 0|t ; y)) + 1 -ᾱt-1 -σ 2 t ϵ θ (x t , t) + σ t ϵ t , which is exactly the formula used in MPGD.DPS (Algorithm 2) and FreeDoM (Algorithm 4). We prove both algorithms together as DPS is a special case of FreeDoM (without recurrence). Specifically, any hyper-parameter {ρ t } T t=1 , time travel step r and distance function D(c, •) in Algorithm 4 is equivalent to TFG with f (x 0|t ) = exp{-D(c, x 0|t )} and hyper-parameterN recur = r, N iter = 0, γ = 0, ρ = ( √ α 1 ρ 1 , • • • , √ α T ρ T ) ⊤ , µ = 0.To show this, notice that both algorithms have the identical resampling step from x t-1 to x t , so it suffices to prove that the formula to generate x t-1 in each recurrent step is the same. For FreeDoM, we havex t-1 = Sample(x t , x 0|t , t) -ρ t ∇ xt D(c, x 0|t ) = Sample(x t , x 0|t , t) -( √ α t ρ t )∇ xt log f (x 0|t )/ √ α t ,and the last line equals the combination of Line 7 and Line 7 in TFG.LGD (Algorithm 6). Any hyper-parameter {ζ t } T t=1 , n in LGD is equivalent to TFG with f (x) = exp{-ℓ y (x)}, sample size n in Line 4, and hyper-parameterN recur = 1, N iter = 0, γ = 1, ρ = 0, µ = (ζ 1 , • • • , ζ T ) ⊤ .Notice that ᾱt in TFG equals 1/(1 + σ 2 t ) in the DPS algorithm. With this, the equivalence is clear from the pseudo-code of both algorithms.Any hyper-parameter k, m, s(t) in UGD is equivalent to TFG with f (x) = exp{-ℓ(c, f (x))} and hyper-parameterN recur = k, N iter = m, γ = 0, ρ = (- √ α 1 s(1)δ 1 , • • • , - √ α T s(T )δ T ) ⊤ , µ = (- α 1 1 -ᾱ1 δ 1 , • • • , - α T 1 -ᾱT δ T ) ⊤ ,where δ t is the coefficient of ϵ θ (x t , t) in sampler S in the UGD algorithm (which is negative). To show this, notice that∆ t = ρ t ∇ xt log f (x 0|t ) = -ρ t ∇ xt ℓ(c, f (x 0|t )) = √ α t s(t)δ t ∇ xt ℓ(c, f (x 0|t )). By replacing this into Line 9, the equivalence of guidance Variance Guidance can be easily observed. A similar deduction can be made for the mean guidance as well.Proof. Recall thatx 0|t = xt- √ 1-ᾱtϵθ(xt,t) √ ᾱt. According to a simple chain rule, we haveEQUATIONThe perfect optimization assumption implies the relationship between ϵ θ and the score of p t (x t ), which we leverage and obtainEQUATIONEQUATIONThus, it remains to draw a connection between the conditional covariance between Σ 0|t and ∇ 2 xt log p t (x t ). Omit the subscript x t in ∇ xt , we haveEQUATIONNotice that∇ 2 p t|0 (x t |x 0 ) = p t|0 (x t |x 0 )∇ 2 log p t|0 (x t |x 0 ) + 1 p t|0 (x t |x 0 ) ∇p t|0 (x t |x 0 )(∇p t|0 (x t |x 0 )) ⊤ = - p t|0 (x t |x 0 ) 1 -ᾱt I + p t|0 (x t |x 0 )( x t - √ ᾱt x 0 1 -ᾱt )( x t - √ ᾱt x 0 1 -ᾱt ) ⊤ .Replacing the LHS in the above equation in Equation ( 14), and noticing that E[x0 |x t ] = xt+(1-ᾱt)∇ log pt(xt) √ ᾱt, we have∇ 2 log p t (x t ) = - 1 1 -ᾱt I + x0∈X p 0|t (x 0 |x t )( x t - √ ᾱt x 0 1 -ᾱt )( x t - √ ᾱt x 0 1 -ᾱt ) ⊤ dx 0 -∇ log p t (x t )(∇ log p t (x t )) ⊤ = - 1 1 -ᾱt I + x0∈X p 0|t (x 0 |x t )( x t - √ ᾱt x 0 1 -ᾱt )( x t - √ ᾱt x 0 1 -ᾱt ) ⊤ dx 0 - √ ᾱt E[x 0 |x t ] -x t 1 -ᾱt √ ᾱt E[x 0 |x t ] -x t 1 -ᾱt ⊤ = - 1 1 -ᾱt I + Cov √ ᾱt x 0 -x t 1 -ᾱt |x t = - 1 1 -ᾱt I + ᾱt (1 -α t ) 2 Σ 0|t .The proof is finished by substituting ∇ 2 log p t (x t ) in Equation ( 12) by the above result.In computer vision, the Gaussian deblur task addresses the challenge of restoring clarity to images that have been blurred by a Gaussian process. Gaussian blur, a common image degradation, simulates effects such as out-of-focus photography or atmospheric disturbances. It is characterized by the convolution of an image with a Gaussian kernel, a process that spreads the pixel values outwards, leading to a smooth, uniform blur [33] . The deblurring task seeks to reverse this effect, aiming to retrieve the original sharp image.Guidance target. Specifically, we apply a 61 × 61 sized Gaussian blur with kernal intensity 3.0 to original 256 × 256 images. A random noise with a variance of σ 2 = 0.05 2 is added to the noisy images. If we denote the above process as a blurring operator A blur : x → y, where x ∈ R 256×256×3 , y ∈ R 256×256×3 are original images and noisy images, then the target of Gaussian Deblur is to generate a clean image x 0 such that:max x0 p(x 0 ) = max x0 exp(-∥A blur (x 0 ) -y∥ 2 ),which means if we project the generated image into the noisy space, the noisy samples A blur (x) should be similar to the ground truth noisy images y.Evaluation metrics. In our experiments, we evaluate each guidance method on a set of 256 samples generated by Cat-DDPM. We apply the FID [26] to assess the fidelity, Learned Perceptual Image Patch Similarity (LPIPS) [80] to evaluate the guidance validity. Super-resolution in computer vision refers to the process of enhancing the resolution of an imaging system, aimed at reconstructing a high-resolution image from one or more low-resolution observations. This technique is fundamental in overcoming the inherent limitations of imaging sensors and improving the detail and quality of digital images. Super-resolution has broad applications, ranging from satellite imaging and surveillance to medical imaging and consumer photography [46] .Guidance target. Specifically, we simply down-sample to original 256 × 256 images to the resolution of 64 × 64. A random noise with a variance of σ 2 = 0.05 2 is also added to the noisy images. If we denote the above process as a degradation operator A down : x → y, where x ∈ R 256×256×3 , y ∈ R 256×256×3 are original images and down-sampled images, then the target of superis to generate a high resolution image x 0 such that:max x0 p(x 0 ) = max x0 exp(-∥A down (x 0 ) -y∥ 2 ),which means if we project the generated image into the downsampled image space, the downsampled samples A down (x) should be similar to the ground truth downsampled images y.Evaluation metrics. Similar to Gaussian Deblur, we evaluate each guidance method on a set of 256 samples generated by Cat-DDPM. We apply FID [26] to assess the fidelity, Learned Perceptual Image Patch Similarity (LPIPS) [80] to evaluate the guidance validity. Label guidance is a standard task for conditional generation studied in previous literature [7, 23] . The target is to generate images conditioned on a certain label. We found this standard task is rarely studied in training-free guidance work and there exist an evident performance gap between training-based guidance and existing training-free guidance as shown in Section 3.Label sets. In our experiments, we studied labels from CIFAR10 [30] and ImageNet [55] . We average the results on 10 labels from CIFAR10 if there is no extra explanation. For ImageNet, which is resource-hungry to do comprehensive inference, we randomly select 4 labels (111, 222, 333, 444) to evaluate the methods. The corresponding label-ID and their names are as follows: p(x 0 ) = max x0 softmax(h(x 0 )) i ,where i is the label-ID of the target label, and h(•) is the logits of x 0 produced by the pre-trained classifier. For CIFAR10 and ImageNet, we use a pre-trained classifier based on ResNet [17] and VIT [9] that are provided from [75] and [9] respectively. The image resolution for CIFAR10 and ImageNet are 32 × 32 and 256 × 256 respectively.Evaluation metrics. We follow the image generation literature to use Fréchet inception distance (FID) [20] to assess the fidelity of generated images. The reference images are filtered from the entire dataset of CIFAR10 or ImageNet with the target label, and we set sample sizes as 2560 and 256 for CIFAR and ImageNet respectively. For validity, we use another pre-trained classifier to compute accuracy other than the one used in providing guidance to avoid over-confidence:accuracy = #classified as target label #generated samplesFor CIFAR10, we use a pre-trained ConvNeXT [36] downloaded from HuggingFace Hub 7 . And for ImageNet, we use the pre-trained DeiT [69] downloaded from HuggingFace Hub8 . An interesting scenario for conditional generation is to assign multiple targets for a single sample. Conditional generation with multiple conditions is crucial in machine learning for enhancing the relevance and applicability of AI across complex, real-world scenarios. It enables models to produce more contextually appropriate and personalized outputs, crucial for fields requiring high customization.Motivations. Combined guidance is to use multiple target functions to guide the same sample towards multiple targets for the same sample. It is more efficient for training-free guidance to do combined guidance as the space of combinatorial targets is potentially huge, which makes it unrealistic to train all target combinations for training-based guidance. We also find it related to the topic of spurious correlations [60] , where certain combinations of attributes may dominant the other combinations. For example, hair color may have a strong correlation with gender in CelebA dataset [37] . It is beneficial to explore training-free guidance on reducing the bias of generation models trained on these biased data and address the concerns related to fairness and equality.Guidance target. We study combined guidance on CelebA-DDPM, which is trained on CelebA-HQ [26] dataset. The image resolution is 256 × 256. We choose two settings of combined guidance with two attributes: (gender, hair color) and (gender, age). Each attribute has two labels, where gender∈ {male, female}, age∈ {young, old}, and hair color∈ {black, blonde}. We have a binary classifier for each attribute that is downloaded from HuggingFace Hub 91011 . The target is to sample images that maximize the marginal probability:max x0 p combined (x 0 ) = max x0 p target1 (x 0 )p target2 (x 0 ),where p target (x 0 ) is computed using label guidance as shown in Appendix D.3.Evaluation metrics. As it is hard to filter many reference images for combined targets in CelebA-HQ dataset, we adopt Kernel Inception distance (KID) [5] to assess fidelity of generated samples using 1,000 random sampled images of CelebA-HQ as reference images. We generate 256 samples for each evaluated method. We follow MPGD [18] to use the logarithm of KID, i.e. KID (log). For validity, we use another three attribute classifiers to compute the accuracy considering the conjunction of attributes:accuracy = # ∧ target label (classified as target label) #generated samplesThe evaluation classifiers are also downloaded from HuggingFace Hub 121314 .Figure 12 : Quantitative comparison of different training-free guidance methods on combined guidance task (male+young). Our TFG method can produce images with high fidelity and validity compared to all the baselines. Notice that we use the fixed seed for all the methods in this figure and do not conduct cherry picking. N recur is set to 1 for all methods.Fine-grained classification is a specialized task in computer vision that focuses on distinguishing between highly similar subcategories within a larger, general category. This task is particularly challenging due to the subtle differences among the objects or entities being classified. For example, in the context of animal species recognition, fine-grained classification would involve not just distinguishing a bird from a cat, but identifying the specific species of birds, such as differentiating between a crow and a raven [79] .Studying fine-grained generation in the context of generative models like Stable Diffusion or DALL-E presents unique challenges due to the inherent complexity of generating highly detailed and specific images. Fine-grained generation involves creating images that not only belong to a broad category but also capture the subtle nuances and specific characteristics of a narrowly defined subcategory. For example, generating images of specific dog breeds in distinct poses or environments requires the model to understand and replicate minute details that distinguish one breed from another.Motivations. To develop personalized AI, it is important to explore if the foundational generative models can synthesize fine-grained, accurate target samples according to user-defined target. However, this usually requires high-quality and detailed training data, and the model should be highly sensitive to small variations in input to accurately produce the desired output, which can be difficult for strong text2image generation models DALL-E15 or Imagen 16 . We first study this problem in a training-free guidance scenario.Guidance target. We study the out-of-distribution fine-grained label guidance on ImageNet-DDPM, which learns the generation of some species of birds but not comprehensively. We use an EfficientNet trained to classify 525 fine-grained bird species downloaded from HuggingFace Hub. 17 The classifier is trained on Bird Species dataset on Kaggle 18 . We follow the same way in label-guidance to maximize softmax probability for target fine-grianed label. We randomly sample 4 labels (111, 222, 333, 444) in Bird Species dataset, which are: Evaluation metrics. Similar to label guidance, we use FID to evaluate generation fidelity by filtering data of the target label as reference images. We also compute the FID with 256 sampled images. For accuracy, we adopt another downloaded pre-trained classifier trained on Bird Species dataset from HuggingFace Hub. 19 .Figure 13 : The sampled 256 images for fine-grained guidance with target label 222 (fairy tern) by ImageNet-DDPM with TFG. A key feature of fairy tern is its black-colored head. We observe that TFG generally samples images with black, round shapes and successfully generates some birds with target feature (red circled).Style transfer is a significant task in computer vision (CV) that focuses on applying the stylistic elements of one image onto another while preserving the content of the target image. This task is pivotal because it bridges the gap between artistic expression and technological innovation, allowing for the creation of novel and aesthetically pleasing visual content. Applications of style transfer are vast, ranging from enhancing user engagement in digital media and advertising to aiding in architectural design by visualizing changes in real-time.Guidance target. The target in our experiments is to guide a text-to-image latent diffusion model Stable-Diffusion-v-1-5 [53] 20 to generate images that fit both the text input prompts and the style of the reference images. The guidance objective involves calculating the Gram matrices [25] of the intermediate layers of the CLIP image encoder for both the generated images and the reference style images. The Frobenius norm of these matrices serves as the metric for the objective function. Specifically, for a reference style image x ref and a decoded image D(z 0|t ) generated from the estimated clean latent variable z 0|t , we compute the Gram matrices G(x ref ) and G(D(z 0|t )). These matrices are derived from the features of the 3rd layer of the image encoder, in accordance with the methodologies described in MPGD [18] and FreeDoM [78] . The target function is computed as follows:max x0 p style (x 0 ) = max x0 exp(-∥G(x ref ) -G(D(z 0|t ))∥ 2 F ),where ∥ • ∥ 2 F denotes the Frobenius norm of a matrix.Evaluation metrics. We use Style score and CLIP score to assess the guidance validity and fidelity, respectively. For reference style images and text prompts, we select 4 images from WikiArt [56] that are also used by MPGD [18] , and 64 text prompts from Partiprompts [77] . For each style, we generate 64 images based on the 64 different text prompts. To avoid over-confidence of CLIP score, we use two different CLIP models downloaded from HuggingFace Hub to compute guidance and evaluation metrics, respectively. 2122 . The style images and examplar prompts are shown as follows. Setup. Our benchmark setup generally follows [24, 3] but with certain specifications to guarantee the overall framework abides by the training-free regime. For dataset, we employ QM9 [50] and adopt the split in [24] with 100,000 training samples. Following [24] and [3] , the training set is further split into two halves that guarantees there is no data leakage. The first half is leveraged to train a property prediction network with EGNN [57] as the backbone, which serves as the ground truth oracle to provide the label used for MAE computation. We reuse the checkpoints released by [3] for the labeling network for all 6 properties. The second half is used to train the diffusion model as well as the guidance network. We adopt the unconditional generation version of EDM [24] as the diffusion model. The guidance network in general takes the same architecture as defined by [3] that, again, features EGNN as the backbone but outputs a single scalar as the predicted quantum mechanics property. The only difference lies in at training time we mask the diffusion time step by zeros and always use the original clean molecule structure as input, ensuring training-free objective.All the pretrained models are trained separately for different properties. At sampling time, we employ DDIM [62] sampler with 100 sampling steps, as opposed to [24, 3] that take 1000 sampling steps.Guidance target. We study training-free guided generation of molecules on 6 quantum mechanics properties, including polarizability (α), dipole moment (µ), heat capacity (C v ), highest orbital energy (ϵ HOMO ), lowest orbital energy (ϵ LUMO ) and their gap (∆ϵ). Denote the oracle property prediction network as E. Our guidance target in this case is given byEQUATIONwhere x is the input molecule and c is the target property value.Evaluation metrics. We use Mean Absolute Error (MAE) and validity as our evaluation metrics.In particular, MAE is computed between the target value and the predicted value gathered from the labeling network. Validity is computed by RDKit which measures whether the molecule is chemically feasible. We generate 4096 molecules for each property for evaluation. Our TFG generates valid molecules with better design target, while baselines often fail to produce valid molecules or offer poor guidance towards the design target. The molecules generated by our approach are increasingly polarizable as α goes up.Audio declipping is a task in digital audio restoration where distorted audio signals are repaired. Clipping occurs when the amplitude or frequency of an audio signal exceeds the maximum limit that the recording system can handle, leading to a harsh, distorted sound with portions of the waveform "cut off." Declipping aims to reconstruct the missing parts of these clipped waveforms, restoring the audio's original dynamics and reducing distortion. This process improves the quality and clarity of the audio, making it more pleasant to listen to while preserving the original sound's integrity.Guidance target. Specifically, we apply a distortion operation to zero out the high frequency and low frequency (for the highest and lowest 40 dimensions) signal in the space of mel spectrograms. If we denote the above process as a blurring operator A blur : x → y, where x ∈ R 256×256 , y ∈ R 256×256 are mel spectrograms and noisy mel spectrograms for 5s audios, then the target of Audio Declipping is to generate a clean audio x 0 such that:max x0 p(x 0 ) = max x0 exp(-∥A blur (x 0 ) -y∥ 2 ,which means if we project the generate mel spectrogram into the noisy space, the noisy samples A blur (x) should be similar to the ground truth noisy mel spectrogram y.Evaluation metrics. In our experiments, we evaluate each guidance method on a set of 256 samples generated by Audio-diffusion. We apply the Dynamic time warping (DTW) [44] to assess the guidance validity, and Fréchet Audio Distance (FAD) [28] to assess the generation fidelity.Audio inpainting is a digital audio restoration task that involves filling in missing or corrupted segments of an audio signal. Similar to image inpainting in the visual domain, this technique reconstructs the missing portions of sound by analyzing the surrounding context and seamlessly restoring the lost information. Applications of audio inpainting range from repairing damaged recordings to reconstructing gaps in audio streams due to data loss. The goal is to produce a natural-sounding result that preserves the continuity and overall quality of the original audio.Guidance target. Specifically, we apply a distortion operation to zero out the middle 80 dimensions in the space of mel spectrograms. If we denote the above process as a blurring operator A blur : x → y, where x ∈ R 256×256 , y ∈ R 256×256 are mel spectrograms and noisy mel spectrograms for 5s audios, then the target of Audio Declipping is to generate a clean audio x 0 such that:max x0 p(x 0 ) = max x0 exp(-∥A blur (x 0 ) -y∥ 2 ,which means if we project the generate mel spectrogram into the noisy space, the noisy samples A blur (x) should be similar to the ground truth noisy mel spectrogram y.Evaluation metrics. In our experiments, we evaluate each guidance method on a set of 256 samples generated by Audio-diffusion. We apply the Dynamic time warping (DTW) [44] to assess the guidance validity, and Fréchet Audio Distance (FAD) [28] to assess the generation fidelity.In Table 1 , we study the effect of Monte-Carlo sample sizes in estimating the expectation of Line 4 in TFG algorithm. As the noise is added on both Mean Guidance (∆ 0 ) and Variance Guidance (∆ t ), we decouple the effect into adding noise solely on ∆ 0 (Mean only) or ∆ t (Variance only). In the setting of Variance only, we set µ = 0, N iter = 0, N recur = 4, s ρ (t) = "increase", and pick the best ρ and γ via hyper-parameter search. In the setting of Mean only, we set ρ = 0, N iter = 4, N recur = 1, s µ (t) = "increase", and pick the best μ and γ via hyper-parameter search. We found that the sample size used in Monte-Carlo method play a neglect-able role on the performance if we set the optimal hyper-parameter. It is also noteworthy that the Monte-Carlo sampling does affect the performance of generated quality. For example, we can find that different targets shown in Appendix E.3 have different searched γ. This indicates that the best γ for many targets are apparently not zero.We compare the performance of our beam search parameters with the full grid search ones on CIFAR-10 label guidance task (Table 10 ). Overall, the performance of both search methods is identical, while grid search is much slower than our search strategy, indicating that our beam search strategy is effective and efficient.Table 10 : The searched (ρ, μ, γ) of exhaustive grid search and our beam search strategy on the CIFAR-10 label guidance task. We show the validity metric of the corresponding results and the gap ∆ = ∥validity beamvalidity grid ∥. Overall, the performance of both methods is identical. In this section, we present the hyper-parameters searched via the strategy introduced in Section 4 and the corresponding experimental results for TFG as shown in Table 11 . We list several observations below.• Overall, optimal parameters vary widely between problems and datasets. For example, even with the same model and objective (e.g., label classifier on ImageNet or CIFAR10), the best hyperparameters vary widely from target to target. This highlights the importance of hyperparameter search. • The improvement of TFG over existing methods depends heavily on the difference between the optimal parameters and the subspaces of existing methods. For example, the ρ for UGD is the same as TFG for gender-age guidance task, where TFG only has 0.133% validity improvement over UGD. On the contrary, their values differ on the fine-grained classification task, and TFG has an 18.7% validity improvement over UGD. Overall, we suppose this depends on whether the optimal parameter lies in the subspace that existing methods can find. • Though the baselines mentioned in our paper should be a special case of TFG, the results for the highest MO energy guidance in Table 3 show that MPGD outperforms TFG slightly. We want to point out that the reason TFG could occasionally have slightly worse performance in practice is due to the beam search computation limit we currently pose. More specifically, we allow TFG to search at most six steps (for all hyper-parameters) and all other methods for seven steps (in their subspaces). For the MO energy task, the searched parameter for MPGD is that μ = 0.016 (this is the only parameter that we need to search for MPGD), where the best (and last step) of TFG is that μ = 0.004 (because it uses one step to double another parameter). If we allocate more computational budget for the beam search steps, TFG will outperform MPGD on this target (in fact, eight steps suffice).Table 11 : The parameter (ρ, μ, γ) selected by beam search strategy for all methods, tasks, and targets. The search space of each method can be found in Section 3.1. For the detailed semantics of each task, please refer to Appendix D. Molecule Property α 0.005 0 0 0.005 0 1 0 0.01 0 0.01 0 0 0.02 0.02 0 0.016 0.001 0.0001 µ 0.02 0 0 0.01 0 1 0 0.005 0 0.02 0 0 0.005 0.005 0 0.001 0.002 0.1 C v 0.005 0 0 0.005 0 1 0 0.005 0 0.005 0 0 0.005 0.005 0 0.004 0.001 0.001 ϵ HOMO 0.005 0 0 0.005 0 1 0 0.01 0 0.005 0 0 0.005 0.005 0 0.002 0.004 0.001 ϵ LUMO 0.005 0 0 0.01 0 1 0 0.01 0 0.005 0 0 0.005 0.005 0 0.016 0.002 0.0001 ∆ 0.005 0 0 0.01 0 1 0 0.01 0 0.01 0 0 0.005 0.005 0 0.032 0.001 0.001 In the codebase of FreeDoM 23 , the schedule of guidance strength for different applications is different. For example, the guidance strength has a schedule coefficient √ ᾱt for face generation, and the schedule for style transfer is complex and involves a correction term, the mean of gradients' norm, and a specific constant coefficient 0.2. The paper does not mention this particular schedule, leaving the rationale for choosing these schedules unclear. We choose not to include the tricks and find that with our unified hyper-parameter searching strategy, the performance of FreeDoM is similar.We run most of the experiments on clusters using NVIDIA A100s. We implemented our experiments using PyTorch [49] and the HuggingFace library. 24 Overall, we estimated that a total of 2,000 GPU hours were consumed. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes]Justification: See Sec. 5.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: See Sec. 6.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: It is not applicable to the concern of this paper.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes]Justification: We have cited the related papers.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.MPGD additionally proposed to use an auto-encoder to improve the quality of x 0|t . However, an autoencoder is usually inaccessible or requires training; thus, we don't apply it in our training-free scenario.We notice that the FID worsens when Nrecur or Niter are too large (e.g., 10).https://huggingface.co/teticio/audio-diffusion-256https://huggingface.co/ahsanjavid/convnext-tiny-finetuned-cifar10https://huggingface.co/facebook/deit-small-patch16-224https://openai.com/index/dall-e-2/https://deepmind.google/technologies/imagen-2/https://huggingface.co/chriamue/bird-species-classifierhttps://www.kaggle.com/datasets/gpiosenka/100-bird-species/datahttps://huggingface.co/dennisjooo/Birds-Classifier-EfficientNetB2https://huggingface.co/runwayml/stable-diffusion-v1-5Guidance: https://huggingface.co/openai/clip-vit-base-patch16Evaluation: https://huggingface.co/openai/clip-vit-base-patch32https://github.com/vvictoryuki/FreeDoMhttps://huggingface.co/ 
 
 Appendix:  


Decision: This paper addresses the challenge of generating samples with desired target properties using an unconditional diffusion model and a target property predictor without requiring additional training. It critiques existing methods for lacking theoretical grounding and rigorous benchmarking, which can lead to inconsistent performance across tasks. The authors introduce a novel algorithmic framework that unifies existing methods under a broader, algorithm-agnostic design space. They also propose an efficient hyper-parameter search strategy and demonstrate its effectiveness through extensive benchmarking, showing a 7.4% average performance improvement across various models and tasks. The reviewers have been generally positive about the contribution of this work, and believe that the proposed benchmarking will be interesting to future works in the related areas.

Reviews: review: paper_summary: paper_summary: This paper focuses on the unification of training-free guidance methods for diffusion models. It defines each method within a unified framework and finds that restricting the hyperparameter space is consistent with existing methods. This framework can be broadly categorized into mean guidance, variance guidance and implicit dynamics. The paper demonstrates performance improvements in several experimental scenarios.
strength_weakness: strength_weakness: * Algorithm 1 seems to be one of the most important parts of this paper, but it lacks a detailed explanation. An explanation of Algorithm 1 along with the corresponding hyperparameters from Definition 3.1 in Section 3.1 would greatly aid understanding.

* In Algorithm 1, the iteration part in line 8 could be explicitly described using a for loop in the algorithm, rather than as a comment.

* I believe that Theorem 3.2 and its proof are not well formulated mathematically. If the authors want to formalize it as a theorem, they need to show mathematically that the same generated distribution can be produced. In my opinion, an explanation at an analytical level would be acceptable.
questions: questions: Please see the Weaknesses part.
limitations: limitations: They provided in the last section.
paper_summary: paper_summary: This paper scrutinizes existing works on training-free guidance in diffusion models and proposes a unified framework that includes all existing methods as special cases. With this unified framework, this work presents a detailed and informed investigation of the design choices and hyperparameters within this framework. Additionally, it proposes a comprehensive benchmark involving 14 task types and 38 targets to evaluate the performance of this unified framework by optimizing the hyperparameters in the design choices.
strength_weakness: strength_weakness: 1. While I appreciate the comprehensiveness of the experiments on the proposed benchmark, I think the author fails to provide an informative analysis of the results. Currently, the results simply show that by optimizing the hyperparameters in the unified framework, we obtain better performance, which is a natural outcome since the existing methods are included in the framework. From a research perspective, I can think of several questions worth investigating, such as whether the optimal hyperparameters vary between tasks and, when using fixed models, how the optimal hyperparameters are affected by the target objective function. With this investigation, it would be best to reach some general conclusions to guide users in tuning the hyperparameters in practice, rather than just relying on grid search.

2. It seems that for around half of the tasks, the improvement is only marginal, while for a few other settings, the improvement is significant. I think this phenomenon is worth further investigation to understand in what scenarios TFG will provide improvements.
questions: questions: 1. Why does the highest MO energy in Table 3 show a negative improvement? MPGD should be a special case of TFG. Is this because the grid-searched hyperparameters for TFG are not optimal?

2. Can the author comment on other lines of work [1,2,3,4], specifically direct optimization approaches, on training-free optimization of diffusion models with a target objective? The task setting is the same as training-free guidance. Although it is evident that the direct optimization approaches are slower than training-free guidance, it is not clear how their performance compares.

[1] Bram Wallace, Akash Gokul, Stefano Ermon, and Nikhil Naik. End-to-end diffusion latent optimization improves classifier guidance. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7280–7290, 2023b.

[2] Heli Ben-Hamu, Omri Puny, Itai Gat, Brian Karrer, Uriel Singer, and Yaron Lipman. D-flow: Differentiating through flows for controlled generation. arXiv preprint arXiv:2402.14017, 2024.

[3] Korrawe Karunratanakul, Konpat Preechakul, Emre Aksan, Thabo Beeler, Supasorn Suwajanakorn, and Siyu Tang. Optimizing diffusion noise can serve as universal motion priors. arXiv preprint arXiv:2312.11994, 2023.

[4] Tang, Zhiwei, et al. "Tuning-Free Alignment of Diffusion Models with Direct Noise Optimization." arXiv preprint arXiv:2405.18881 (2024).
limitations: limitations: See my comments on the weaknesses above. I think there are a few unclear points worth discussing.
paper_summary: paper_summary: The authors propose a framework (TFG) for training-free guidance of unconditional diffusion models, enabling their application to conditional generation tasks such as super-resolution, deblurring, etc. via the use of a predictor that evaluates the quality of a clean sample.  TFG, as in related past methods, aim to sample from the conditional distribution of samples defined in Equation 4 without either training a conditional diffusion model or a predictor evaluating the quality of noisy samples as in classifier guidance.  The gradient of the predictor on noisy samples combined with the unconditional score produce the correct conditional score function in Equation 5.  The challenge of training-free methods is to approximate this gradient somehow and a variety of approaches have been previously proposed.  TFG defined in Algorithm 1 is shown to include five such previous training-free methods as special cases for particular hyperparameters, demonstrating that these methods can be viewed and understood in a unified framework.  The authors then propose a procedure to optimize TFG's hyperparameters jointly.  By leveraging the larger design space with optimized hyperparameters, performance gains are observed across 14 tasks, 6 diffusion models, and 38 target predictors compared to the past methods subsumed by TFG.
strength_weakness: strength_weakness: - Some statements and claims are overly broad.  The authors claim all existing training-free approaches fit into their framework which is unlikely.  Many training-free methods have been proposed and the authors could better place their framework in the context of recent literature.  Section 3 Figure 1's demonstration that training-based conditional generation outperforms training-free is unsurprising, given that training-free methods are approximations.
- While hyperparameter search was helpful, the unified design space discussion in Section 4.1 led to limited theoretical insight.  The authors could consider expanding this section to improve their methodological contributions.
- The hyperparameter setting strategy described in Section 4.2, key to getting gains versus past methods, could use more detailed explanation (see questions 2-4 below). 
- The benchmark Section 5 decides to not compare generation fidelity in favor of comparing the best algorithm in terms of generation validity.  However, throughout the rest of the paper (e.g. figure 1, 2, and 3) tradeoffs between fidelity and validity are emphasized.  Looking at Table 2, TFG is sometimes but not always the best algorithm in terms of both fidelity and validity simultaneously.  The focus on only validity seems unjustified here.
questions: questions: 1. Why does increasing $N_{recur}$ and $N_{tier}$ eventually hurt performance?
2. In section 4.2, does the "structure analysis" that "increase" is best hold more generally on other tasks than label guidance?   
3. The number of Monte Carlo samples in the Implicit Dynamic is set to 1 based on Table 1, but that choice needs more justification.  Does more samples than 4 help?  Why is implicit dynamic helpful with few samples?  This seems contrary to the original motivation of estimating an average.
4. In section 4.2, the beam search in "Searching strategy" could use more detail.  What sample sizes are used and how did that enable quick search? Also, what metric is used when deciding the top K candidates in this search?
limitations: limitations: Limitations are briefly discussed in Section 6, focusing on why training-free guidance remains relevant given language-based image generators.
paper_summary: paper_summary: The paper introduces Training-Free Guidance (TFG), a novel framework designed to enhance the generation of samples with desired properties using diffusion models, without necessitating additional model training. TFG aims to resolve the shortcomings of existing training-free methods by offering a unified algorithmic framework that simplifies the comparison and application of such methods across a wide range of tasks. By theoretically and empirically analyzing a hyper-parameter design space within this framework, the authors develop an effective strategy for hyper-parameter selection applicable to various tasks. Their comprehensive benchmarks across multiple tasks and targets demonstrate TFG's superior performance, achieving an average improvement of 7.4% over existing methods.
strength_weakness: strength_weakness: There are several areas where improvements could significantly enhance its contributions. Addressing these points satisfactorily would make a strong case for elevating the paper's status to "Accept" or "Strong Accept".

1. While the paper provides valuable insights into a specific line of training-free guidance, it is important to acknowledge the existence of a broader spectrum of works outside this category. This observation suggests that the title of the paper may slightly overreach, potentially implying a more comprehensive coverage than is actually presented. (Question 1)

2. The paper commendably supports the motivations behind conditional diffusion in Appendix A, offering a solid foundation for its relevance. However, the rationale for focusing specifically on training-free approaches appears less extensively articulated. (Question 2)

3. The authors claimed that they have theoretically grounded their unified framework. However, upon review, the theoretical underpinnings presented seem to require further elaboration to fully substantiate this claim. (Question 3, 4, 5)

4. The authors claimed that "the studies of training-free methods become the study within the hyperparameter space of our framework". Nonetheless, it appears that the search space defined within this framework might not fully encompass the range of hyper-parameters considered in existing literature. This limitation could potentially restrict the framework's applicability or comparative analysis capabilities. (Question 6, 7)
questions: questions: 1. The field of training-free guidance encompasses a wide array of studies beyond those specifically addressed in this paper (e.g. [1]-[3]). Could you elaborate on how these works relate to the scope and contributions of your paper?

2. Your paper operates under the assumption that the forward model is known, leveraging training-free guidance for solving inverse problems. Given that knowing the forward model allows for the generation of a large number of samples to train a conditional diffusion model at a relatively low computational cost (e.g., 10 A100 GPU hours) and training-based approaches are much better than training-free ones, could you discuss the significant motivations of adopting a training-free approach in this context?

3. The theoretical foundation of your paper seems to rest significantly on Lemma 4.1, which revisits the variance of MMSE estimator of the signal corrupted by Gaussian noise (e.g., (2.8) in [4]). This formula is widely adopted in diffusion papers (e.g., [5]). Given its established nature and previous applications, could you elaborate on how this lemma specifically contributes to the novel aspects of your framework?

4. Concerning Lemma 4.1, it appears there is no assurance that the generated image accurately follows the conditional distribution, nor is there a guarantee that the loss decreases at each iteration as suggested by equation (7). Could you provide further clarification or additional theoretical support to address these concerns?

5. Several techniques introduced in the paper lack direct theoretical underpinning:
- The concept of "time-travel" being an Ornstein-Uhlenbeck process is intriguing but lacks a detailed derivation. Could you expand on how the theory of the Ornstein-Uhlenbeck process quantifies the benefits of time-travel in your framework?
- The selection of hyperparameters seems not to be grounded in theory. Could you discuss the rationale behind these choices and any potential theoretical support?

6. The paper posits that the study of training-free methods can be encapsulated within the hyperparameter space of your framework. However, specific hyperparameter settings critical for the performance in existing works, such as the step sizes used in Face generation (at.sqrt()) and Style transfer ((correction * correction).mean().sqrt().item() * unconditional_guidance_scale / (norm_grad * norm_grad).mean().sqrt().item() * 0.2) in FreeDoM, are not explicitly covered. Could you address the omission of these settings and their impact on the comprehensiveness of your study?

7. The hyperparameter search settings for the baselines in your comparative analysis are not disclosed, raising questions about the fairness and validity of the comparisons. Could you provide more details on these settings to ensure a transparent and equitable comparison?

[1] Feng, Weixi, et al. "Training-free structured diffusion guidance for compositional text-to-image synthesis." ICLR 2023.

[2] Chen, Minghao, et al. "Training-free layout control with cross-attention guidance." WACV 2024.

[3] Mo, Sicheng, et al. "Freecontrol: Training-free spatial control of any text-to-image diffusion model with any condition." CVPR 2024.

[4] Efron, Bradley. "Tweedie’s formula and selection bias." JASA 2011.

[5] Kadkhodaie, Zahra, et al. "Generalization in diffusion models arises from geometry-adaptive harmonic representation." ICLR 2024.
limitations: limitations: Yes

Comments: title: A gentle reminder
A gentle reminder
Thanks for the review
Thanks for your feedback!
Response to the followup questions (cont')
Response to the followup questions
Additional responses
Additional references
comment: I appreciate the author's response. I expect my concerns are well reflected in the revised version, and I think the current score is appropriate, so I keep it.
Dear Reviewer J9FB,

The deadline of the discussion period is soon approaching. We wonder whether our answers to your questions have addressed your concerns. If there are any additional discussion points or questions, we are happy to discuss.

We look forward to your comments. Thank you again for your time.

Best,

Authors of Paper8134
Dear Reviewer DCpp,

The deadline of the discussion period is soon approaching. We wonder whether our answers to your questions have addressed your concerns. If there are any additional discussion points or questions, we are happy to discuss.

We look forward to your comments. Thank you again for your time.

Best,

Authors of Paper8134
We sincerely appreciate your timeply reply and considerate response about the short experiments window. We promise to add them in the revised paper. Thanks!
Thank you for your detailed response. I have adjusted my score to an 8. In particular, **I believe the reproducible benchmark used in this paper will influence the trajectory of future research in training-free guidance**. To ensure a comprehensive comparison, I strongly recommend that you include motion diffusion guidance from the LGD paper (a new application related to diffusion planning) and phase retrieval from the DPS paper (a nonlinear non-NN case) as additional benchmarks in the final manuscript. I have not listed this as a weakness, as I understand that these experiments may not be feasible within the rebuttal period.
We thank the reviewer for the timely response. We highly appreciate it the reviewer could increase the evaluation score accordingly if you believe that the revised paper is improved.
> In relation to Question 5, I would be interested in a more detailed discussion on the practical implications of the connections between the Ornstein-Uhlenbeck process and time travel. Specifically, how do these connections yield new guarantees or insights that could potentially enhance the methodology of time travel?
> 

We are willing to give a fine-grained theoretical discussion about the insights behind recurrence. Specifically, let’s define a parameterized step $u_t (x)$ that tries to simulate the transformation from distribution $p_{t+1}$ to $p_t$. Then, assume that up to time step $t+1$, the error of distribution estimation is $err_{t+1}$, e.g., if we use the total variation between the resulting distribution $p^\theta_{t+1}$ and $p_{t+1}$, then $err_{t+1} = TV(p_{t+1}^\theta, p_{t+1})$.

In such case, consider the following OU process:

$$
x_t = u_t(x_{t+1}), 
$$
$$
x_{t+1} = x_t + \epsilon,
$$

where the recurrent step is $K$, then informally, the Wasserstein-1 distance between resulting distribution $p^\theta_t$ and the ground truth distribution $p_t$ can be controlled by the following terms:

$$
c_0 (1-\lambda)^K \text{err}_{t+1} + (K+1)c_1,
$$

where $c_0, c_1$, and $\lambda \in (0,1)$ are some constants that depend on many variables, including score estimation errors, time step $t$, configurations of Gaussian noise $\epsilon$, and more. Intuitively, the first term implies that the contracted error will shrink as we traverse, thanks to the convergent property of the OU process; the second term is an accumulated error due to inaccurate estimation of the transformation from $t+1$ to $t$. The gradient of the upper bound is 

$$
c_0 \text{err}_{t+1} (1-\lambda)^K \ln (1-\lambda) + c_1,
$$

which, under some assumptions, is negative when $K$ is small and possible after $K$ increases over some certain point. This could intuitively explain why the recurrence is helpful when we increase $K$ from 1 to some certain values, but will lead to under-qualified samples when $K$ becomes overly large: it’s because that recurrence helps find a balance between the contracted error inherited from previous steps and the accumulated error in this step.

We are happy to add more discussion to the paper, but we want to emphasize a few issues why we cannot make it a formal theorem. First, notice that we can only control the Wasserstein-1 distance at step $t$ using the total variation distance at step $t+1$, which is not transmissible since W1 distance cannot bound TV distance. Second, even if we demonstrate that the upper bound has a decrease-then-increase pattern, it’s not guaranteed that the actual error follows a similar pattern in theory (although it is in practice). The concrete techniques and discussions about the limitations can be found in [D]. We hope that this can help provide insights into the issue you are concerned about.

We thank you for your time reviewing our paper and for providing supportive comments and constructive feedback. Please let us know if any further clarifications are needed.

[C] Song, Jiaming, et al. “Loss-Guided Diffusion Models for Plug-and-Play Controllable Generation”. *Proceedings of the 40th International Conference on Machine Learning*, PMLR 202:32483-32498, 2023.

[D] Xu, Yilun, et al. "Restart sampling for improving generative processes." Advances in Neural Information Processing Systems 36 (2023): 76806-76838.
We thank the reviewer for the moderately improved evaluation and we are delighted to address the follow-up concerns.

> Regarding Question 2, I have experimented with several methods to fine-tune a network trained on clean images for use with noisy images [A, B]. My findings indicate that the training can be completed in one A100 hour and the performance is quite satisfactory. Could you please provide further clarification on this point?
> 

We have gone through both papers you provided, and we believe that there are major difference between our methods and theirs. Overall, the training efficiency of their method (even if we don’t think about the advantage of “training-free” at all) is highly determined by the structure of the guidance function, and the situation of using CLIP for guidance might be completely different from that of using a general classifier. 

For example, notice that in [A] the guidance function $F(x_t, t,l)$ is expected to have a special structure, i.e., $F(x_t, t, l) = E(x_t,t) \cdot E(l)$, where $l$ is the language embeddings and $x_t$ is the noisy images. In such case, using undesired images generated from unconditional diffusion models to help learn $E(x_t, t)$ from CLIP model $E(x_0)$ is likely **generalizable** to desired images since the relationship between images and the requirements specified by $l$ is simply multiplicative. However, consider a more general case where the given classifier $f(x_0)$ corresponds to a very special property and only a tiny proportion of images gives $f(x_0) \approx 1$ (e.g., whether the generated image is Larosterna inca, a bird species native to the coastal regions of western South America). Most unconditional images will not even have activated embeddings of the last layer of $f$ since they have a close-to-zero classifier output. In such case, the property of being able to match the embeddings of $f(x_t,t)$ and $f(x_0)$ is unlikely generalizable to desired images with $f\approx 1$, since both noisy and clean undesired images could have close-to-zero embeddings but both noisy and clean desired images do not. Consequently, an extremely large amount of unconditional generation is required to help learn a time-dependent classifier of one particular property. We are willing to see if future works can give a more quantitative result about the training efficiency for general classifiers.

> For Question 3, you assert that Theorem 3.2 represents the main theoretical contribution. However, it appears to be straightforward as the proposed method is a combination of these methods. Could you explain why the proof is non-trivial?
> 

We would like to clarify that the major contribution of the unification algorithm does not lie in the novelty or difficulty of the proof of theorem 3.2, but rather in recognizing and quantifying the importance of unifying different methods in the same hyper-parameter space for training-free guidance. For instance, while different existing methods overlap in certain techniques, their intuition and explanation of each of the technique is different, some of which are even incorrect (for example, [C] unconsciously falls into a “fake” training-free guidance setting, as we have discussed in Appendix A.2). By figuring out a way to construct an algorithm with reasonable hyper-parameter space that can encompass existing methods, we unify different techniques and offer a clean way to study the training-free guidance problem. Theorem 3.2 mainly aims to justify the unification theoretically, and the proof is not technically special.
I thank the authors for their rebuttal and believe the discussed changes may improve the paper upon revision.
I appreciate the authors' effort in addressing my concerns, which has led me to moderately increase the evaluation score. 

Regarding Question 2, I have experimented with several methods to fine-tune a network trained on clean images for use with noisy images [A, B]. My findings indicate that the training can be completed in one A100 hour and the performance is quite satisfactory. Could you please provide further clarification on this point?

For Question 3, you assert that Theorem 3.2 represents the main theoretical contribution. However, it appears to be straightforward as the proposed method is a combination of these methods. Could you explain why the proof is non-trivial?

In relation to Question 5, I would be interested in a more detailed discussion on the practical implications of the connections between the Ornstein-Uhlenbeck process and time travel. Specifically, how do these connections yield new guarantees or insights that could potentially enhance the methodology of time travel?

[A] More Control for Free! Image Synthesis with Semantic Diffusion Guidance

[B] Towards practical plug-and-play diffusion models
> The selection of hyperparameters seems not to be grounded in theory. Could you discuss the rationale behind these choices and any potential theoretical support?
> 

Our beam-search hyperparameter selection strategy depends on an intuition (assumption) that when fixing all other parameters, the loss function of the remaining parameter is a decrease-then-increase function, i.e., as the parameter increases from 0 to infinity, the loss first goes down (or never goes down) and then increases. This is observed and summarized from experiments, and under such an assumption, it can be proved that the (near-)optimal hyperparameter can be found with sufficient search step and small enough stepping size (multiplying factor). We would add this to our paper as suggested.  

4. The coverage of the proposed space in existing literature.

> The paper posits that the study of training-free methods can be encapsulated within the hyperparameter space of your framework. However, specific hyperparameter settings critical for the performance in existing works, such as the step sizes used in Face generation (at.sqrt()) and Style transfer ((correction * correction).mean().sqrt().item() * unconditional_guidance_scale / (norm_grad * norm_grad).mean().sqrt().item() * 0.2) in FreeDoM, are not explicitly covered. Could you address the omission of these settings and their impact on the comprehensiveness of your study?
> 

We are delighted to explain this further. Theoretically speaking, TFG aims to provide a general framework that unifies different sub-hyperparameter spaces, and the practical implementation tricks are omitted (as this trick only appears in FreeDoM code, not its original paper). That said, our framework can naturally represent this nuanced setting. For example, if we want the gradient to be $\frac{\nabla g}{\| \nabla g\|}$ for a given target function $g$, we could simply set the $f$ in our algorithm as $\exp\{\int \frac{\mathrm dg}{\| \nabla g\|}\}$, in which case the gradient of $f$ becomes exactly what we want.

In practice, however, we observe that with the aid of hyperparameter searching, the output quality of our FreeDoM algorithm is comparable to the original implementation. Because of this, and to be consistent with the paper, we simply focus on the setting without normalization. Overall, we believe that such normalization should be similar to normalizing $\mathbf \rho$.

> The hyperparameter search settings for the baselines in your comparative analysis are not disclosed, raising questions about the fairness and validity of the comparisons. Could you provide more details on these settings to ensure a transparent and equitable comparison?
> 

We apologize for not providing a more comprehensive discussion beyond what was presented in Appendix E. We'd like to offer additional details here. For all methods discussed in our paper, we employed an identical beam search strategy with three beam search trials and a maximum of 7 steps. For each search, we start with initial parameters (identical across all methods) and double parameters to see if performance improved. Then, we keep the top 3 performances in the beam list after each step until the list becomes unchanged or the maximum step is reached. This beam search program is method-agnostic and compatible with all methods, ensuring fair and objective comparisons. To promote transparency and facilitate future research, we will open-source our code, all beam search runs, the best parameters for each algorithm, and their corresponding performance results.

Again, we sincerely appreciate your helpful review. Please let us know whether there is any additional concern that we could address to improve your evaluation of our work.
[1] Controllable Music Production with Diffusion Models and Guidance Gradients. 

[2] A Framework for Conditional Diffusion Modelling with Applications in Protein Design and Inverse Problems.

[3] Solving Audio Inverse Problems with a Diffusion Model.

[4] Diffusion Models for Audio Restoration.

[5] Vrdmg: Vocal Restoration via Diffusion Posterior Sampling with Multiple Guidance.

[6] Motion Guidance: Diffusion-Based Image Editing with Differentiable Motion Estimators.

[7] Training-free Multi-objective Diffusion Model for 3D Molecule Generation.

[8] Control3diff: Learning Controllable 3D Diffusion Models from Single-view Images.

[9] Steered Diffusion: A Generalized Framework for Plug-and-Play Conditional Image Synthesis.

[10] Contrastive Energy Prediction for Exact Energy-Guided Diffusion Sampling in Offline Reinforcement Learning.