FreSca: Unveiling the Scaling Space in Diffusion Models
Authors: name: Chao Huang, name: Susan Liang, name: Yunlong Tang, name: Li Ma, name: Yapeng Tian, name: Chenliang Xu

Diffusion models offer impressive controllability for image tasks, primarily
through noise predictions that encode task-specific information and
classifier-free guidance enabling adjustable scaling. This scaling mechanism
implicitly defines a ``scaling space'' whose potential for fine-grained
semantic manipulation remains underexplored. We investigate this space,
starting with inversion-based editing where the difference between
conditional/unconditional noise predictions carries key semantic information.
Our core contribution stems from a Fourier analysis of noise predictions,
revealing that its low- and high-frequency components evolve differently
throughout diffusion. Based on this insight, we introduce FreSca, a
straightforward method that applies guidance scaling independently to different
frequency bands in the Fourier domain. FreSca demonstrably enhances existing
image editing methods without retraining. Excitingly, its effectiveness extends
to image understanding tasks such as depth estimation, yielding quantitative
gains across multiple datasets.