FreeScale: Unleashing the Resolution of Diffusion Models via Tuning-Free
  Scale Fusion
Authors: name: Haonan Qiu; name: Shiwei Zhang; name: Yujie Wei; name: Ruihang Chu; name: Hangjie Yuan; name: Xiang Wang; name: Yingya Zhang; name: Ziwei Liu
Published: 2024-12-12 18:59:59

Abstract: Visual diffusion models achieve remarkable progress, yet they are typically
trained at limited resolutions due to the lack of high-resolution data and
constrained computation resources, hampering their ability to generate
high-fidelity images or videos at higher resolutions. Recent efforts have
explored tuning-free strategies to exhibit the untapped potential
higher-resolution visual generation of pre-trained models. However, these
methods are still prone to producing low-quality visual content with repetitive
patterns. The key obstacle lies in the inevitable increase in high-frequency
information when the model generates visual content exceeding its training
resolution, leading to undesirable repetitive patterns deriving from the
accumulated errors. To tackle this challenge, we propose FreeScale, a
tuning-free inference paradigm to enable higher-resolution visual generation
via scale fusion. Specifically, FreeScale processes information from different
receptive scales and then fuses it by extracting desired frequency components.
Extensive experiments validate the superiority of our paradigm in extending the
capabilities of higher-resolution visual generation for both image and video
models. Notably, compared with the previous best-performing method, FreeScale
unlocks the generation of 8k-resolution images for the first time.