FouriScale: A Frequency Perspective on Training-Free High-Resolution
  Image Synthesis
Authors: name: Linjiang Huang; name: Rongyao Fang; name: Aiping Zhang; name: Guanglu Song; name: Si Liu; name: Yu Liu; name: Hongsheng Li
Published: 2024-03-19 17:59:33

Abstract: In this study, we delve into the generation of high-resolution images from
pre-trained diffusion models, addressing persistent challenges, such as
repetitive patterns and structural distortions, that emerge when models are
applied beyond their trained resolutions. To address this issue, we introduce
an innovative, training-free approach FouriScale from the perspective of
frequency domain analysis. We replace the original convolutional layers in
pre-trained diffusion models by incorporating a dilation technique along with a
low-pass operation, intending to achieve structural consistency and scale
consistency across resolutions, respectively. Further enhanced by a
padding-then-crop strategy, our method can flexibly handle text-to-image
generation of various aspect ratios. By using the FouriScale as guidance, our
method successfully balances the structural integrity and fidelity of generated
images, achieving an astonishing capacity of arbitrary-size, high-resolution,
and high-quality generation. With its simplicity and compatibility, our method
can provide valuable insights for future explorations into the synthesis of
ultra-high-resolution images. The code will be released at
https://github.com/LeonHLJ/FouriScale.