MegaFusion: Extend Diffusion Models towards Higher-resolution Image
  Generation without Further Tuning
Authors: name: Haoning Wu; name: Shaocheng Shen; name: Qiang Hu; name: Xiaoyun Zhang; name: Ya Zhang; name: Yanfeng Wang
Published: 2024-08-20 16:53:34

Abstract: Diffusion models have emerged as frontrunners in text-to-image generation for
their impressive capabilities. Nonetheless, their fixed image resolution during
training often leads to challenges in high-resolution image generation, such as
semantic inaccuracies and object replication. This paper introduces MegaFusion,
a novel approach that extends existing diffusion-based text-to-image generation
models towards efficient higher-resolution generation without additional
fine-tuning or extra adaptation. Specifically, we employ an innovative truncate
and relay strategy to bridge the denoising processes across different
resolutions, allowing for high-resolution image generation in a coarse-to-fine
manner. Moreover, by integrating dilated convolutions and noise re-scheduling,
we further adapt the model's priors for higher resolution. The versatility and
efficacy of MegaFusion make it universally applicable to both latent-space and
pixel-space diffusion models, along with other derivative models. Extensive
experiments confirm that MegaFusion significantly boosts the capability of
existing models to produce images of megapixels and various aspect ratios,
while only requiring about 40% of the original computational cost.