Nested Diffusion Processes for Anytime Image Generation
Authors: name: Noam Elata; name: Bahjat Kawar; name: Tomer Michaeli; name: Michael Elad
Published: 2023-05-30 14:28:43

Abstract: Diffusion models are the current state-of-the-art in image generation,
synthesizing high-quality images by breaking down the generation process into
many fine-grained denoising steps. Despite their good performance, diffusion
models are computationally expensive, requiring many neural function
evaluations (NFEs). In this work, we propose an anytime diffusion-based method
that can generate viable images when stopped at arbitrary times before
completion. Using existing pretrained diffusion models, we show that the
generation scheme can be recomposed as two nested diffusion processes, enabling
fast iterative refinement of a generated image. We use this Nested Diffusion
approach to peek into the generation process and enable flexible scheduling
based on the instantaneous preference of the user. In experiments on ImageNet
and Stable Diffusion-based text-to-image generation, we show, both
qualitatively and quantitatively, that our method's intermediate generation
quality greatly exceeds that of the original diffusion model, while the final
slow generation result remains comparable.