CogView3: Finer and Faster Text-to-Image Generation via Relay Diffusion
Authors: name: Wendi Zheng; name: Jiayan Teng; name: Zhuoyi Yang; name: Weihan Wang; name: Jidong Chen; name: Xiaotao Gu; name: Yuxiao Dong; name: Ming Ding; name: Jie Tang
Published: 2024-03-08 07:32:50

Abstract: Recent advancements in text-to-image generative systems have been largely
driven by diffusion models. However, single-stage text-to-image diffusion
models still face challenges, in terms of computational efficiency and the
refinement of image details. To tackle the issue, we propose CogView3, an
innovative cascaded framework that enhances the performance of text-to-image
diffusion. CogView3 is the first model implementing relay diffusion in the
realm of text-to-image generation, executing the task by first creating
low-resolution images and subsequently applying relay-based super-resolution.
This methodology not only results in competitive text-to-image outputs but also
greatly reduces both training and inference costs. Our experimental results
demonstrate that CogView3 outperforms SDXL, the current state-of-the-art
open-source text-to-image diffusion model, by 77.0\% in human evaluations, all
while requiring only about 1/2 of the inference time. The distilled variant of
CogView3 achieves comparable performance while only utilizing 1/10 of the
inference time by SDXL.