Scalable High-Resolution Pixel-Space Image Synthesis with Hourglass
  Diffusion Transformers
Authors: name: Katherine Crowson; name: Stefan Andreas Baumann; name: Alex Birch; name: Tanishq Mathew Abraham; name: Daniel Z. Kaplan; name: Enrico Shippole
Published: 2024-01-21 21:49:49

Abstract: We present the Hourglass Diffusion Transformer (HDiT), an image generative
model that exhibits linear scaling with pixel count, supporting training at
high-resolution (e.g. 1024 times 1024) directly in pixel-space. Building on
the Transformer architecture, which is known to scale to billions of
parameters, it bridges the gap between the efficiency of convolutional U-Nets
and the scalability of Transformers. HDiT trains successfully without typical
high-resolution training techniques such as multiscale architectures, latent
autoencoders or self-conditioning. We demonstrate that HDiT performs
competitively with existing models on ImageNet 256^2, and sets a new
state-of-the-art for diffusion models on FFHQ-1024^2.