SnapGen: Taming High-Resolution Text-to-Image Models for Mobile Devices
  with Efficient Architectures and Training
Authors: name: Dongting Hu; name: Jierun Chen; name: Xijie Huang; name: Huseyin Coskun; name: Arpit Sahni; name: Aarush Gupta; name: Anujraaj Goyal; name: Dishani Lahiri; name: Rajesh Singh; name: Yerlan Idelbayev; name: Junli Cao; name: Yanyu Li; name: Kwang-Ting Cheng; name: S. -H. Gary Chan; name: Mingming Gong; name: Sergey Tulyakov; name: Anil Kag; name: Yanwu Xu; name: Jian Ren
Published: 2024-12-12 18:59:53

Abstract: Existing text-to-image (T2I) diffusion models face several limitations,
including large model sizes, slow runtime, and low-quality generation on mobile
devices. This paper aims to address all of these challenges by developing an
extremely small and fast T2I model that generates high-resolution and
high-quality images on mobile platforms. We propose several techniques to
achieve this goal. First, we systematically examine the design choices of the
network architecture to reduce model parameters and latency, while ensuring
high-quality generation. Second, to further improve generation quality, we
employ cross-architecture knowledge distillation from a much larger model,
using a multi-level approach to guide the training of our model from scratch.
Third, we enable a few-step generation by integrating adversarial guidance with
knowledge distillation. For the first time, our model SnapGen, demonstrates the
generation of 1024x1024 px images on a mobile device around 1.4 seconds. On
ImageNet-1K, our model, with only 372M parameters, achieves an FID of 2.06 for
256x256 px generation. On T2I benchmarks (i.e., GenEval and DPG-Bench), our
model with merely 379M parameters, surpasses large-scale models with billions
of parameters at a significantly smaller size (e.g., 7x smaller than SDXL, 14x
smaller than IF-XL).