DiT-Air: Revisiting the Efficiency of Diffusion Model Architecture
  Design in Text to Image Generation
Authors: name: Chen Chen, name: Rui Qian, name: Wenze Hu, name: Tsu-Jui Fu, name: Lezhi Li, name: Bowen Zhang, name: Alex Schwing, name: Wei Liu, name: Yinfei Yang

In this work, we empirically study Diffusion Transformers (DiTs) for
text-to-image generation, focusing on architectural choices, text-conditioning
strategies, and training protocols. We evaluate a range of DiT-based
architectures--including PixArt-style and MMDiT variants--and compare them with
a standard DiT variant which directly processes concatenated text and noise
inputs. Surprisingly, our findings reveal that the performance of standard DiT
is comparable with those specialized models, while demonstrating superior
parameter-efficiency, especially when scaled up. Leveraging the layer-wise
parameter sharing strategy, we achieve a further reduction of 66% in model size
compared to an MMDiT architecture, with minimal performance impact. Building on
an in-depth analysis of critical components such as text encoders and
Variational Auto-Encoders (VAEs), we introduce DiT-Air and DiT-Air-Lite. With
supervised and reward fine-tuning, DiT-Air achieves state-of-the-art
performance on GenEval and T2I CompBench, while DiT-Air-Lite remains highly
competitive, surpassing most existing models despite its compact size.