Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with
  Fine-Grained Chinese Understanding
Authors: name: Zhimin Li; name: Jianwei Zhang; name: Qin Lin; name: Jiangfeng Xiong; name: Yanxin Long; name: Xinchi Deng; name: Yingfang Zhang; name: Xingchao Liu; name: Minbin Huang; name: Zedong Xiao; name: Dayou Chen; name: Jiajun He; name: Jiahao Li; name: Wenyue Li; name: Chen Zhang; name: Rongwei Quan; name: Jianxiang Lu; name: Jiabin Huang; name: Xiaoyan Yuan; name: Xiaoxiao Zheng; name: Yixuan Li; name: Jihong Zhang; name: Chao Zhang; name: Meng Chen; name: Jie Liu; name: Zheng Fang; name: Weiyan Wang; name: Jinbao Xue; name: Yangyu Tao; name: Jianchen Zhu; name: Kai Liu; name: Sihuan Lin; name: Yifu Sun; name: Yun Li; name: Dongdong Wang; name: Mingtao Chen; name: Zhichao Hu; name: Xiao Xiao; name: Yan Chen; name: Yuhong Liu; name: Wei Liu; name: Di Wang; name: Yong Yang; name: Jie Jiang; name: Qinglin Lu
Published: 2024-05-14 16:33:25

Abstract: We present Hunyuan-DiT, a text-to-image diffusion transformer with
fine-grained understanding of both English and Chinese. To construct
Hunyuan-DiT, we carefully design the transformer structure, text encoder, and
positional encoding. We also build from scratch a whole data pipeline to update
and evaluate data for iterative model optimization. For fine-grained language
understanding, we train a Multimodal Large Language Model to refine the
captions of the images. Finally, Hunyuan-DiT can perform multi-turn multimodal
dialogue with users, generating and refining images according to the context.
Through our holistic human evaluation protocol with more than 50 professional
human evaluators, Hunyuan-DiT sets a new state-of-the-art in Chinese-to-image
generation compared with other open-source models. Code and pretrained models
are publicly available at github.com/Tencent/HunyuanDiT