StyleAvatar3D: Leveraging Image-Text Diffusion Models for High-Fidelity
  3D Avatar Generation
Authors: name: Chi Zhang; name: Yiwen Chen; name: Yijun Fu; name: Zhenglin Zhou; name: Gang YU; name: Billzb Wang; name: Bin Fu; name: Tao Chen; name: Guosheng Lin; name: Chunhua Shen
Published: 2023-05-30 13:09:21

Abstract: The recent advancements in image-text diffusion models have stimulated
research interest in large-scale 3D generative models. Nevertheless, the
limited availability of diverse 3D resources presents significant challenges to
learning. In this paper, we present a novel method for generating high-quality,
stylized 3D avatars that utilizes pre-trained image-text diffusion models for
data generation and a Generative Adversarial Network (GAN)-based 3D generation
network for training. Our method leverages the comprehensive priors of
appearance and geometry offered by image-text diffusion models to generate
multi-view images of avatars in various styles. During data generation, we
employ poses extracted from existing 3D models to guide the generation of
multi-view images. To address the misalignment between poses and images in
data, we investigate view-specific prompts and develop a coarse-to-fine
discriminator for GAN training. We also delve into attribute-related prompts to
increase the diversity of the generated avatars. Additionally, we develop a
latent diffusion model within the style space of StyleGAN to enable the
generation of avatars based on image inputs. Our approach demonstrates superior
performance over current state-of-the-art methods in terms of visual quality
and diversity of the produced avatars.