SEEAvatar: Photorealistic Text-to-3D Avatar Generation with Constrained
  Geometry and Appearance
Authors: name: Yuanyou Xu; name: Zongxin Yang; name: Yi Yang
Published: 2023-12-13 14:48:35

Abstract: Powered by large-scale text-to-image generation models, text-to-3D avatar
generation has made promising progress. However, most methods fail to produce
photorealistic results, limited by imprecise geometry and low-quality
appearance. Towards more practical avatar generation, we present SEEAvatar, a
method for generating photorealistic 3D avatars from text with SElf-Evolving
constraints for decoupled geometry and appearance. For geometry, we propose to
constrain the optimized avatar in a decent global shape with a template avatar.
The template avatar is initialized with human prior and can be updated by the
optimized avatar periodically as an evolving template, which enables more
flexible shape generation. Besides, the geometry is also constrained by the
static human prior in local parts like face and hands to maintain the delicate
structures. For appearance generation, we use diffusion model enhanced by
prompt engineering to guide a physically based rendering pipeline to generate
realistic textures. The lightness constraint is applied on the albedo texture
to suppress incorrect lighting effect. Experiments show that our method
outperforms previous methods on both global and local geometry and appearance
quality by a large margin. Since our method can produce high-quality meshes and
textures, such assets can be directly applied in classic graphics pipeline for
realistic rendering under any lighting condition. Project page at:
https://seeavatar3d.github.io.