ClotheDreamer: Text-Guided Garment Generation with 3D Gaussians
Authors: name: Yufei Liu; name: Junshu Tang; name: Chu Zheng; name: Shijie Zhang; name: Jinkun Hao; name: Junwei Zhu; name: Dongjin Huang
Published: 2024-06-24 17:25:39

Abstract: High-fidelity 3D garment synthesis from text is desirable yet challenging for
digital avatar creation. Recent diffusion-based approaches via Score
Distillation Sampling (SDS) have enabled new possibilities but either
intricately couple with human body or struggle to reuse. We introduce
ClotheDreamer, a 3D Gaussian-based method for generating wearable,
production-ready 3D garment assets from text prompts. We propose a novel
representation Disentangled Clothe Gaussian Splatting (DCGS) to enable separate
optimization. DCGS represents clothed avatar as one Gaussian model but freezes
body Gaussian splats. To enhance quality and completeness, we incorporate
bidirectional SDS to supervise clothed avatar and garment RGBD renderings
respectively with pose conditions and propose a new pruning strategy for loose
clothing. Our approach can also support custom clothing templates as input.
Benefiting from our design, the synthetic 3D garment can be easily applied to
virtual try-on and support physically accurate animation. Extensive experiments
showcase our method's superior and competitive performance. Our project page is
at https://ggxxii.github.io/clothedreamer.