GarVerseLOD: High-Fidelity 3D Garment Reconstruction from a Single
  In-the-Wild Image using a Dataset with Levels of Details
Authors: name: Zhongjin Luo; name: Haolin Liu; name: Chenghong Li; name: Wanghao Du; name: Zirong Jin; name: Wanhu Sun; name: Yinyu Nie; name: Weikai Chen; name: Xiaoguang Han
Published: 2024-11-05 12:30:07

Abstract: Neural implicit functions have brought impressive advances to the
state-of-the-art of clothed human digitization from multiple or even single
images. However, despite the progress, current arts still have difficulty
generalizing to unseen images with complex cloth deformation and body poses. In
this work, we present GarVerseLOD, a new dataset and framework that paves the
way to achieving unprecedented robustness in high-fidelity 3D garment
reconstruction from a single unconstrained image. Inspired by the recent
success of large generative models, we believe that one key to addressing the
generalization challenge lies in the quantity and quality of 3D garment data.
Towards this end, GarVerseLOD collects 6,000 high-quality cloth models with
fine-grained geometry details manually created by professional artists. In
addition to the scale of training data, we observe that having disentangled
granularities of geometry can play an important role in boosting the
generalization capability and inference accuracy of the learned model. We hence
craft GarVerseLOD as a hierarchical dataset with levels of details (LOD),
spanning from detail-free stylized shape to pose-blended garment with
pixel-aligned details. This allows us to make this highly under-constrained
problem tractable by factorizing the inference into easier tasks, each narrowed
down with smaller searching space. To ensure GarVerseLOD can generalize well to
in-the-wild images, we propose a novel labeling paradigm based on conditional
diffusion models to generate extensive paired images for each garment model
with high photorealism. We evaluate our method on a massive amount of
in-the-wild images. Experimental results demonstrate that GarVerseLOD can
generate standalone garment pieces with significantly better quality than prior
approaches. Project page: https://garverselod.github.io/