Single-View 3D Human Digitalization with Large Reconstruction Models
Authors: name: Zhenzhen Weng; name: Jingyuan Liu; name: Hao Tan; name: Zhan Xu; name: Yang Zhou; name: Serena Yeung-Levy; name: Jimei Yang
Published: 2024-01-22 18:08:22

Abstract: In this paper, we introduce Human-LRM, a single-stage feed-forward Large
Reconstruction Model designed to predict human Neural Radiance Fields (NeRF)
from a single image. Our approach demonstrates remarkable adaptability in
training using extensive datasets containing 3D scans and multi-view capture.
Furthermore, to enhance the model's applicability for in-the-wild scenarios
especially with occlusions, we propose a novel strategy that distills
multi-view reconstruction into single-view via a conditional triplane diffusion
model. This generative extension addresses the inherent variations in human
body shapes when observed from a single view, and makes it possible to
reconstruct the full body human from an occluded image. Through extensive
experiments, we show that Human-LRM surpasses previous methods by a significant
margin on several benchmarks.