360^circ Reconstruction From a Single Image Using Space Carved
  Outpainting
Authors: name: Nuri Ryu; name: Minsu Gong; name: Geonung Kim; name: Joo-Haeng Lee; name: Sunghyun Cho
Published: 2023-09-19 03:07:26

Abstract: We introduce POP3D, a novel framework that creates a full 360^circ-view 3D
model from a single image. POP3D resolves two prominent issues that limit the
single-view reconstruction. Firstly, POP3D offers substantial generalizability
to arbitrary categories, a trait that previous methods struggle to achieve.
Secondly, POP3D further improves reconstruction fidelity and naturalness, a
crucial aspect that concurrent works fall short of. Our approach marries the
strengths of four primary components: (1) a monocular depth and normal
predictor that serves to predict crucial geometric cues, (2) a space carving
method capable of demarcating the potentially unseen portions of the target
object, (3) a generative model pre-trained on a large-scale image dataset that
can complete unseen regions of the target, and (4) a neural implicit surface
reconstruction method tailored in reconstructing objects using RGB images along
with monocular geometric cues. The combination of these components enables
POP3D to readily generalize across various in-the-wild images and generate
state-of-the-art reconstructions, outperforming similar works by a significant
margin. Project page: http://cg.postech.ac.kr/research/POP3D