LRM: Large Reconstruction Model for Single Image to 3D
Authors: name: Yicong Hong; name: Kai Zhang; name: Jiuxiang Gu; name: Sai Bi; name: Yang Zhou; name: Difan Liu; name: Feng Liu; name: Kalyan Sunkavalli; name: Trung Bui; name: Hao Tan
Published: 2023-11-08 00:03:52

Abstract: We propose the first Large Reconstruction Model (LRM) that predicts the 3D
model of an object from a single input image within just 5 seconds. In contrast
to many previous methods that are trained on small-scale datasets such as
ShapeNet in a category-specific fashion, LRM adopts a highly scalable
transformer-based architecture with 500 million learnable parameters to
directly predict a neural radiance field (NeRF) from the input image. We train
our model in an end-to-end manner on massive multi-view data containing around
1 million objects, including both synthetic renderings from Objaverse and real
captures from MVImgNet. This combination of a high-capacity model and
large-scale training data empowers our model to be highly generalizable and
produce high-quality 3D reconstructions from various testing inputs including
real-world in-the-wild captures and images from generative models. Video demos
and interactable 3D meshes can be found on this website:
https://yiconghong.me/LRM/.