Instant3D: Fast Text-to-3D with Sparse-View Generation and Large
  Reconstruction Model
Authors: name: Jiahao Li, name: Hao Tan, name: Kai Zhang, name: Zexiang Xu, name: Fujun Luan, name: Yinghao Xu, name: Yicong Hong, name: Kalyan Sunkavalli, name: Greg Shakhnarovich, name: Sai Bi

Text-to-3D with diffusion models have achieved remarkable progress in recent
years. However, existing methods either rely on score distillation-based
optimization which suffer from slow inference, low diversity and Janus
problems, or are feed-forward methods that generate low quality results due to
the scarcity of 3D training data. In this paper, we propose Instant3D, a novel
method that generates high-quality and diverse 3D assets from text prompts in a
feed-forward manner. We adopt a two-stage paradigm, which first generates a
sparse set of four structured and consistent views from text in one shot with a
fine-tuned 2D text-to-image diffusion model, and then directly regresses the
NeRF from the generated images with a novel transformer-based sparse-view
reconstructor. Through extensive experiments, we demonstrate that our method
can generate high-quality, diverse and Janus-free 3D assets within 20 seconds,
which is two order of magnitude faster than previous optimization-based methods
that can take 1 to 10 hours. Our project webpage: https://jiahao.ai/instant3d/.