MicroDreamer: Zero-shot 3D Generation in sim20 Seconds by Score-based
  Iterative Reconstruction
Authors: name: Luxi Chen; name: Zhengyi Wang; name: Chongxuan Li; name: Tingting Gao; name: Hang Su; name: Jun Zhu
Published: 2024-04-30 12:56:14

Abstract: Optimization-based approaches, such as score distillation sampling (SDS),
show promise in zero-shot 3D generation but suffer from low efficiency,
primarily due to the high number of function evaluations (NFEs) required for
each sample. In this paper, we introduce score-based iterative reconstruction
(SIR), an efficient and general algorithm for 3D generation with a multi-view
score-based diffusion model. Given the images produced by the diffusion model,
SIR reduces NFEs by repeatedly optimizing 3D parameters, unlike the single
optimization in SDS, mimicking the 3D reconstruction process. With other
improvements including optimization in the pixel space, we present an efficient
approach called MicroDreamer that generally applies to various 3D
representations and 3D generation tasks. In particular, retaining a comparable
performance, MicroDreamer is 5-20 times faster than SDS in generating neural
radiance field and takes about 20 seconds to generate meshes from 3D Gaussian
splitting on a single A100 GPU, halving the time of the fastest zero-shot
baseline, DreamGaussian. Our code is available at
https://github.com/ML-GSAI/MicroDreamer.