MVDream: Multi-view Diffusion for 3D Generation
Authors: name: Yichun Shi; name: Peng Wang; name: Jianglong Ye; name: Mai Long; name: Kejie Li; name: Xiao Yang
Published: 2023-08-31 07:49:06

Abstract: We propose MVDream, a multi-view diffusion model that is able to generate
geometrically consistent multi-view images from a given text prompt. By
leveraging image diffusion models pre-trained on large-scale web datasets and a
multi-view dataset rendered from 3D assets, the resulting multi-view diffusion
model can achieve both the generalizability of 2D diffusion and the consistency
of 3D data. Such a model can thus be applied as a multi-view prior for 3D
generation via Score Distillation Sampling, where it greatly improves the
stability of existing 2D-lifting methods by solving the 3D consistency problem.
Finally, we show that the multi-view diffusion model can also be fine-tuned
under a few shot setting for personalized 3D generation, i.e. DreamBooth3D
application, where the consistency can be maintained after learning the subject
identity.