Magic-Boost: Boost 3D Generation with Mutli-View Conditioned Diffusion
Authors: name: Fan Yang; name: Jianfeng Zhang; name: Yichun Shi; name: Bowen Chen; name: Chenxu Zhang; name: Huichao Zhang; name: Xiaofeng Yang; name: Jiashi Feng; name: Guosheng Lin
Published: 2024-04-09 16:20:03

Abstract: Benefiting from the rapid development of 2D diffusion models, 3D content
creation has made significant progress recently. One promising solution
involves the fine-tuning of pre-trained 2D diffusion models to harness their
capacity for producing multi-view images, which are then lifted into accurate
3D models via methods like fast-NeRFs or large reconstruction models. However,
as inconsistency still exists and limited generated resolution, the generation
results of such methods still lack intricate textures and complex geometries.
To solve this problem, we propose Magic-Boost, a multi-view conditioned
diffusion model that significantly refines coarse generative results through a
brief period of SDS optimization (sim15min). Compared to the previous text
or single image based diffusion models, Magic-Boost exhibits a robust
capability to generate images with high consistency from pseudo synthesized
multi-view images. It provides precise SDS guidance that well aligns with the
identity of the input images, enriching the local detail in both geometry and
texture of the initial generative results. Extensive experiments show
Magic-Boost greatly enhances the coarse inputs and generates high-quality 3D
assets with rich geometric and textural details. (Project Page:
https://magic-research.github.io/magic-boost/)