ShowRoom3D: Text to High-Quality 3D Room Generation Using 3D Priors
Authors: name: Weijia Mao; name: Yan-Pei Cao; name: Jia-Wei Liu; name: Zhongcong Xu; name: Mike Zheng Shou
Published: 2023-12-20 15:31:38

Abstract: We introduce ShowRoom3D, a three-stage approach for generating high-quality
3D room-scale scenes from texts. Previous methods using 2D diffusion priors to
optimize neural radiance fields for generating room-scale scenes have shown
unsatisfactory quality. This is primarily attributed to the limitations of 2D
priors lacking 3D awareness and constraints in the training methodology. In
this paper, we utilize a 3D diffusion prior, MVDiffusion, to optimize the 3D
room-scale scene. Our contributions are in two aspects. Firstly, we propose a
progressive view selection process to optimize NeRF. This involves dividing the
training process into three stages, gradually expanding the camera sampling
scope. Secondly, we propose the pose transformation method in the second stage.
It will ensure MVDiffusion provide the accurate view guidance. As a result,
ShowRoom3D enables the generation of rooms with improved structural integrity,
enhanced clarity from any view, reduced content repetition, and higher
consistency across different perspectives. Extensive experiments demonstrate
that our method, significantly outperforms state-of-the-art approaches by a
large margin in terms of user study.