DreamSpace: Dreaming Your Room Space with Text-Driven Panoramic Texture
  Propagation
Authors: name: Bangbang Yang; name: Wenqi Dong; name: Lin Ma; name: Wenbo Hu; name: Xiao Liu; name: Zhaopeng Cui; name: Yuewen Ma
Published: 2023-10-19 19:29:23

Abstract: Diffusion-based methods have achieved prominent success in generating 2D
media. However, accomplishing similar proficiencies for scene-level mesh
texturing in 3D spatial applications, e.g., XR/VR, remains constrained,
primarily due to the intricate nature of 3D geometry and the necessity for
immersive free-viewpoint rendering. In this paper, we propose a novel indoor
scene texturing framework, which delivers text-driven texture generation with
enchanting details and authentic spatial coherence. The key insight is to first
imagine a stylized 360{\deg} panoramic texture from the central viewpoint of
the scene, and then propagate it to the rest areas with inpainting and
imitating techniques. To ensure meaningful and aligned textures to the scene,
we develop a novel coarse-to-fine panoramic texture generation approach with
dual texture alignment, which both considers the geometry and texture cues of
the captured scenes. To survive from cluttered geometries during texture
propagation, we design a separated strategy, which conducts texture inpainting
in confidential regions and then learns an implicit imitating network to
synthesize textures in occluded and tiny structural areas. Extensive
experiments and the immersive VR application on real-world indoor scenes
demonstrate the high quality of the generated textures and the engaging
experience on VR headsets. Project webpage:
https://ybbbbt.com/publication/dreamspace