GALA3D: Towards Text-to-3D Complex Scene Generation via Layout-guided
  Generative Gaussian Splatting
Authors: name: Xiaoyu Zhou; name: Xingjian Ran; name: Yajiao Xiong; name: Jinlin He; name: Zhiwei Lin; name: Yongtao Wang; name: Deqing Sun; name: Ming-Hsuan Yang
Published: 2024-02-11 13:40:08

Abstract: We present GALA3D, generative 3D GAussians with LAyout-guided control, for
effective compositional text-to-3D generation. We first utilize large language
models (LLMs) to generate the initial layout and introduce a layout-guided 3D
Gaussian representation for 3D content generation with adaptive geometric
constraints. We then propose an object-scene compositional optimization
mechanism with conditioned diffusion to collaboratively generate realistic 3D
scenes with consistent geometry, texture, scale, and accurate interactions
among multiple objects while simultaneously adjusting the coarse layout priors
extracted from the LLMs to align with the generated scene. Experiments show
that GALA3D is a user-friendly, end-to-end framework for state-of-the-art
scene-level 3D content generation and controllable editing while ensuring the
high fidelity of object-level entities within the scene. Source codes and
models will be available at https://gala3d.github.io/.