Global-Local Tree Search for Language Guided 3D Scene Generation
Authors: name: Wei Deng, name: Mengshi Qi, name: Huadong Ma

Large Vision-Language Models (VLMs), such as GPT-4, have achieved remarkable
success across various fields. However, there are few studies on 3D indoor
scene generation with VLMs. This paper considers this task as a planning
problem subject to spatial and layout common sense constraints. To solve the
problem with a VLM, we propose a new global-local tree search algorithm.
Globally, the method places each object sequentially and explores multiple
placements during each placement process, where the problem space is
represented as a tree. To reduce the depth of the tree, we decompose the scene
structure hierarchically, i.e. room level, region level, floor object level,
and supported object level. The algorithm independently generates the floor
objects in different regions and supported objects placed on different floor
objects. Locally, we also decompose the sub-task, the placement of each object,
into multiple steps. The algorithm searches the tree of problem space. To
leverage the VLM model to produce positions of objects, we discretize the
top-down view space as a dense grid and fill each cell with diverse emojis to
make to cells distinct. We prompt the VLM with the emoji grid and the VLM
produces a reasonable location for the object by describing the position with
the name of emojis. The quantitative and qualitative experimental results
illustrate our approach generates more plausible 3D scenes than
state-of-the-art approaches. Our source code is available at
https://github.com/dw-dengwei/TreeSearchGen .