Direct Numerical Layout Generation for 3D Indoor Scene Synthesis via Spatial Reasoning
Authors: Xingjian Ran
Yixuan Li
Linning Xu
Mulin Yu
Bo Dai
Venue: NeurIPS 2025 poster ()
Keywords: 3D indoor scene synthesis
text-to-3D layout generation
spatial reasoning
Primary area: applications

Abstract: Realistic 3D indoor scene synthesis is vital for embodied AI and digital content creation. It can be naturally divided into two subtasks: object generation and layout generation. While recent generative models have significantly advanced object-level quality and controllability, layout generation remains challenging due to limited datasets. Existing methods either overfit to these datasets or rely on predefined constraints to optimize numerical layout that sacrifice flexibility. As a result, they fail to generate scenes that are both open-vocabulary and aligned with fine-grained user instructions. We introduce DirectLayout, a framework that directly generates numerical 3D layouts from text descriptions using generalizable spatial reasoning of large language models (LLMs). DirectLayout decomposes the generation into three stages: producing a Bird's-Eye View (BEV) layout, lifting it into 3D space, and refining object placements. To enable explicit spatial reasoning and help the model grasp basic principles of object placement, we employ Chain-of-Thought (CoT) Activation based on the 3D-Front dataset. Additionally, we design CoT-Grounded Generative Layout Reward to enhance generalization and spatial planning. During inference, DirectLayout addresses asset-layout mismatches via Iterative Asset-Layout Alignment through in-context learning. Extensive experiments demonstrate that DirectLayout achieves impressive semantic consistency, generalization and physical plausibility.

Direct Numerical Layout Generation for 3D Indoor Scene Synthesis via Spatial Reasoning
Xingjian Ran
Yixuan Li
Linning Xu Mulin Yu?
Bo Dail*
The University of Hong Kong "Shangaai Artificial Intelligence Laboratory Thhe Chinese University of Hong Kong
https://directlaag: http:ectllllouug.ggguu..bbbb.. github.io/
LayoutGPT
I-Design
Holodeck
Ours
A fecsball table is at the center of game room, with two gaming chairs positioned its long sides, large TV sits on a wooden stand against the wall, behind the foosball
left
silver fridge. On displayed on the wolls, one noxt to the booksheif and one next to the TV.
bookshelf filled
gomes. posters
Figure 1: Our method synthesizes 3D indoor scenes from text descriptions via direct numerical layout generation, demonstrating strong performance both instruction compliance and physical plausibility. In contrast, existing methods often suffer from issues related to inappropriate placement and size, as highlighted by the red circles, Furthermore, they struggle to identify all the entities in fine-grained user instruction resulting in object omission, indicated by the yellow circles. All methods share the same assets generated by 3D object generation method to ensure fair comparison.
Abstract
Realistic 3D indoor scene synthesis is vital for embodied AI and digital content creation. It can be naturally divided into two subtasks: object generation and layout generation. While recent generative models have significantly advanced object-level quality and controllability layout generation remains challenging due limited datasets. Existing methods either overfit to these datasets or rely on predefined constraints to optimize numerical layout that sacrifice flexibility. As a result, they fail to generate scenes that are both open- vocabullay and aligned with fine ine-grained user instructions. We introduce Direct LiyoutLayout, a framework that directly generates numerical 3D layouts from text descriptions using generalizable spatial reasoning of large language models (LLMs). DirectLayout decomposes the generation into three stages: producing Bird's E-Eye View (BEV) layout, lifting it into 3D space, and refining object placements. To enable explicit spatial reasoning and help the model grasp basic principles of object placement, employ Chain-of- Thought (CoT) Activation based on the 3D-Front dataset. Additionally, we design CoT- -rounddddd Generative Layout Reward enhance generalization and spatial planning. During inference, Direct ayout addresses asset-layout mismatches via Iterative Asset- Layout Alignment through in-context learning. Extensive experiments demonstrate that DirectLayout achieves impressive semantic consistency, generalization and physical plausibility.
"Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Introduction
High-fidelity, spatially coherent 3D indoor seenes are crucial for embodied AI, virtual reality, and film production. Particularly in advancing embodied AI, the training of agents for tasks like navigation and object manipulation relies heavily on realistic and diverse simulation environments. To address the complexity of 3D scene synthesis, the task can be divided into two stages: object generation and layout generation. Recent 3D generative models [28 33 351 have sigmificantly advanced the quality and controllability of object generation. In contrast, layout generation remains underexplored, with key challenges including physical plausibility, semantic alignment, and diversity. Physical plausibility requires that objects are arranged in ways consistent with real-world physics, for instance, they should rest stably on surfaces and avoid unnatural overlaps. Semantic alignment means the layout must accurately reflect the user's textual instructions, such as placing bed next to a window when requested. Diversity involves the ability to generate a wide variety of object types and room layouts, extending beyond typical bedrooms and living rooms to spaces like kitchens, offices, or playrooms.
Due to limitations in scale, diversity, and realism, existing 3D indoor layout datasets struggle to reflect the true distribution of real-world scenes. Directly training generative models such datasets [6 19 30), often leads to models that simply memorize dataset-specific patterns, resulting in poor generalization
novel room and object types. To alleviate this data bottleneck, recent approaches [ g 31 introduce manually defined spatial constraints, guiding the numerical layout generation process through intermediate representations such as scene graphs. These methods improve physical plausibility and reduce hallucinations. However, it inherently sacrifices flexibility and makes challenging to accommodate fine- fine-grained user instructions. This limitation stems from the reliance on predefined constraints, which struggle to capture the diversity and complexity of real-world layouts. Because object placements frequently depend on contextual, functional, and aesthetic factors, all of which are difficult to exhaustively specify with fixed rules.
To address the data challenge, enable the model to learn generalizable spatial reasoning and directly generate numerical layouts without relying on predefined rules. Our pipeline uses large language models (LLMs) generate 3D layout and object descriptions from textual input through decomposed processes. During generation, the core of spatial reasoning in LLMs lies in Chain-of- Thought (CoT), which comprises two key components. First, CoT Activation is introduced during training to enable structured reasoning steps, helping the model grasp fundamental spatial logic and forming a prior for spatial planning. This is achieved by supervising a four-step reasoning process that decomposes layout generation into subtasks, allowing the model to better understand and organize spatial information. Second, CoT-Grounded Generative Layout Reward provides learning signal to improve generalizable spatial reasoning under limited data, by assessing the plausibility of object placements and their consistency with the COT reasoning process. It introduces dual- evaluator framework, where dision-language model (VLM) captures high-level spatial and semantic violations, while reasoning LLM identifies fine-grained numerical and logical inconsistencies, together offering structured, interpretable evaluation. After generation, we incorporate Iterative Asset- Layout Alignment to refine layout-object consistency and enhance scene realism based on spatial and semantic feedback provided by the aforementioned dual-evaluator. As shown in Figure [ existing methods often suffer from inappropriate placements and object omission. In contrast, our approach generates 3D scenes with higher physical plausibility and better alignment with the user instruction.
Our contributions are summarized follows:
We propose novel framework for 3D indoor scene synthesis that generates and refines layouts directly from text descriptions, bypassing the need for constrained optimization. This is achieved through carefully designed task decomposition, which simplifies the process and improves overall efficiency.
We introduce CoT reasoning into 3D scene synthesis, enhancing the spatial planning capabilities of the model. By connecting CoT reasoning with reinforcement learning through CoT-Grounded Generative Layout Reward, we significantly improve the model's reasoning accuracy and reduce hallucination. Additionally, we curate a CoT dataset for 3D indoor layouts to support this approach.
Our DirectLayout outperforms existing methods in general 3D scene synthesis, demonstrating that direct generation of numerical layouts is highly effective. The generated layouts are more consistent with physical laws and show improved semantic consistency. Moreover, the direct generation of numerical layouts allows for better fine-graines control.

---

2 Related Work
2.1 3D Scene Synthesis Approaches
3D indoor scene synthesis methods vary in representation and generation strategy. Rule-based methods 4 [2] provide structured layouts and are particularly suited for task-driven evaluation. In contrast, recent research has focused on learning-based models [1 (7), which aim generalize from data and facilitate the generation of more diverse and semantically rich scenes. ATISS [99 uses permutation-equivariiee transformer to generate object sets conditioned on floor plan, supporting interactive editing. LayoutGPT [6] treats an LLM as a visual planner to predict layouts from text using CSS-like prompts, performing competitively in both 2D) and 31) settings. Structured approaches like InstructScene [16] and AnyHome 8 incorporate scene graphs as intermediates to enhance control and semantic alignment. Holodeck 31 encodes relational constraints via L.M and solves constraint optimization problem for object placement. LayoutVL.N LLM (23] combines visual grounding with differentiable optimization, refining object placements through clatton-based objectives and initial numerical poses, while making deliberate trade-off's between flexibility and realism. [5] propose hierarchical planning strategy for 3D scene generation using VLMs. Their method decomposes scenes into room, region, and object levels, using tree search to explore layout options and improve spatial plausibility. Previous approaches mainly focus on better modeling the distribution of indoor scene layouts, whereas our method leverages limited data to learn the underlying placement logic of indoor layouts.
2.2 Reasoning Augmentation in Vision
Recent advancements 99
32 34 36 have utilized the general reasoning capabilities of LLMs and
VLMs to tackle visual tasks. However, other works delve deeper into methods designed to explicitly enhance their reasoning abilities and performance within the visual domain. For instance, [3] prompts LLMs to interpret complex textual descriptions step-by-step, transforming them into structured 2D layouts that improve compositional accuracy, which are subsequently used to condition diffusion models. LLaVA-CoT [29 introduces a multi-stage reasoning framework which substantially boosts performance complex visual question-answering benchmarks. SpatialCoT [17] enhances embodied task planning by aligning vision-language inputs with spatial coordinates and applying chain-of- thought spatial grounding, resulting in superior performance in navigation and manipulation tasks. (5) also enhance layout reasoning by discretizing the scene into symbolic grids and prompting VLMs to iteratively generate object placements, combining structured spatial reasoning with visual grounding. Furthermore, [II) integrates learned reward models, PARM and PARM++, into autoregressive image generation, facilitating stepwise verification that improves image quality and alignment. Drawing inspiration from these works, our method seeks to enhance layout reasoning through the use of explicit logical outputs and reward-based feedback.
Method
3.1 Problem Formulation
Given general textual descriptions and user-specified room dimensions, our goal is to synthesize physically and semantically plausible 3D indoor scenes. The synthesis process focuses on generating structured layouts, while 3D object assets are handled by existing generative methods. We represent the 2D top-down layout (BEV layout) as:
LBEV [(1,, Vo, 4, W3y o:)) on))
where (z,, y1) is the center coordinate of object i, (li, wi) are its length and width, and is its orientation (rotation angle around the z-axis).
To extend this into 3D, we define the 3D layout as:
Lap = {z,, Vi, Zis 4G wi, h1, Oi, p:))
(2)
where 2; and h; denote the vertical center and height of object i, and p; is a text prompt used to guide 3D asset generation.
3

---

Traning Stage
Choin
dell metter
Cal-Grourded Generghive Layout Reward
Curation
COnder becid
SFT
Spotial Exslustor
Deventing chall apeect
DPO
30 Front beteset
IEV Layour Data
styplom Dry aia
BEV Layout
Genereter
Quantitative Evelustor
Inference Stage
1D Layoun Gorerator
Aaeeli robed
dessresm teacher's
Iterative Asseh Loyout Algemest
Generator
Object Generater
Quantitotivz
Sparta
Evoluttor
Esoluatoo
Figure 2: Overview of our method. Training Stage: BEV Layout Generator is first fine-tuned on BEV layouts curated from the 3D-Front dataset, guided by CoT annotations generated by GPT-4o. Subsequently, further optimized through DPO, leveraging CoT-Grounded Generative Layout Reward derived from Spatial Evaluator (VLM) and Quantitative Evaluator (reasoning LLM). Infer- ence Stage: Given a text prompt, BEV Layout Generator produces a 2D layout, which then lifted to 3D layout by 3D Layout Generator. Iterative Asset-Layout Alignment refines the 3D scene by using the Spatial Evaluator and Quantitative Evaluator to provide feedback to the layout generators, ensuring consistency between the layout and generated 3D assets from object generator.
3.2 Overview
Despite the powerful reasoning capabilities of recent foundation language models, we observe that they often lack spatial planning ability. To address this limitation, we decompose the scene synthesis task into three stages that are comparatively easier to handle, while preserving the overall spatial consistency, as illustrated in Figure 2 We first use a BEV Layout Generator to convert the input text into 2D top-down layout. Section 3.3 details how enhance the model's spatial reasoning through CoT Activation and CoT-Grounded Generative Layout Reward. The resulting BEV layout is then lifted into 3D layout using the 3D Layout Generator, which also assigns descriptive text prompt to each object instance to facilitate asset generation (Section 3.4p. Finally, we apply Iterative Asset- Layout Aligument to reconcile discrepancies between the predicted layout and available assets during inference, using feedback from evaluators and in-context learning (Section 3.5)). All components in pipeline, including generators and evaluators, are built upon I.LMs VLMSs. Among them, only the BEV Layout Generator is explicitly fine-tuned for layout planning.
3.3 Enhancing Spatial Reasoning in BEV Layout Generation
3.3.1
CoT Activation
We leverage CoT reasoning to activate and guide the basic spatial planning capabilities of the BEV Layout Generator. Inspired by recent advancements in symbolic and mathematical reasoning with LLMs 13 (6)) we design structured four-step CoT process. As illustrated in Figure 2 reasoning begins with entity extraction, where the model parses the input text to identify object categories and their quantities. This is followed by order decision step, which establishes semantically grounded placement sequence by prioritizing foundational objects. Next, spatial reasoning is performed to infer each object's location, size, and orientation, relying common sense and relative spatial relationships. Finally, the answer organization step produces the BEV layout in a structured numerical format suitable for downstream use.
To supervise this reasoning process, we filter approximately 6,500 scenes from the 3D-Front dataset [7] following the pre-processing of ATISS [19,, and use GPT-4o [2] to generate the CoT

---

annotations for ground-truth layouts, guided by carefully designed prompts provided in Appendix A. We then fine-tune BEV Layout Generator using supervised fine-tuning (SFT). While this improve layout quality and reasoning fidelity, we observe occasional hallucinations and inconsistency between reasoning and final placements, especially on unseen object or scene types.
3.3.2 CoT-Grounded Generative Layout Reward
Due to the lack of large-scale and diverse scene layout datasets, it remains challenging for fine-tuned BEV Layout Generator to generalize well unseen complex situations. To further improve gener- alizable reasoning in layout generation, we propose CoTTGGounddd Generative Layout Reward, which assesses layout plausibility through dual-evaluator framework: Spatial Evaluator and Quantitative Evaluator. This dual-evaluator framework benefits from the complementary strengths of a VLM and reasoning LLM, leading to more robust layout assessment. The VLM serves Spatial Evaluator, adept at capturing high-level spatial and semantic implausibility by leveraging both visual and textual context to evaluate the result and its consistency with CoT. In contrast, the reasoning LLM functions as Quantitative Evaluator, excelling at identifying fine fine-grained numerical and logical inconsistencies. particularly those that are more subtle and easily discernibee through visual cues alone. Moreover, instead of relying on absolute score prediction, which is difficult due to the subjective and diverse nature of layout design, our reward focuses on detecting violations of physical and common sense constraints, aligning more closely with the pretraining priors of large models.
Object-Levve Criteria. To provide comprehensive evaluation of layout quality, assess each generated layout at the object level using seven criteria divided between the two evaluators:
Spatial Evaluator:
Relative Alignment (C1): Measures the reasonableness of spatial relations with related objects.
Global Positioning (C2): Assesses if the position aligns with the room context and prompt.
Consistency with CoT (C3): Ensures that object placements align with the spatial instructions inferred explicitly stated in the CoT description.
Quantitative Evaluator:
Inter-object Distance (CA): Checks that spacing between objects supports accessibility and func- tionality. Overlaps may be acceptable if vertical stacking is plausible.
Size Proportion (Cs):: Evaluates whether object sizes are reasonable within the scene.
Orientation Validity (C&): Ensures that object orientations are semantically appropriate.
Quantity Alignment (C7): Validates whether the number of instances per object class aligns with expectations given the room's purpose.
Each criterion Cg (k 2, 4, 5)) is associated with validity ratio rk. which serves as the reward for the layout under that criterion. The ratio is defined as Oz/N, where O₂ denotes the number of objects that meet criterion Ck, and N the total number of objects in the layout. Thus, rk quantifies the degree to which the layout conforms to the corresponding requirement. For C7, the quantity alignment criterion, we define:
Σ
actual expected
_e=1
ne
F7 max
0.
1
M Dan
expected
Mi
where netcuual and ne of object classes.
expocted denote the actual and expected object count for class c, and M the number
By integrating both evaluators, construct reward signal that captures both semantic and numeric fidelity of layouts, providing holistic feedback.
Scenc-Ieve Reward Aggregation. To obtain a unified reward per sample, we apply the entropy weight method, as different scene descriptions impose uneven difficulty across criteria. Entropy captures the information content of each criterion in varying contexts, enabling more adaptive reward
5

---

aggregation. Given T layout samples {Si,..., } for prompt, and their criterion caterioo--wise validity ratios ( for sample j, the entropy of criterion is:
HA
5 Pk
() In Pk
where ()
(4)
In
T
The final reward RD) for sample Sj is computed weighted average of the criteria, where each weight is inversely related to the entropy Hg:
RD) = Σ (1 - (5) k=1 Ek=n -+)
(5)
Model Training, We collect 200 diverse prompts with room dimensions using GPT-4o [12]. featuring varying levels detail (see Appendix A..3). and generate 30 BEV layout samples per prompt. Each sample is assigned reward based on the above procedure, We construct the training set by sampling data pairs whose reward differences exceed a certain threshold (with the higher-scoring sample labeled as chosen and the lower-scoring one as rej rejected), and fine-tune BEV layout generator using the Direct Preference Optimization (DPO) [1]. More training details can be found Appendix A.2
3.4 Layout Lifting and Scene Synthesis
Given the predicted BEV layout, we employ 3D I Layout Generator to infer missing vertical attributes such object heights and vertical positions, guided by the scene description and structured BEV layout. It leverages the LLM's strong prior knowledge of object sizes and height relationships (e.g., beds are lower than tables, lamps above desks), which are generally common sense, enabling effective zero-shot inference. In addition to completing vertical attributes, it generates a descriptive prompt P: for each object, capturing category, shape, and style support semantically aligned 3D asset generation.
To synthesize the final 3D scene, we generate assets from the prompts using existing high-fidelity 3D object generation model (28). Each asset is resized according to (4,, wi, h;) and positioned using (2i, yi, 2;) and o,, thus completing the full scene assembly pipeline.
3.5 Iterative Asset-Layout Alignment
Although the generated prompts contain detailed textual descriptions, the generators lack direct access to visual information about 3D assets, often leading to inconsistencies between predicted layouts and generated objects. Specifically, the orientation of generated objects can inconsistent, and instances of the same category may differ significantly in shape, such as a bathtub filled with water compared to an empty one, leading to uncertainties in the height related objects (see Figure for an example).
To address this, we propose Iterative Asset-Layout Alignment. After rendering the composed 3D scene, we re-evaluate it using the same Spatial Evaluator and Quantitative Evaluator introduced in Section 3.3 based on shared set of object-level criteria C For each violated criterion, the evaluators provide object-specific suggestions to adjust size, position, or orientation. These are then fed back into the layout generators contextual input for iterative refinement. This process repeats until either no further revisions are proposed by the evaluators or a maximum number of refinement steps is reached. Let L denote the 3D layout at iteration t, and let FF()) represent the evaluator feedhack at iteration 1, which consists of a set of suggestions or corrections based on the violation of spatial or semantic criteria. The updated layout is defined as:
L (((+1) Update(L[)) F((1),
66)
where Update applies layout modifications suggested by the evaluators. Through this looped align- ment mechanism, our system able to dynamically correct mismatches between layout and asset geometry, resulting in coherent and realistic 3D scenes.

---

LayoutGPT
I-Design
Holodeck
Ours
kitchen with refrigerator, sink, countertop, table, and several cobinets, with few bar stools
computer clessrcom with individual computer desks, ergonamic chairs, projector screen, a table, printer, and network hub
home theater, positioned against wall, flonked by speakers front co ffffee table holds popcorn chips. safa sits behind the coffee table, and severol bean bags are arranged around it.
wooden table holds leptop the front left with of colored pencils placed vase filled with flowers, and in front of the vase lie a pair block-framed glasses. On the left side the table white coffee cup, while wooden chair positioned front of the table,
the center back the table blue
Figure 3: Qualitative comparisons with scene synthesis methods. We compared generated scenes with existing methods across various scene types and coarse-to- fine prompt granularities. Our results demonstrate better alignment with the text descriptions across different prompt granularities and scene types.
Experiments
4.1 Experimental Setup
We evaluate our approach on a set 15 indoor scene categories, each associated with three distinct prompts and room sizes. These test indoor scenes automatically generated and manually verified to ensure consistency across categories. To assess effectiveness and reproducibility our method, we implement two distinct system configurations. In the Closed-Souree Enhanced setting, we utilize GPT-4o 12 as both 3D Layout Generator and Spatial Evaluator, while ol [3] serves as Quantitative Evaluator. In contrast, the Open-Source Only configuration offers fully reproducible alternative by employing Qwen2.5-77 52B (24] as 3D Layout Generator, Qwen2.5-VE VL--22 [5] as Spatial Evaluator, and QwQ-32B 226 Quantitative Evaluator.
In both configurations, we use fine-tuned Qwen2.5-32B as BEV Layout Generator. We compare method against several recent baselines in text-to-layout generation, including LayoutGPT (6).

---

Table 1: Comparison with existing methods. Our approach outperforms baselines in both physical
plausibility and semantic alignment on general indoor scenes.
Method
Out of Bound
Collision 4
Pos. f
Rot.
PSA t
CLIP Score
Holodeck
36.33
8,32
51.56
50.18
49.78
25.21
I-Design
16,00
18.85
13.47
13.82
29.33
14.09
LayoutGPT
33.91
15.89
57.24
51.31
55.11
21,65
Ours w/ QwQ BEV
18.85
12.50
44.33
43.64
47.11
23.83
Ours (open-source)
9.20
8.45
70.73
67.51
73.78
26.47
Ours (full)
8.74
6.31
72.04
69.87
75.56
27.43
Additionally, we construct a strong baseline by replacing our specialized BEV Layout Generator with a general reasoning LLM (QwQ-32B) to examine the effect of inductive spatial bias.
Holodeck [3I] and I-Design (2].
Evaluation Metrics:
To evaluate physical plausibilityy employ two metrics: Out-of-Bound Rate and Collision Rate. The Out-of-Bound Rate is calculated as the proportion of objects that extend beyond the scene boundaries relative the total number of objects. Similarly, the Collision Rate represents the proportion of objects that collide with other objects relative to the total number of objects. Following LayoutVLM [23], we assess semantic alignment and overall quality using Positional Coherency (Pos.). Rotational Coherency (Rot.), and the Alignment Score (PSA). In Layout these LLM-based metries, evaluated using GPT-4o. have been shown to correlate with human judgments. Additionally, we report the CLIP Score 20 to quantify the consistency between the generated layout and the input text prompt.
Semantic
4.2 Comparison with Baselines
We conduct a series of comparative evaluations aiming to verify the advantages of our approach over baselines. These comparisons reveal the extent to which our model improves physical feasibility, semantic fidelity, and user-controllable scene synthesis.
Comparison with Existing Methods. Table I and Figure 3 summarize the comparative results across key metrics and visual outputs. LayoutGPT directly predicts object placements using in- context examples. However, the absence of logical training leads to outputs that violate physical constraints and deviate from user instructions. Its generated layouts often lack structural coherence and semantic fidelity. Although LayoutGPT employs direct numerical layout generation, its limited spatial reasoning hampers its ability to translate detailed descriptions into coherent layouts.
While I- I-Design leverages cooperative reasoning among multiple agents to coordinate layouts, it suffers from limited robustness when generating layout, particularly due to failures in agent communication or layout grounding. Although it produces physically plausible arrangements, the generated objects tend to cluster locally, lacking global semantic awareness. This is partly because I-Design extracts symbolic constraints before layout generation, which improves physical plausibility but limits the model's flexibility in handling complex and fine-grained spatial relationships often found in natural language, such as "a vase in the center of the table, with pair of glasses in front of it." Its reliance on predefined constraints ultimately restricts controllability and semantic fidelity.
Holodeck adopts a search-basee strategy that effectively avoids object collisions, but similarly falls short in capturing prompt semantics, often generating layouts with limited object diversity and local focus. Like I-Design, Holodeck also relies on symbolic constraint, which enhances physical validity but weakens its ability express detailed spatial instructions described in natural language.
Conversely, our open-source and full model enable both accurate physical constraint satisfaction and faithful semantic grounding. By integrating spatial reasoning directly into the numerical layout generation process without relying on symbolic constraints, our method maintains high physical plausibility and semantic alignment across all levels of prompt granularity from coarse scene and object types to fine- mne-grained spatial instructions as evidenced in Figure 3
Comparison with Reasoning LLM. To isolate the contribution of our specialized BEV Layout Generator, we replace with a general-purposs reasoning model, QwQ-32B, while keeping the rest of our pipeline unchanged. This variant- reffered to as Ours w/ QwQ BEV in Table -eeeees an ablation baseline to evaluate the role of spatial inductive bias. Thanks to its strong capabilities in numerical computation and local logical inference, QwQ-32B handles basic geometric constraints
serves as
8

---

Table 2:
Ablation study. We evaluate the effect of task decomposition and reward granularity.
Method
Out of Bound
Collision
Pos. t
Rot.
PSA f
CLIP Score
w/o decomp
21.42
20.52
67.13
65.47
71.56
26.05
w/o reward
26.04
16.77
64.47
57.18
68.82
25.25
simple reward
13.64
11.89
67.95
64.37
70.76
25.82
Ours (full)
8.74
6.31
72.04
69.87
75.56
27.43
w/o decomp
w/o reward
simple reward
full
Hiii
hallway rack, coot hooks, mirror, bench, umbrella stond, table, a rug.
Figure 4: Ablation Study Results. The experiment validates the effectiveness of task decomposition and proposed Grounddddd Generative Layout Reward.
relatively well, maintaining acceptable rates of boundary and collision violations. However, the model often fails to preserve the holistic semantic context of scene, leading degraded performance in positional coherency, rotational alignment, and overall semantic alignment. This result suggests priors
that ceneral-purposs models, while strong localized inference, lack the required for globally consistent layout synthesis. These findings, as reflected in Table highlight the advantage of incorporating dedicated spatial reasoning: they not only enforce physical feasibility but also enhance high-level understanding of spatial semantics.
4.3
Ablation Study
To gain deeper understanding of the contribution of different components in our design, we conducted a series of ablation experiments focusing on the reward formulation and task decomposition. These experiments were designed to isolate the impact of each factor on the overall performance.
CoT-Grounded Generative Layout Reward. We investigate the impact of different reward con- figurations on the spatial plausibility of the generated layouts. As shown in Table and in Figure 4 when BEV Layout Generator is trained solely with supervised learning, without any reinforcement signal (wo reward), the physical plausibility significantly deteriorated. This is evident in the substantially higher rates of objects being placed outside the scene boundaries and colliding with each other. Introducing simple reward, where VLM provided single overall score for the layout quality (simple reward; see Appendix A. for details), leads to some improvement compared the no-reward scenario. However, this approach still fall short of our full method's 's performance. These results suggest that relying high-level semantic feedback from a general foundation model not sufficient for effectively guiding the learning physically realistic layouts. In contrast, our method, which utilizes fine-grained, spatially informed rewards, achieved considerably better results across all evaluation metrics presented in Table 2 and visualized Figure 4 This underscores the critical role of structured spatial supervision in generating plausible indoor scenes.
and illustrated
Task Decomposition. We evaluate the significance of our architectural modularity by comparing our full model with variant where 3D layout generation and refinement were performed by single model, without the intermediate 2D-to-3D reasoning step (w/o decomp). The results, detailed in Table 2 and qualitatively shown in Figure 4 indicate that this approach leads to decline in physical plausibility and slightly weaker global semantic alignment. These results suggest that our decomposition strategy plays crucial role in enabling more accurate spatial reasoning. By separating the 2D planning stage from the 3D realization, the model appears to be better equipped to handle the inherent complexity and variability of indoor environments.

---

Table Human evaluation results across different prompt granularities. Higher scores indicate better semantic compliance (SemComp). physical plausibility (PhyPlaus), and layout rationality
(aayRat).
Method
Level
SemComp
Phy Plaul 1 LayRat Rat
coarse
3.72
1.84
1.72
LayoutGPT
medium
1.64
1.76
fine
3.00
2.32
2.04
coarse
3.72
2.72
2.44
I-Design
medium
3.08
3.36
2.48
fine
3.32
3.72
2.72
coarse
4.12
3.64
2.84
Holodeck
medium
2.80
2.52
2,48
fine
2.40
2.60
2.04
coarse
4.56
4.64
4.52
Ours
medium
4.60
4.68
4.64
fine
4.84
4.72
4.80
4.4
User Study
To further validate our approach, we conduct a user study with 25 compensated volunteers, all with relevant professional expertise. Participants asked to evaluate scene layouts generated by different methods. The layouts are randomly ordered and anonymized. Each participant scores them according three criteria: Semantic Compliance (faithfulness to the textual description), Layou! Rationality (plausibility of object placement), and Physical Plausibility (realism of object support and absence of penetration). Here, coarse, medium, and fine indicate the granularity of the input prompt, with more detailed descriptions at finer levels (see Appendix A.33 for examples).
As shown in Table 3 our method consistently outperforms all baselines across coarse, medium, and fine prompt levels, with especially large margins at the fine-grained level. This demonstrates our model's strong ability to capture detailed semantic instructions while ensuring physically plausible placements. Compared to prior methods that rely on symbolic constraints or search heuristics, our approach maintains both global semantic consistency and local physical validity.
5 Conclusion
In this paper, present DirectLayout, novel framework for generating 3D indoor scenes from text descriptions via direct numerical layout generation. follows three-stage process including BEV layout generation, 3D lifting, and iterative refinement, enhanced by CoT Activation and CoT-Grounded Generative Layout Reward to improve generalizable spatial reasoning. Extensive experiments show that DirectLayout achieves state-of-the-a performance in generating general 3D scenes that are physically plausible, semantically coherent, and faithful to user prompts. We demonstrate that generating layouts through direct numerical generation allows for higher upper bound handling more detailed and complex user inputs and scene arrangements.
Limitations. Despite these advancements, the task decomposition and Iterative Asset-Layout Align- ment, while effective, introduce additional inference time, which limits real-time interaction. Addi- tionally, the complexity of the generated scenes is constrained by the capacity of the base model as well as the scale and diversity of scenes in the training dataset, making it challenging to generate large number of small objects. Moreover, although representing scenes with intermediate BEV layout generally yields better results than directly generating full 3D layouts, BEV layout inherently reduces layout clarity, when multiple objects stacked along the vertical axis.
Acknowledgments and Disclosure of Funding
The authors would like to thank the support from the HKU Startup Fund. This work is also partially funded by the National Key R&D Program of China (2022ID0160201) and the Shanghai Artificial Intelligence Laboratory.
10

---

References
[1] Rio Aguina-Kang. Maxim Gumin, Do Heon Han, Stewart Morris, Scung Jean Yoo, Aditya Ganeshan, R Kenny Jones, Qiuhong Anna Wei, Kailiang Fu, and Daniel Ritchie. Open-universe indoor scene generation using Ilm program synthesis and uncurated object databases. arXiv preprint urXiv:2403.096: 2024.
[2] Ata Çelen, Guo Han, Konrad Schindler, Luc Van Gool, Iro Armeni, Anton Obukhov, and Xi Wang. I-design: Personalized Ilm interior designer. arXiv preprint arXiv:2404. 002838, 2024.
[3] Xiaohui Chen, Yongfei Liu, Yingxiang Yang. Jianbo Yuan, Quanzeng You, Li- Li-Ping Liu, and Hongxia Yang. Reason out your layout: Evoking the layout master from large language models for text-to-image synthesis. arXiv preprint arXiv:2311.1711 2023.
[4] Matt Deitke, Eli VanderBilt, Alvaro Herrasti, Luca Weihs, Kiana Ehsani, Jordi Salvador, Winson Han, Eric Kolve, Aniruddha Kembhavi, and Roozbeh Mottaghi. Procthor: Large-scale embodied using procedural generation. In NeurlPS, 2022.
[5] Wei Deng, Mengshi Qi, and Huadong Ma. Global-local tree search in vims for 3d indoor scene generation. arXiv preprint 2iv: 18476, 2025.
[6] Weixi Feng. Wanrong Zhu. Tsu-jui Fu, Varun Jampani, Arjun Akula, Xuehai He, Sugato Basu, Xin Eric Wang. and William Yang Wang. Layouigpt: Compositional visual planning and generation with large language models. In NeurlPS, 2023.
[7] Huan Fu, Bowen Cai, Lin Gao, Ling-Xiao Zhang, Jiaming Wang. Cao Li, Qixun Zeng, Chengyue Sun, Rongfei Jia, Binqiang Zhao, al. 3d-front: 3d furnished rooms with layouts and semantics. In ICCV, 2021.
[8] Rao Fu, Zehao Wen, Zichen Liu, and Srinath Sridhar. Anyhome: Open-vocabulary generation of structured and textured 3d homes. In ECCV, 2024.
[9] Gege Gao, Weiyang Liu, Anpei Chen, Andreas Geiger, and Bernhard Schölkopf. Graphdreamer: Composi- tional 3d scene synthesis from seene graphs. CVPR, 2024.
[10] Daya Guo, Dejian Yang. Haowei Zhang, Junxiao Song. Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma. Peiyi Wang. Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501.
2025.
[11] Ziyu Guo, Renrui Zhang, Chengzhuo Tong. Zhizheng Zhao, Peng Gao, Hongsheng Li, and Pheng-Ann Heng. Can we generate images with cot? let's verify and reinforce image generation step by step. arXiv preprint arXiv:2501. 13926, 2025.
[12] Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2400.21276 2024.
[13] Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Hel- yar, Aleksander Madry, Alex Beutel, Alex Carney, al. Openai ol system card. arXiv preprint arXiv:2412.1677 16720, 2024.
[14] Kurt Leimer, Paul Guerrero, Tomer Weiss, and Przemyslaw Musialski. Layoutenhancer: Generating good indoor layouts from imperfect data, In SIGGRAPH Asia. 2022.
[15] Yixuan Li, Xingjian Ran, Linning Xu, Tao La, Mulin Yu, Zhenzhi Wang. Yuanbo Xiangli, Dahua Lin, and Bo Dai. GS: Procedural building generation for city assembly with 3d gaussians. In CVPR, 2025.
[16] Chenguo Lin and Yadong Mu. Instructscene: Instruction- 3d indoor scene synthesis with semantic graph prior. In ICLR, 2024.
[17] Yuecheng Liu, Dafeng Chi, Shiguang Wu, Zhanguang Zhang, Yaochen Hu, Lingfeng Zhang, Yingxue Zhang, Shuang Wu, Tongtong Cao. Guowei Huang, et al. Spatialcot: Advancing spatial reasoning through coordinate alignment and chain-of-thought for embodied task planning. arXiv preprint arXiv:2501.1007. 2025.
[18] Wamiq Para, Paul Guerrero, Tom Kelly, Leonidas J Guibas, and Peter Wonka. Generative layout modeling using constraint graphs. In ICCV, 2021.
[19] Despoina Paschalidou, Amlan Kar, Maria Shugrina, Karsten Kreis, Andreas Geiger, and Sanja Fidler. Atiss: Autoregressive transformers for indoor scene synthesis. In NeurIPS, 2021.
11

---

[20] Alec Radford, Jong Wook Kim, Chris Hallacy. Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In ICML, 2021.
[21] Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model secretly a reward model. In NeurIPS, 2023.
[22] Alexander Raistrick, Lingje Mei, Karhan Kayan, David Yan, Yiming Zuo. Beining Han, Hongyu Wen, Meenal Parakh, Stamatis Alexandropoulos, Lahav Lipson, et al. Infinigen indoors: Photorealistic indoor scenes using procedural generation. In CVPR, 2024.
[23] Fan-Yur Sun, Weiyu Liu, Siyi Gu, Dylan Lim, Goutam Bhat, Federico Tombari, Manling Li, Nick Haber, and Jiajun Wu, Layoutvlm: Differentiable optimization of 3d layout via vision-language models. arXiv preprint 211::2412200299, 2024.
[24] Qwen Team. Qwen2.5: A party of foundation models, September 2024.
[25] Qwen Team. Qwen2.5-vl, January 2025.
[26] Qwen Team. Qwq-32b: Embracing the power of reinforcement learning, March 2025.
[27] Xinpeng Wang. Chandan Yeshwanth, and Matthias Nießner. Sceneforme: Indoor scene generation with transformers. 3DV, 2021.
[28] Jianfeng Xiang, Zelong Lv, Sicheng Xu, Yu Deng, Ruicheng Wang, Bowen Zhang, Dong Chen, Xin Tong, and Jiaolong Yang. Structured 3d latents for scalable and versatile 3d generation. arXiv preprint urXiv:2412.01506, 2024.
[29] Guowei Xu, Peng Jin, Li Hao, Yibing Song, Lichao Sun, and Li Yuan. Llava-cot: Let vision language models reason step-by-step. arXiv preprint arXiv:2411, 10440,, 2024.
[30] Yixuan Yang, Junru Lu, Zixiang Zhao, Zhen Luo, James JQ Yu, Victor Sanchez, and Feng Zheng. Liplace: The indoor scene layout generation and editing via large language model. arXiv preprint arXiv:2406. 03866, 2024.
[31] Yue Yang, Fan-Yin Sun, Luca Weihs, Eli VanderBilt, Alvaro Herrasti, Winson Han, Jiajun Wu, Nick Haber, Ranjay Krishna, Lingjie Liu, et al. Holodeck: Language guided generation of 3d embodied ai environments. In CVPR, 2024.
[32] Tatiana Zemskova and Dmitry Yudin. 3dgraphlim: Combining semantic graphs and large language models for 3d scene understanding. arXiv preprint arXiv:2412.1 18450, 2024.
[33] Longwen Zhang, Ziyu Wang, Qixuan Zhang, Qiwei Qiu, Angi Pang, Haoran Jiang, Wei Yang, Lan Xu, and Jingyi Yu. Clay: A controllable large-scale generative model for creating high-quality 3d assets. In SIGGRAPHI 2024.
[34] Qihang Zhang, Chaoyang Wang, Aliaksandr Siarohin, Peiye Zhuang, Yinghao Xu, Ceyuan Yang, Dahua Lin, Bolei Zhou, Sergey Tulyakov. and Hsin-Y Ying Lee. Towards text-guided 3d scene composition. In CVPR. 2024.
[35] Ziho Zhao, Zeqiang Lai, Qingxiang Lin, Yunfei Zhao, Haolin Liu, Shuhui Yang. Yifei Feng, Mingxin Yang, Sheng Zhang, Xianghui Yang, al. Hunyuan3d 2.0: Scaling diffusion models for high resolution textured 3d assets generation. arXiv preprint wrXiv:2501.12200 2025.
[36] Xiaoyu Zhou, Xingjian Ran, Yajiao Xiong. Jinlin He, Zhiwei Lin, Yongtao Wang. Deqing Sun, and Ming-Hsuan Yang. Gala3d: Towards text-to-3d complex scene generation via layout-guided generative gaussian splatting. In ICML, 2024.
12

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims regarding the novelty and effectiveness of our proposed method are clearly presented the Abstract and Section 1 (Introduction). These claims are thoroughly supported by the methodological details in Section 3 and validated through extensive experimental results and comparisons in Section 4 (Experiments).
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification; We explicitly discuss the limitations of our approach in Section 5 (Conclusion), including constrained scene complexity and the online interaction of the generator with the environment. This reflects our intent to transparently acknowledge areas for future improvement.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model vell-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-tex" system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledger in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
13

---

3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification: The paper does not present any theoretical results or formal proofs. The contributions are primarily empirical and methodological.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proof' in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide all necessary details to reproduce our main results, including algorithmic procedures Section 3 (Method), experimental setups and configurations in Section 4 (Experiments), as well as prompts and hyperparameters in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
14

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: We have not released the code and data at the time of submission. However, plan to make both publicly available with detailed instructions upon paper acceptance, ensuring full reproducibility.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cr
jublic/guidest Stwpe//oSoouuuesiiiiiiiiaa wit.suubiisioooocccrc.oooo for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to to reproduce the results. See the NeurIPS code and data submission guidelines (https gewic//ggueeeeeeeeeu www.eiddeeeooeuubmmmceeee wwt.SSuummsiiioPooo.o..ooo ssionPolicy for more details.
/
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide comprehensive training and testing details in the supplementary material ensure clarity and reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: We did not report enror bars or statistical significance in the current version. However, we acknowledge the importance of reporting variability and plan to include such analysis in future revisions.
15

---

Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer:
[Yes]
Justification: We provide detailed information on the computational resources used in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported didn't Itttt make it into the paper).
paper (e.g., preliminary or failed experiments that NeurlPS Code of Ethics https://neuripps http://wwwpppppppp/cpcccccccaaaa
9. Code of ethics
Question: Does research conducted in the paper conform, in every respect, with the
Answer: [Yes]
Justification: The research presented this paper fully adheres to the NeurlPS Code of Ethics, ensuring ethical standards in all aspects of the work.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
16

---

Answer: [Yes]
Justification: The paper includes a discussion on broader impacts in the Appendix, acknowl- edging the potential positive implications of the work in applications such as virtual reality, gaming, and design. While potential negative societal impacts are not directly addressed in detail within the technical content, the impact statement clarifies that this aspect not the focus of the study.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has socictal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g.. gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: The work does not involve high-risk models or datasets. The proposed indoor scene synthesis method is unlikely to raise concerns regarding misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators original owners of assets (c.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All third-party assets used in this work, including datasets, code, and models, are properly credited. We have carefully followed the associated licenses and terms of use.
17

---

Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: Although new models and experiments are presented in the paper, do not release any new assets at the time of submission. We plan to release them upon acceptance.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/model as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: This work does not involve any crowdsourcing experiments research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
18

---

Answer: [NA]
Justification: This work does not involve any research with human subjects and thus does not require IRB equivalent ethical approval.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [Yes]
Justification: Our method uses LLM core component directly generate structured numeric indoor layouts from textual prompts. This LLM- generation process is central the overall pipeline and constitutes key technical contribution. The usage of LLMs is described in detail in the paper.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to LLM for what should or should not be described.
19

---

A Implementation Details
A.1 Prompts
In this section, we provide prompts used for training and testing the model.
CoT Data Generation. In order for the model to explicitly output logical thinking, we use the following prompt to generate CoT data for BEV Layout Generator (In ablation experiments, we use the same prompt to generate 3D layout CoT data, simply replacing the BEV layout format with the 3D layout format):
Given a BEV layout for a scene, first output a short prompt summarizing the scene, and then write a logical thought process when planning this layout, modeled after the following chain of thought example. The layout follows the style, where each line starts with the object description and is followed by its absolute position. Formally, each line should be like "object flength: ?px; width: ?pz;
center_x: ?px; center_y: ?px; orientation: ? degrees;)". You can simply interpret the length and width
the dimensions the object itself, with center_x and center_y indicating translation (or the center point of the object) and orientation indicating rotation. Note that the 0 degrees representation is aligned with the direction of the positive -alf-axis the axes where "width" and "center_y" are located. The image is (max_ (max_length)px long and (max_width)px wide. Therefore, all bounding boxes should try not to exceed 256pz after rotation.
Below are the steps the chain of thought:
1. Extract from the text description which objects should be placed in the scene and how many of each of these are specifically needed.
List the order in which place each type of object, generally starting with the large and major objects, then moving on to the decorative and minor objects associated with them.
3. Place each object the order in Each object is placed by the object, then the rotation, and finally calculating the center point coordinates based where it should be in the scene. The process should take into account the position of the object in the scene, its relative position
first giving the dimension
the placed objects, and the constraints between the objects. In this step, you need to not only place the object, but also give a detailed reason for placing it
4. Organize the that meets format requirements.
given in 3. produce final output
Here is how the json format output should look: {
"prompt": "(The prompt you generate)", "response": {
"Entity Extraction"; "(Explanation of the objects extracted from the prompt)",
"Order Decision"; "(Explanation of the order in which the objects should be placed, usually starting with major large objects
the scene)
"Spatial Reasoning": "(Explanation of the dimensions, rotation, and position (i.e. center point) of each object reasoning about each object should be detailed
20

---

possible and take into account the scene and other objects)",
Answer Organization": "(Final output the required format using lines like "object flength: ?px; width: ?px; center_x: ?px; center_y: ?px; orientation: ? ")grees;)"))
}
Layout Lifting. To complete each object's 3D pose and textual description, we use the following prompt as input to 3D Layout Generator:
Given sentence prompt that will be used generate scene and the BEV layout of this scene, lifting 2D layout to 3D layout (i.e., predicting the range of heights an object occupies in space) and designing prompt for each object for object asset generation. The generated layout should follow the CSS style, where each line starts with the object
description and folloved by its absolute position.
Formally, each line should like "object (length: ?px; width: ?px; height: ?px; center_x: ?px; center ?px; center_2: ?px; orientation: ? d" The given BEV layout contains information other than height and center_z, so the only information you need to add in the z-axis. Be careful not to let objects that don't make sense appear overlapping. For example, chairs can under tables, so they table, so they can't 't overlap. The space long. (max_width)px wide, and 160px high. Therefore, the height of the bounding box should not exceed (max_height)ps i.e. center_z +/- height/2 needs to be between 0 and { max_height).. At the objects should be one sentence describing its category, shape, and style. Finally give a list in the same order as the objects in the layout, like obji.prompt, obj2.prompt, ...].
about the objects
overlap. And lamp must go top of
(max, lengthlpx
time, for each object, prompt for natural language,
Here is how the json format output should look:
"3D_layout":
"bed flength: 88px; width: 40px; height: 36px; center x:: 120 px; center.y: 60px; center_z: 18px; orientation: d
1,
[ modern single bed with a rectangular frame and wooden headboard.",
}
Prompt:
[text_description)
BEV layout:
{bev_layout)
Quantitative Evaluator Feedback. To obtain detailed, numerical feedback, we use the following prompt as input to Spatial Evaluator:
21

---

You are an expert assistant who is well versed in indoor scene layout design. As an impartial judge, you are asked to make scene layout (BEV layout), the top
in-depth evaluation which given by the bounding box each object view and labeled with the object category near the bounding box. You can refer to the metadata of the layout to make judgment. You need to judge whether each type of object in input
scene layout each of the following four dimensions:
reasonable position on
1. Distance between objects: Evaluate whether objects are spaced appropriately to ensure functionality and accessibility. For example, in organized environments such classrooms offices, furniture should be arranged with sufficient separation to allow for easy movement. Desks in a classroon should be spaced out students can walk between them comfortably, and sofas Note that boxes may physically reasonable when objects are vertically stacked, such as ceiling lamp above a bed, or a computer placed on desk. In such cases, evaluating bounding box intersections alongside height contert is essential.
desks in offices should bounding
overlap
top-down view,
overlap
Quantitative alignment of objects: Assess whether quantities of related objects are consistent with functional expectations. For instance, in an office setting, the number of desks and chairs should correspond eaapond-each each desk should typically be paired with objects within the scene: Check whether
chair.
3. Size proportion the relative sizes of objects make sense vithin the spatial layout. For example, equipment such as computers in an office lab instruments in laboratory should be smaller than the tables or workbenches they are placed on. Unnatural size ratios, like a monitor larger than its desk, can suggest spatial implausibility.
4. Orientation of objects: Verify that objects
oriented within the layout. For
appropriately for their intended example, chairs in a classroon should face the blackboard, and computer monitors in an office should be directed toward the associated seating positions.
Please note that your choice must be based on a thorough understanding, analysis and evaluation
the image and the problem. After the explanation of each dimension, answer the final evaluation. Finally, return your judgment in legal JSON format, evaluating Yes if the location of particular object is considered reasonable in this dimension, it is not. The json format and field definitions are as follows:
No if
blbjecctccass_nanm ["Yes" or "No", "Yes" "No", "Yes" or "No "Yes" or "No"] (four judgments correspond to the previous four dimensions)
}
scene description: (scene_ descripiscri)
max_length, (max_length) px (horizontal axis) max_width: (max_width) px (vertical axis)
BEV layout: {bev_ ]
{metadata)
Spatial Evaluator Feedback. To obtain high-level, semantic feedback, we use the following prompt input Spatial Evaluator:
22

---

You are an expert assistant who is well versed in indoor scene layout design. As an impartial judge, you are asked to make the 2D
in-depth evaluation which given by the bounding box of each object in the top view and labeled with the object category near the bounding box. You can refer to both the picture and the layout metadata for judging. You need to judge whether the the input BEV scene
layout (BEV layout),
information about each kind object layout is reasonable in each of the following three dimensions:
1. Spatial alignment between objects: For example, the podium and projector in classroom should be centrally aligned either horizontally or vertically, and lockers in locker rooms should be arranged in an orderly grid.
2. Position of the objects within the layout: Consider whether each object's 's position is appropriate for the overall layout context. For example, near the center, and in a bank the counter should near a wall.
single office the desk should be
3. Consistency with Chain -f-ouhghht Descriptions: The physical objects should align with any provided textual chain of thoughh descriptions. For example, the chain of thought mentions placing the whiteboard next to the wall, but in the picture and digital layout the whiteboard is in the middle of the scene, which is unreasonable.
placements
Please note that your choices must be based on a thorough understanding, analysis, and evaluation of the image and problem. After the explanation of each dimension, answer the final evaluation. Return your judgment dimension
the end of each legal JSON format, evaluating Yes if the placement of a particular object is considered reasonable in that dimension, and No if not. The json format and field definitions
follows:
{
"object_class_name ["Yes" or "No", "Yes" " or "No", "Yes" or "No "] (three judgmenta correspond to the previous three dimensions)
}
scene description: srcene_descrpption
max_length, length) px (horizontal axis in BEV)
max_vidth: (max_vidth} px (vertical axis in BEV)
BEV layout:
{bev_layout)
chain of thought: (CoT}
The above two prompts used to provide input the evaluators during training, in order compute CoT-Grounded Generative Layout Reward. During inference, we use similar prompts as input to the evaluators to perform Iterative Asset-Layout Alignment, with the only change being that the output format is adapted to provide object-level suggestions (while keeping the evaluation criteria unchanged).
Scene Description Generation Prompt. construct diverse training and evaluation data, we use the following prompt to instruct language model to generate indoor scene descriptions at three levels of granularity across multiple scene categories:
23

---

You are asked to generate indoor scene descriptions at three levels of gramularity: coarse, medium, and fine- ane-ggrined. Please follow the instructions carefully.
There are three types of granularity: 1. Coarse: List the main objects in the room without mentioning where they are.
Example: "A home gym with treadmill, yoga mat, dumbbell rack,
dispenser, and large mirror." 2. Medium: Describe the approximate spatial relationships between major object groups.
Example: "In playroom,
toy shelf stands against the right wall, bean bag lies in the a round play mat is placed in the center."
near the window, and
3. Fine Line-gran graingraied:: Provide precise, detailed spatial arrangements among individual objects.
Example: "A small square table is placed in the center of the room. On the front right corner of the table sits red toolbox, with measuring tape coiled beside it. yellow stool is tucked in on the left side, and a desk lamp stands at the rear center of the table."
You will generate scene descriptions for (run_scene_types: different indoor scene categories, excluding common
categories such as bedroom, living room, dining room, and study.
For each scene category, generate:
descriptions at coarse granularity, _er_ttpe] at medium granularity, (num_fine_per__ypp at fine-grained granularity.
Each scene description should be accompanied by dimensions: length, width, height (each must be an integer 256).
Output the results in a strict json format as follous:
"scene_ type:: "laundry room", "granularity": "coarse", "description": "A laundry room with a washing machine, dryer laundry basket, shelves, and detergent bottles.", "room_size";
"length": 256,
width": 171,
"height": 240
},
]
Simple Reward. To verify the effectiveness of CoT-Grounded Generative Layout Reward, we use GPT-4o to give overall score for BEV layout.
You are an expert assistant who is well versed in indoor scene layout design. As an impartial judge, you are asked to make in-depth evaluation of the 2D scene layout (BEV layout), which given by the bounding box of each object in the top view and labeled with the object category near the bounding box. You can refer to both the picture and the layout metadata for judging. Please score the scene based
24

---

physical plausibility, semantic consistency, and degree of instruction compliance.
Return your answer in legal JSON format. The format and field definitions are as follous:
{
"score": 1-100
}
scene description: escene_description.
max_lengthn (max_length) (horizontal axis in BEV)
max_width: (max_uidth) px (vertical axis in BEV)
BEV layout:
{bev_layout)
chain of thought: {CoT}
A.2 Training Details
We adopt two-stage training pipeline fine-tune the Qwen-2.5-32B. In the first stage, SFT is conducted on a generated CoT-augmented dataset containing approximately 6,500 CoT data of BEV layout. -ow-annk Adaptation (LoRA) with rank of is applied to all target modules. The model is trained for 3 epochs using a batch size of and a gradient accumulation step of 8, with a learning rate of 10 44 A cosine learning rate scheduler with 10% warm-up is employed, and training performed using bfloat precision. The SFT stage is conducted on a single NVIDIA A800 GPU and takes approximately 8 hours to complete.
In second stage, DPO employed to further enhance spatial reasoning CoT-Grounded Generative Layout Reward. The DPO dataset contains around 12,000 preference pairs, sampled such that the reward difference between the preferred and rejected generations exceeds 20. The same LoRA configuration and optimization hyperparameters
used in the SFT stage. The preference loss function is set to sigmoid with 3 = 0.1. This stage is trained on NVIDIA A800 GPUs for approximately 11 hours.
A.3 Indoor Scene Description Generation Details
To support both training and evaluation, we generate a diverse set of textual indoor scene descriptions three levels granularity using GPT-4o following the prompt described in Appendix A..1 These levels are designed to test and guide the model's ability to interpret spatial semantics:
Coarse: Lists the key objects in the scene without specifying spatial relationships. Example: "A kitchen with refrigerator, stove, sink, countertop, dining table, and several cabinets, along with few bar stools."
2. Medium: Describes basic spatial layout and object grouping.
Example: "In a home theater, large screen is positioned against a wall, flanked by two speakers. In front of it. round coffee table holds popcorn and potato chips. A sofa sits behind the coffee table, and several bean bags are arranged around it."
Fine-grained:
Specifies detailed relative positions among objects.
Example: "A wooden table holds laptop on the front left corner, with glass of colored pencils placed to its right. At the center back of the table sits a blue vase filled with flowers, and in front of the vase lie pair of frammed glasses. On the left side of the table is white coffee cup, while a wooden chair is positioned in front of the table."
Training, For DPO training, we generate 200 scene descriptions across 40 indoor scene categories that are not present the 3D-Front dataset (e-g., excluding bedrooms, living rooms, dining rooms, and studies). Each category includes 5 descriptions and room sizes, with the three granularity levels distributed in 2:2:1 ratio (coarse:mediunffin -in-ggginddd). These descriptions are directly used as prompts to generate layout samples for DPO training.
25

---

Evaluation. For evaluation, we construct separate set of 45 test cases across 15 indoor scene categories, also distinct from those 3D-Front. Each category associated with three manually crafted and verified descriptions (one per granularity level), along with specific room size. All descriptions are manually reviewed ensure logical spatial consistency. Minor corrections are made where necessary- for example, adding missing chairs in office scenes to ensure realistic and challenging testbed for layout generation.
B More Generated Results
In Figure S we present additional qualitative results a variety of scene types, including garages, classrooms, bathrooms, and wine cellars. Our method consistently achieves strong physical plausibil- ity and semantic alignment with language instructions, allowing users to exert fine-grained control over the generated content.
A child's bathroom with a low toilet, step a bathtub an arrangement of student desks ducks, colorful and small sink
classrcom is furnished with front, facing
A fectures large centrally located, sterage shelves lining the back wall. A are
wine cellar with storage and tasting depicted. Along two opposing walls, extensive rocks wine bottles. On another wall, collection of ook barrels is stocked, with wooden crcte. A accompanied single choir, centrally located within the On table, there are two wine glasses and corkscrew temperature gouge mounted the wall above the onk barrels, and two circulor lighting fixtures on celling
workbench positioned beneath pegboard. while bicycle lownnower arranged nearby
on the front wall, and projector hangs from the ceiling bookshelf also stonds the side
Figure 5: More generated results based on language instructions with different granularities.
C Illustration of Iterative Asset-Layout Alignment
Figure 6 provides a visual illustration of Iterative Asset- Asset-Layout Alignment process described in Section 3.5 This figure highlights the feedback loop between scene generation and evaluation, where each rendered 3D scene undergoes spatial and semantic evaluation to identify inconsistencies. Based on the evaluator's feedback, targeted corrections are applied to the layout, including object size, position, and orientation. The updated layout is then used to regenerate the scene. This iterative refinement continues until the evaluators no longer detect violations or preset iteration limit reached.
26

---

diilisttedd be roder with white cerani freds
ths o reand body
persalavani
white buttnd Alt dinechogyaar shops
thep stod with sppore
Object Generation
make the tub taller
lower rubber duck
Spatial Evaluator
Quantitative Evaluctor
BEV Layout Generator
3D Layout Generator
Figure 6: Illustration of Iterative Asset-Layout Alignment
D Inference Time
We report inference time under consistent hardware and network conditions, measured from text input to 3D layout generation. Table summarizes results 30 samples.
Table 4: Inference time comparison across methods (30 samples).
Method
Average (s)
Min (s)
Max (s)
LayoutGPT
5.87
5.32
6.67
IDesign
166,86
148.12
229.74
Holodeck
210.23
172.59
291.88
Ours
185.24
109.45
298.13
Our approach is competitive with existing baselines. A large portion of the runtime (~~66 seconds per object) comes from the Trellis object generation module, which is modular and be replaced with object retrieval. As object generation models continue to improve, this component is expected to become significantly faster.
E
Impact Statement
This paper focuses on the technical udvancements in realistic 3D indoor scene synthesis. The work aims to enhance applications virtual reality, gaming, and design, which could have positive societal implications in these domains. However, this study does not directly address potential societal impacts, including possible negative consequences such as malicious or unintended uses (e.g., generating fake scenes), fairness considerations, privacy concerns, security risks that might arise from the application of this technology. The paper primarily presents technical research and does not discuss the deployment of the technology or potential mitigation strategies for negative impacts.
27


Reviews:
[{"review_id": "vdROGMkGl6", "reviewer": "Reviewer_y4q3", "summary": "The authors propose a new system for open-vocab, languaged-conditioned, indoor layout synthesis. The system uses LLMs/VLMs to (1) iterate on a language specification of a birdseye view floorplan, then (2) convert 2d plans to likely 3d object heights/specs and finally (3) refining object locations. This is comprised of structured prompts for eliciting CoT, describing BEV layouts, and LLM-reward for DPO finetuning. The provided qualitative results are strong, and the method's results are preferred in LLM-based quantitative evaluations.", "questions": "How were the prompts of Figure 3 selected? Are the results shown random samples (e.g. with a pre-determined seed = 0 for each method, best-of-1)? Are the Fig3 results representative in quality of what I might see if I ran your method for seeds 0..5 on identical prompts as prior work? (e.g. the starwars office prompt of Holodeck Fig 1)\n\nHow can we be assured of the usefulness of larger samples of scenes for downstream use (by humans, in simulators or other graphics) ? Please comment on the validity of quantitative results (see above). I estimate it will difficult to address without a tightly controlled user study (randomized, >7 participants, well crafted survey) which is beyond the scope of a rebuttal.\n\nWhat is the mean/min/max CPU runtime, API credit usage, and/or GPU of the method under various settings? Is this new cost-performance tradeoff appealing for downstream users?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "qualitative results are strong but not plentiful or controlled:\n- strength: the model produces very good layouts with good instruction following, especially in the kitchen and home-theater examples - these feature objects not attached to walls which are minimally constrained and difficult to place correctly.\n- weakness: needs larger & tightly controlled qualitative results\n\t- only one example is shown for each granularity of prompt. we do not know the variance / stability of the method when repeatedly sampled with the same prompt. \n\t- not stated how the prompts were selected - are they representative/random/standardized, or designed to showcase the strength of the authors method?\n\t- ATISS & Infinigen provide large samples of 25+ _randomly selected_ qualitative results in their supplements, this work does not.\n\nquantitative results have two major flaws:\n- Authors follow LayoutGPT in using metrics which are _evaluated by GPT4o_ (with no human study) which I believe is a major flaw in evaluation of this work.\n\t- using GPT-4o as the validation score, when iterating against GPT-4o's criticisms is an integral part of the method, is invalid. This is akin to optimizing against the validation set - if we can see (somewhat stochastic samples of) the final validation metric and change our answer, it is no longer a validation metric.\n\t- I recognize LLMs-as-judges is a widely used paradigm in NLP, but I believe it is invalid for layout as VLMs are not known to possess strong spatial reasoning without significant scaffolding / extra engineering (as evidenced by goals of this work)\n\t- Every baseline method in table 1 provides a user study (Holodeck, I-Design, LayoutGPT). The authors do not perform a user study, they instead cite that LLMs have correlated with users in the past. But this is questionable since the authors method is different & more LLM/VLM reliant than prior methods.\n\t- Out-of-bounds/Collision metrics are quantitative but not sufficiently informative. Objects could topple when settled in simulation, or have their affordances violated (objects blocking chairs/ovens) so these are useful to include but not sufficient as a sole evaluation.\n- Table 1 omits comparison with ATISS, which is well known and which I consider closely related in method. Less critically, the authors could also evaluate DiffuScene. \n\nDiscussion of the method's cost is inadequate. \n- only mention is \"introduce additional inference time\" L315\n- The authors should know the average spend on OpenAI APIs per scene, but the cost is not stated.\n- Tab1 should list costs of each method. Fig.3 could report per-scene costs (is the cost higher for more complex scenes? this would be interesting). See questions below. \n- If the cost is high, is the method better than baselines on an equal-cost basis (baselines run for longer, or authors method is allowed fewer iterations?)\n- The authors fail to argue that the performance-cost tradeoff of adopting their method is worthwhile for downstream tasks. Would roboticist use the method if scalability costs $X/scene in API credits/GPU-time? We cannot know as the cost is not stated.\n- Increased cost would be a predictable downside of their method of replacing constraint solvers with LLM reasoning, which is the core proposal. One gains flexiblity / open-vocab, but loses the computational efficiency of a constraint solver.\n\nDistinction from prior work, especially related to \"constraints\", is imprecise:\n- \"mainly focus on modelling the distribution of indoor scene layouts, wheras our method leverages limited data to learn the underlying placement logic of indoor layouts\" (L85) - what is the difference getwen these two? is \"learning the logic from data\" not an example of learning the distribution?\n- \"explicit logical outputs\" L101 instead of constraints - the methods outputs (L155) are very similar to constraints but are \"solved\" by an LLM\n- \"predefined constraints\"... \"inherently sacrifice flexibility\" L36 - not necessarily - if the predefined DSL is sufficiently low-level (has primitives that are universal to all scenes, not specific to objects), they may apply to many/all scenes\n\nThe authors presuppose many ambitious properties of LLMs/VLMs e.g \" general reasoning capabilites\" (L88). these are still open areas of research and evaluation - LLM reasoning can fail on examples that are trivial for humans. \n\nQuality Justification (technically sound?): lacks valid evaluation results, lacks sufficient qualitative results, lacks any discussion of cost. \n\nClarity Justification (clearly written?) 3. fairly clear, especially once one reads the prompts in the supplement. short excerpts of these could be moved to the main paper. \n\nSignificance Justification (impactful for community? likely to build on ideas?): Generating useful simulation environments for robotics is potentially very significant. But it is impossible to predict adoption given unknown cost.\n\nOriginality (new insights?):  CoT or DPO are now standard practice in using LLMs, I do not find the nature of their use in this paper insightful. The novelty of the work is a set of structured prompts for layout specification, eliciting standard LLM chain-of-thoughts, and/or rewards. LLMs with structured outputs, that are prompted to perform CoT, and/or rewards from another LLM prompt, are prevalent.", "was_revised": true, "final_justification": "My original rating was based primarily on concerns or misunderstandings on evaluation, which the authors have fully addressed in rebuttal. The authors' use of VLM-graded perceptual studies is in line with prior work, they added a user study, additional baseline systems, and provided the requested runtime / cost averages.\n\nI share other reviewers' view that the method is not hugely insightful or technically interesting. The authors state their novelty lies in \"the new perspective [CoT/DPO] enable for 3D scene layout generation\", however the transition into less structured and lower level language primitives strikes me as more gradual over the course of multiple prior works e.g. Holodeck. The output here remains somewhat structured, and feedback still comes from a geometric checker.  \n\nHowever, my explicit concerns have been largely addressed, and the work's favourable evaluation results mean it may be useful for directly generating 3D scenes for robotics, and as an exemplar of maximally-LLM-based arrangement as a comparison for future work.", "rebuttal": "Thank you sincerely for your valuable time and insightful comments.\n\n### **W1 & Q1: Qualitative results**\n\nWe provide multiple examples per granularity in the main paper and supplementary materials, though only one visualization per prompt is shown. Due to rebuttal formatting limits, more visuals cannot be included here. Instead, we present a quantitative study measuring the standard deviation of our model over ten repeated generations using the same prompt or scene type. Each table row shows results from one randomly sampled prompt under the given method.\n\n| Model    |   Pos |   Rot |   PSA |\n|:---------|----------:|----------:|----------:|\n| Ours (fine)     |      5.83 |      5.48 |      6.78 |\n| Ours (medium)   |      4    |      6.78 |      6.32 |\n| Ours (coarse)   |      5.83 |      2.93 |      4    |\n| ATISS    |      2    |      4.9  |      7.8  |\n| Holodeck |      4    |      7.28 |      6.75 |\n\nAs noted in the Appendix A.3, we generated 45 scenes per granularity. For I-Design that sometimes fails to produce valid results, we exclude those cases. Prompts in Figure 3 are randomly chosen from successful generations. We include 10+ randomly selected qualitative results in the paper and supplement, and will add more examples in a revision.\n\n### **W2 & Q2: GPT-4o based evaluation is flawed; no human study**\n\nWe adopt the GPT-based metric from LayoutVLM (CVPR 2025), which has been validated to align well with human judgments. While VLMs have limits in fine-grained spatial reasoning, we distinguish between generative spatial reasoning (precise 3D placement) and visual content understanding. The former requires exact spatial computation, the latter image perception. Therefore, properly calibrated VLM-based evaluation like LayoutVLM remains a valuable reference for layout quality.\n\nIn our pipeline, this metric is used only for evaluation, avoiding overfitting since training rewards come from detecting objective errors (e.g., collisions, CoT mismatch), which differ from the evaluation metric. We follow prior works like LayoutGPT and LayoutVLM using these to provide extra information. To further validate, we conduct a user study with 25 compensated volunteers experienced in computer science and 3D vision, who score layouts from various methods on three criteria: Semantic Compliance (alignment with textual description), Layout Rationality (plausibility of object placement), Physical Plausibility (realism of object support, no penetration, etc.).\n\nEach method’s results were randomly ordered and anonymized. The average scores are reported below:\n\n| Method      | Level   | SemComp | PhyPlaus | LayRatl |\n|-------------|---------|---------|----------|---------|\n| LayoutGPT   | coarse  | 3.72    | 1.84     | 1.72    |\n|             | medium  | 2.88    | 1.64     | 1.76    |\n|             | fine    | 3.00    | 2.32     | 2.04    |\n|             | overall | 3.20    | 1.93     | 1.84    |\n| I-Design    | coarse  | 3.72    | 2.72     | 2.44    |\n|             | medium  | 3.08    | 3.36     | 2.48    |\n|             | fine    | 3.32    | 3.72     | 2.72    |\n|             | overall | 3.37    | 3.27     | 2.55    |\n| Holodeck    | coarse  | 4.12    | 3.64     | 2.84    |\n|             | medium  | 2.80    | 2.52     | 2.48    |\n|             | fine    | 2.40    | 2.60     | 2.04    |\n|             | overall | 3.11    | 2.92     | 2.45    |\n| Ours        | coarse  | 4.56    | 4.64     | 4.52    |\n|             | medium  | 4.60    | 4.68     | 4.64    |\n|             | fine    | 4.84    | 4.72     | 4.80    |\n|             | overall | 4.67    | 4.68     | 4.65    |\n\nWe also include comparison with ATISS, DiffuScene, and Infinigen:\n\n| Method     | PhyPlaus |  LayRatl |\n|:-----------|----------|----------|\n| ATISS      | 3.66     | 3.54     |\n| Infinigen  | 4.66     | 4.44     |\n| Diffuscene | 3.46     | 3.1      |\n| Ours       | 4.62     | 4.48     |\n\nWhile they perform strongly overall, our method matches or surpasses their performance on certain scene categories, and additionally supports open-vocabulary generation.\n\n### **W3: Table 1 omits ATISS and DiffuScene**\n\nATISS and DiffuScene are not directly comparable to our method due to fundamental differences in input and scope: 1. Our method supports open-vocabulary text inputs, while ATISS and DiffuScene are trained to generate specific scene types (e.g., bedrooms or living rooms). 2. Our model is LLM-based, while theirs are layout diffusion or scene synthesis networks that do not mainly rely on natural language inputs.\n\nDespite this, to facilitate a fairer comparison, we randomly generate 20 samples for bedrooms and living rooms and evaluated across standard metrics:\n\n| Method     | Scene      |   Pos |   Rot |   PSA |\n|:-----------|:-----------|------:|------:|------:|\n| ATISS      | bedroom    | 81    | 70    |    86 |\n|            | livingroom | 71.5  | 79.5  |    76 |\n|            | overall    | 76.25 | 74.75 |    81 |\n| Diffuscene | bedroom    | 66    | 63    |    70 |\n|            | livingroom | 66.5  | 66    |    60 |\n|            | overall    | 66.25 | 64.5  |    65 |\n| Ours       | bedroom    | 82.5  | 75    |    96 |\n|            | livingroom | 86    | 75.4  |    80 |\n|            | overall    | 84.25 | 75.2  |    88 |\n\n\n### **W4 & Q3: Method cost**\nWe discuss the cost here and will add more discussions in the revised paper.\n\nMonetary cost:\nWe don’t report per-scene monetary costs due to inconsistent baselines using different models (e.g., GPT-3.5, GPT-4, GPT-4o) with varying API pricing, which makes fair comparison difficult. To ensure transparency, we release a fully open-source pipeline that can run locally or on open LLMs, where cost-free variants outperform original baselines on layout quality. If needed, we can run experiments with a unified model (e.g., GPT-4o) to report average per-scene costs.\n\nInference time cost:\nWe evaluate inference time systematically by measuring average, minimum, and maximum times for generating layouts from 10 random prompts at three granularity levels, all on the same platform and network. Except for LayoutGPT, which is limited to specific scene types, our method achieves better results with comparable inference times (Unit: seconds):\n\n| Method                      |   Average |   Min |   Max |\n|:----------------------------|---------------------------:|-----------------------:|-----------------------:|\n| LayoutGPT                   |                       5.87 |                   5.32 |                   6.67 |\n| IDesign (~60% failure rate) |                     166.86 |                 148.12 |                 229.74 |\n| Holodeck                    |                     210.23 |                 172.59 |                 291.88 |\n| Ours                        |                     185.24 |                 109.45 |                 298.13 |\n\n\n### **W5: Ambiguity in constraint-related distinction**\nWe will clarify the details in the revised version.\n\nPrior learning-based methods directly learn distributions of numerical layouts. Our method instead learns the logic behind object placement, which generalizes better across unseen scenes. For example, ATISS, LayoutGPT has difficulty generating scenes with open vocabulary.\n\nPredefined constraints can theoretically generalize, but manually defining them is costly and often fails across diverse scene types. Prior work mainly uses coarse spatial relations like relative directions or distances. While LLMs handle such constraints more easily than generating exact layouts, this limits realism and diversity and struggles with fine-grained descriptions. For example, “the cake is placed at the top-left corner of the table” cannot be captured by simple rules like “A is left/right/above/below B” or “A is X meters from B.” In our method, such detailed spatial relations are treated as constraints generated by the LLM. User study show clear advantages in handling these fine-grained descriptions.\n\n### **W6: Originality**\n\nOur novelty lies not in the use of CoT or DPO themselves, but in the new perspective they enable for 3D scene layout generation. Prior methods (e.g., ATISS) rely on fully supervised training over limited datasets, learning fixed spatial distributions that struggle to generalize to new categories or diverse scenes. Recent efforts (e.g., Holodeck, I-Design) have begun using LLMs to bridge this gap, but these typically depend on manually defined spatial constraints and external solvers, limiting flexibility and fine-grained spatial control. A common consensus regarding this type of work is that LLM finds it difficult to generate precise numerical layouts. In contrast, our method uses CoT and DPO to enhance generalizable reasoning, enabling LLMs to generate complete numerical layouts directly and opening up new ways of thinking.\n\nWe also introduce an entropy weight–based reward integration scheme to balance multiple layout quality aspects. While prior LLM research mainly targets math or coding, 3D spatial reasoning offers a valuable testbed. We provide a feasible method for LLM to handle generative 3D spatial reasoning problems. We will open source the CoT dataset and code to support further progress in this area.\n\n### **W7: Ambitious Properties**\n\nWe apologize for the wording that may have caused misunderstanding. While LLMs are not flawless, our work only leverages their commonsense reasoning for spatial relation generation, which is a narrower and more stable use case supported by prior work and empirical results. We will use more precise wording to reduce misunderstandings.\n\nWe hope our responses can address your concerns. Please feel free to inform us of any further issues."}, {"review_id": "NJ6ef27YWF", "reviewer": "Reviewer_A1tB", "summary": "This paper studies 3D layout generation. The proposed pipeline follows the widely used LLM-agent pipeline, trained SFT using GPT-generated CoT data, and then standard DPO. The contributions of this paper mainly lie in the engineering tricks within this pipeline, including the specific reward designed for the alignment between CoT and the generated layout and the final interactive asset-layout alignment for refinement. Both qualitative and quantitative measurements show that the proposed method is effective.", "questions": "- I wonder if GPT-4o can generate reasonable CoTs, instead of just generating perfunctory analysis to overfit the answer. It would be beneficial to provide examples of GPT-generated CoTs.\n- The proposed method leverages the visual grounding of a trained model to generate a layout, which reasons on top of the coordinates. I would like to see the raw outputs of the planner, to see how it reasons with the coordinates, and how precise they are.\n- Please provide more explanations about the seven terms in \"Object-Level Criteria\".\n- If the author(s) disagree with the reviewer's claim that \"the novelty of the paper is engineering tricks on top of a widely adopted framework\", please argue the novelties that can be regarded as the novelty compared with the whole LLM agent field to improve general LLM agent designs, instead of the novelties in the layout generation task.", "limitations": "The authors mentioned the limitations. Some others mentioned in \"Weaknesses\" about the lack of 3D z-axis capability.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 1, "significance": 3, "originality": 2, "strengths_and_weaknesses": "### Strengths\n- The high-level ideas of the method are straightforward to understand. The engineering tricks are relatively well-motivated.\n- All the visualizations in this paper are impressive, showing that the generated layout is reasonable (while also thanks to the out-of-the-box object generator).\n- Both quantitative and qualitative scores show that the proposed method is effective and outperforms state-of-the-art.\n- Specifically, the experiment part considers the use of different LLMs and tests the proposed method in different settings, e.g., open-source only, to make a fair and holistic comparison.\n\n### Weaknesses\n- As the paper employs the widely adopted LLM agent pipeline, with CoT as also a widely adopted extension, even if it might be the first to employ it in layout generation, transferring such a well-studied pipeline to the new task should not be solely regarded as a contribution/novelty.\n- **Crucial.** The method section is not well-written. Based on such a widely used framework, the actual novelties of this paper are engineering tricks. This makes the paper more like a technical report, revealing numerous details; however, on the other hand, these details are still mainly text descriptions with uninformative formulas, which are still far from being reproduced. \n    - For example, Sec. 3.1.2's \"Object-Level Criteria\" introduces the reward function with seven terms. This is very detailed, but okay, as they are related to the main contribution. \n    - However, none of these terms are defined rigidly or explained in detail - simply mentioning \"Measures the reasonableness of spatial relations with related objects\" or \"Checks that spacing between objects supports accessibility and func162 tionality. Overlaps may be acceptable if vertical stacking is plausible\" is far from sufficient, which cannot tell how they are defined and distinguished.\n    - All the formulas (3)-(7) are just uninformative and simple rephrases of texts. For example, (3) is just a simple yet space-consuming explanation of \"the proportion of objects in the layout that satisfy the corresponding requirement\" (L169). All those formulas are just placeholders to satisfy \"there should be a formula\".  \n    - All these make the paper hard to reproduce, even if the engineering tricks are the focus of the paper.\n- Like all LLM agent methods, the inference requires a full round between the LLM planner and the layout generator, which can be time-consuming. Specifically, the proposed method involves an object generator for per-object generation, which further increases the time consumption.\n- As using BEV, the proposed method may not be able to generate complicated z-axis, e.g., a large wardrobe hanging above a table with several fruits on it, under which there is a chair. As shown in the supplementary video \"bathroom_wo_refine\", the model seems to be confused about the height location of the object. This hurts the scope of the proposed method, raising concerns of whether the proposed method is still only generating 2D/pseudo-3D layouts instead of real 3D.", "was_revised": true, "final_justification": "I would like to maintain the current score, as the current paper has significant issues in both writing and reliability of LLM/VLM-based evaluation, and concerns about model capability.", "rebuttal": "Thank you sincerely for your valuable time and insightful comments.\n\n### **W1 & Q4: Novelty of using LLM agent pipeline and CoT**\n\nOur primary contribution and key insight lie in presenting a new perspective specifically for the 3D scene synthesis domain. Prior approaches (e.g., ATISS) mainly rely on fully supervised training over limited datasets, which learn fixed spatial distributions and thus struggle to generalize to new object categories or diverse scenes. Recent efforts (e.g., Holodeck, I-Design) have begun leveraging large language models (LLMs) to address this gap, but these methods often depend on manually defined spatial constraints and external solvers, which limit their flexibility and ability to handle fine-grained spatial relations. It is widely recognized that LLMs find it challenging to directly generate precise numerical layouts. In contrast, our approach integrates CoT reasoning with DPO to significantly enhance the LLM’s generalizable spatial reasoning capabilities, enabling it to produce complete numerical 3D layouts directly. This opens new avenues for thinking about generative 3D scene layout design.\n\nMoreover, for LLM agent, we use an entropy-weighted reward integration mechanism that dynamically balances multiple layout quality criteria. While most existing LLM research focuses on domains like mathematics or programming, 3D spatial reasoning is an important and challenging application scenario. Our work demonstrates a practical approach for enabling LLMs to tackle generative 3D spatial reasoning tasks. To facilitate further research, we plan to release the CoT dataset and source code upon paper acceptance.\n\n### **W2 & Q3: Method section clarity and reproducibility**\n\nWe acknowledge that the method section can be improved for clarity. We sincerely appreciate the valuable feedback and will incorporate these improvements in the subsequent version.\n\n\n(1) **On evaluation criteria definitions**:\n  The spatial and functional “criteria” mentioned (e.g., “reasonableness of spatial relations”, “accessibility and functionality”) serve as guiding directions for evaluators rather than rigidly   defined quantitative metrics. These criteria are clarified with concrete prompt examples in Appendix A.1. We also provide more detailed explanations:\n\nRelative Alignment (C1)\nThis criterion assesses the geometric congruence and spatial relationship between related objects. It evaluates whether objects that are functionally or conceptually linked exhibit appropriate alignment, such as central, horizontal, or vertical congruity, or organized arrangements like grids, based on their intended spatial interaction.\n\nGlobal Positioning (C2)\nThis criterion evaluates the semantic appropriateness of an object's placement within the overarching room context. It assesses whether the object's position aligns with conventional spatial norms and functional requirements relative to the entire layout, ensuring it contributes to a coherent and functional environment.\n\nConsistency with CoT (C3)\nThis criterion verifies the fidelity of object placements against any explicit or inferred spatial instructions provided within a Chain-of-Thought (CoT) description. It ensures that the physical arrangement of objects in the scene directly corresponds to the textual guidance, identifying discrepancies where the visual representation deviates from the described intent.\n\nInter-object Distance (C4)\nThis criterion measures the functional spacing between distinct objects, ensuring adequate separation for accessibility, navigability, and intended use. While minimal bounding box overlaps may be permissible in instances of plausible vertical stacking (e.g., an object placed on another), the primary assessment focuses on maintaining sufficient clearance for unimpeded interaction and movement.\n\nTo manage space in this rebuttal, we provide detailed explanations for four key criteria (C1–C4). If you’d like, we’d be happy to provide the full descriptions of all seven criteria in a follow-up response or the revised version.\n\n(2) **On formulas (3)-(7)**:\n  These formulas are intended to clearly explain the detailed process of reward computation, aiming to enhance the clarity and reproducibility of the paper. However, we acknowledge that some formulas overlap with the textual descriptions. For example, Equation (3) can be more concisely described in words, and Equation (5) corresponds to entropy calculation, which can also be expressed textually. On the other hand, Equation (4) is difficult to describe clearly using only words, while Equations (6) and (7) could be combined but represent a complex weighting calculation that cannot be fully captured in simple language. We will streamline the paper by removing redundant overlaps between text and formulas, thereby improving the clarity and conciseness of the presentation.\n\n(3) On reproducibility:\n  All prompts and detailed experimental parameters are provided in the appendix. We will open source code and data upon acceptance to ensure full reproducibility.\n\n### **W3: Inference time concerns**\n\nOur inference time, measured under consistent hardware and network conditions, covers the full pipeline from text input to 3D layout generation. As shown in the table (30 samples), our method performs competitively with baselines (Unit: seconds). Object generation (Trellis) takes ~26 seconds per object, but it's modular and can be replaced with object retrieval. As generation tools improve, this time is expected to drop. We don't consider this to be a fundamental bottleneck of our method.\n\n| Method                      |   Average |   Min |   Max |\n|:----------------------------|---------------------------:|-----------------------:|-----------------------:|\n| LayoutGPT                   |                       5.87 |                   5.32 |                   6.67 |\n| IDesign (~60% failure rate) |                     166.86 |                 148.12 |                 229.74 |\n| Holodeck                    |                     210.23 |                 172.59 |                 291.88 |\n| Ours                        |                     185.24 |                 109.45 |                 298.13 |\n\n\n### **W4: Handling of z-axis and true 3D layout**\n\nWhile BEV layouts represent objects as bounding boxes in the XY-plane, our framework allows overlapping bounding boxes to reflect vertical stacking. We leverage LLMs’ strong priors on object height ranges and relationships, enabling accurate height estimation despite the 2D base layout. As shown in Table 2, decomposing the problem (BEV layout + height reasoning) outperforms direct 3D generation attempts. \n\nThe issue in the bathroom_wo_refine example is not a general failure in height prediction, but a rare case of incorrect object perception, the model initially assumed the bathtub was filled with water, leading it to place the rubber duck on top. To address such cases, we introduce an Iterative Asset-Layout Alignment module that refines the layout based on feedback from generated assets, successfully correcting the error in later iterations. This is the result of the ablation experiment (not our full model) and our framework is specifically designed to handle this. Therefore, this case should not be considered a main weakness of our method, nor does the ablation visualization reflect a fundamental flaw.\n\n\n### **Q1: CoT Reasoning Quality**\nTo demonstrate the quality of reasoning, we provide below an example of a GPT-4o–generated CoT  (only a partial example shown due to space constraints):\n```\n\"Entity Extraction\": \"The scene contains: 2 multi-seat sofas, 1 armchair, 1 coffee table, 2 console tables, 1 cabinet, and 2 stools. These pieces suggest a living room setup focused on seating and moderate storage/display furniture.\",\n\n\"Order Decision\": \"First, we place the two multi-seat sofas since they are the largest and central to the scene. Next, we position the coffee table in relation to the sofas to form a functional conversation area...\",\n\n\"Spatial Reasoning\": \"The layout starts by defining the primary seating area. A multi-seat sofa is placed along the right side of the room, oriented vertically to establish one edge of the seating space. Its center is set around (162px, 176px), with dimensions of 60px in length and 25px in width, rotated -90 degrees. To complement this, a second multi-seat sofa is positioned horizontally below, forming an L-shaped arrangement. It is centered at (110px, 133px), maintaining a 0-degree orientation, creating a natural corner ideal for conversation...\"\n```\nThis example shows that GPT-4o produces structured and context-aware reasoning. The CoT is not superficial, it breaks down the task and considers spatial relationships rather than overfitting the final answer.\n\n### **Q2: Raw Planner Output**\nBelow is an example of raw planner output, showcasing its multi-step reasoning process (only a partial example shown due to space constraints):\n```\n\"Entity Extraction\": \"The objects that should be placed in the scene include a multi-seat sofa, a lounge chair, a coffee table, a TV stand, a dining table, multiple dining chairs, and pendant lamps.\"\n\n\"Order Decision\": \"The order of placement should start with the largest and most central objects, which are the multi-seat sofa and the dining table, as they likely serve as the focal points of the living and dining areas...\"\n\n\"Spatial Reasoning\": \"To begin shaping the room, a multi-seat sofa is positioned as the anchor of the seating area. It’s placed along the right side of the space, rotated vertically to make efficient use of the wall without cutting into circulation paths. With a generous width and length—measuring 79px by 34px—it defines the main visual boundary of the living zone. The center is set at (194px, 65px), maintaining openness in the central floor area...\"\n```\n\nWe hope our responses can address your concerns. Please feel free to inform us of any further issues."}, {"review_id": "hi821rVgf7", "reviewer": "Reviewer_QFty", "summary": "The paper proposes DirectLayout, a framework that generates 3D indoor layouts directly from textual descriptions. Unlike existing methods that either regress numerical layouts from data (with poor generalization) or use intermediate symbolic representations (with rigidity and poor fine-grained control), DirectLayout introduces a three-stage pipeline:\n\n1. BEV layout generation, enhanced with Chain-of-Thought (CoT) reasoning.\n2. 3D lifting using commonsense priors.\n3. Iterative asset-layout alignment using feedback from dual evaluators (LLM + VLM).", "questions": "How sensitive is the performance of your BEV Layout Generator to the quality of the GPT-4o-generated CoT traces? Did you experiment with noisy or incorrect CoT traces to evaluate robustness?\nYour CoT-Grounded Layout Reward combines both VLM and LLM feedback. How critical is this dual-evaluator setup?\nHow well does DirectLayout generalize to unusual room types or unseen object categories not represented in the 3D-Front dataset or training prompts?\nWhat are the most common failure modes you’ve observed?", "limitations": "Yes. But there are more limitations, especially on the usage of BEV as the intermediate representation.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "The paper clearly defines the limitations of existing paradigms and motivates its simple yet effective algorithm based on Chain-of-Thought (CoT) reasoning and numeric generation. The model outperforms LayoutGPT, Holodeck, and I-Design across nearly all metrics.\n\nThe main weakness of the paper is the lack of evaluation on out-of-distribution generalization to novel categories of scenes not present in the training set. In particular, the method relies on heavy fine-tuning through Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO), which may hinder its generalization compared to methods that keep the base LLM frozen.\n\nAnother limitation is that, being based on a Bird’s-Eye View (BEV) representation, the method is not well-suited for handling scene layouts with significant vertical stacking or clutter—such as books on bookshelves. While this is not the primary focus of the paper, it does limit its broader applicability.\n\nFinally, although the authors use LLM-based evaluators (as in LayoutVLM), they do not report any human studies to directly assess alignment or plausibility.", "was_revised": true, "final_justification": "All my concerns have been addressed. I appreciate the authors' efforts. I would still encourage the author to discuss the limitations of BEV representations. I am leaning towards accepting this paper.", "rebuttal": "Thank you sincerely for your valuable time and insightful comments.\n\n### **W1 & Q3: Lack of out-of-distribution (OOD) evaluation; impact of SFT/DPO on generalization**\n\nWe appreciate the reviewer’s insightful concern. In fact, the qualitative and quantitative evaluations in both the paper and appendix already assess generalization to unseen scene types. During the Supervised Fine-Tuning (SFT) stage, we use data derived from 3D-Front, which only contains four scene categories: bedroom, living room, study, and dining room. However, the scene prompts used in DPO training and final evaluations are generated separately and explicitly exclude those scene types, ensuring no category-level overlap between training and evaluation. This separation was enforced via two disjoint prompt sets detailed in Appendix A.3.\n\n### **W2: BEV limitation in handling vertical stacking (e.g., books on bookshelves)**\n\nWe agree with the reviewer that BEV has inherent limitations in modeling vertical stacking. However, many such cases (e.g., “books on bookshelf”) can be treated as composite assets in our system. For example, “a filled bookshelf” can be generated as a single object, which captures the visual semantics without modeling each individual book.\n\nMoreover, in our experiments, we find that explicitly separating BEV layout and 3D lifting leads to better overall performance than jointly modeling them, highlighting the effectiveness of our two-stage design. In practice, scenes with extremely dense vertical stacking (e.g., cluttered desktops or fully packed shelves) are relatively rare and not the primary focus of our work. That said, fully modeling fine-grained vertical clutter remains an important future direction, which may require more powerful spatial reasoning capabilities.\n\n### **W3: No human study reported to directly assess alignment or plausibility**\n\nWe thank the reviewer for pointing this out. In addition to automated evaluations (e.g., via LayoutVLM), we conduct a human evaluation study to directly assess alignment and plausibility from a user-centric perspective. Specifically, we recruite 25 compensated participants with relevant backgrounds in computer science and 3D vision. Each participant evaluated layout samples from multiple methods, randomized and anonymized, based on three axes (All ratings are collected using a 5-point Likert scale):\n\nSemantic Compliance: Whether the layout reflects the input prompt faithfully.\n\nPhysical Plausibility: Whether objects are well-supported and physically reasonable.\n\nLayout Rationality: Whether the spatial arrangement is typical and convincing.\n\n| Method              |   Semantic Compliance |   Physical Plausibility |   Layout Rationality |\n|:--------------------|----------------------:|------------------------:|---------------------:|\n| LayoutGPT (coarse)  |                  3.72 |                    1.84 |                 1.72 |\n| LayoutGPT (medium)  |                  2.88 |                    1.64 |                 1.76 |\n| LayoutGPT (fine)    |                  3    |                    2.32 |                 2.04 |\n| LayoutGPT (overall) |                  3.2  |                    1.93 |                 1.84 |\n| I-Design (coarse)   |                  3.72 |                    2.72 |                 2.44 |\n| I-Design (medium)   |                  3.08 |                    3.36 |                 2.48 |\n| I-Design (fine)     |                  3.32 |                    3.72 |                 2.72 |\n| I-Design (overall)  |                  3.37 |                    3.27 |                 2.55 |\n| Holodeck (coarse)   |                  4.12 |                    3.64 |                 2.84 |\n| Holodeck (medium)   |                  2.8  |                    2.52 |                 2.48 |\n| Holodeck (fine)     |                  2.4  |                    2.6  |                 2.04 |\n| Holodeck (overall)  |                  3.11 |                    2.92 |                 2.45 |\n| Ours (coarse)       |                  4.56 |                    4.64 |                 4.52 |\n| Ours (medium)       |                  4.6  |                    4.68 |                 4.64 |\n| Ours (fine)         |                  4.84 |                    4.72 |                 4.8  |\n| Ours (overall)      |                  4.67 |                    4.68 |                 4.65 |\n\nOur method consistently received the highest average scores across all categories, showing strong agreement with human preferences. This study complements the VLM-based evaluation and confirms that our method produces layouts that are not only quantitatively strong but also preferred by human evaluators.\n\nWe also include comparison with ATISS, DiffuScene, and Infinigen:\n\n| Method     |   Physical Plausibility |   Layout Rationality |\n|:-----------|------------------------:|---------------------:|\n| ATISS      |                    3.66 |                 3.54 |\n| Infinigen  |                    4.66 |                 4.44 |\n| Diffuscene |                    3.46 |                 3.1  |\n| Ours       |                    4.62 |                 4.48 |\n\nWhile baselines perform strongly overall, our method matches or surpasses their performance on certain scene categories, and additionally supports open-vocabulary generation.\n\n### **Q1: Robustness to noisy CoT traces**\n\nWe conduct controlled experiments to assess robustness by:\n\n* **Shuffling** the order of CoT steps, and\n* **Removing** the CoT reasoning entirely.\n\n| Setting      | Pos   | Rot   | PSA   | OOB↓  | Collision↓ | Clip  |\n| ------------ | ----- | ----- | ----- | ----- | ---------- | ----- |\n| Normal CoT   | 64.47 | 57.18 | 68.82 | 26.04 | 16.77      | 25.27 |\n| Shuffled CoT | 60.33 | 54.20 | 62.90 | 29.40 | 16.90      | 22.14 |\n| No CoT       | 56.80 | 50.95 | 54.75 | 34.60 | 18.55      | 21.76 |\n\nShuffling degrades performance modestly, while removing CoT causes a significant drop. These results show that structured CoT reasoning is essential for layout quality.\n\n### **Q2: Importance of dual reward evaluators**\n\nOur reward signal integrates:\n\n* VLM-based evaluator focuses on high-level visual and physical plausibility, judging the overall realism and global coherence of the generated layout.\n\n* Reasoning LLM-based evaluator operates at a fine-grained, object-level, verifying that each object's position, orientation, and relation to others.\n\nThese components are complementary. Our ablations show that removing either leads to imbalanced layout optimization. As shown in Table 2, the full model using a dual-evaluator significantly outperforms the simple reward setting that relies on a single evaluator. Thus, the dual-evaluator setup is critical for robust and generalizable learning.\n\n\nWe hope our responses can address your concerns. Please feel free to inform us of any further issues."}, {"review_id": "DJ62NZuZ5s", "reviewer": "Reviewer_okhS", "summary": "This paper proposed a model that can generate renderable indoor scenes from text prompts. This is achieved by training several specialized LLM agents. The authors decompose the task into BEV layout generation, 3D lifting, and refinement, each stage handled with a specialized agent. The agents are trained with guidance from other VLMs/LLMs such as GPT4o through SFT and DPO. The authors show visual examples of the generated scenes and do and numerical comparisons with a few baselines. I find the overall quality of the generated scene pretty good and plausible. Overall, this is a technically sound and well-written paper solving a useful task, but I have some questions regarding evaluations and limitations.", "questions": "- How is the shape and size of the room determined? Is it a fixed size across training and validation, or input to the layout generation framework? In the real world, rooms are often not square-shaped and may have additional walls inside them. Is your model capable of handling these additional constraints?\n- The scenes shown are kind of toyish and empty, with only a few objects. How well can your model scale up to more complex scenes? For example, an auditorium with hundreds of chairs, or some special types of rooms such as a conservatory in a botanical garden filled with plants? The limitations section touches on this a little bit, but I think the discussion is not enough. What is the exact boundary of complexity that the current model can push? Can we scale it up in the future with more data/more GPUs, or are there any fundamental issues you would foresee?\n- I'm very interested in seeing how well the proposed method can handle the composition of objects (for example, many small objects placed on a shelf/table). In Fig 3 last row, are the objects on the table generated separately and placed on the table, or is the \"table with object\" a single asset? The bathtub with ducks example is interesting, but in general, I would like to see more examples of how these compositions are achieved and gradually modified to be physically plausible.", "limitations": "The authors discussed limitations in inference time and complexity constraints, but I want to see more in-depth discussions on the complexity constraints (see Questions). Also I would like to see some failure cases.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength:\n- Decomposing the task into layout generation, 3D lifting, and refinement makes sense intuitively.\n- Training specialized agents with SFT followed by RL-based finetuning, although being a standard pipeline for LLM finetuning, makes sense in this task.\n- The paper is well-written and easy to follow.\n\nWeaknesses:\n- Evaluation: The authors do automatic evaluations with VLM in Tab 1, which is valuable, but the authors should also add human preference evaluations, which is more straightforward and convincing. \n- Baselines: The authors compare against a few baselines using LLM agents, but it would also be interesting to see how rule-based methods work. For example, the method used in Infinigen-Indoors can handle pretty complex layout planning. Since there aren't many objects in the rooms generated, my guess is that the rule-based method would be a pretty strong baseline.\n- There are some other limitations I find that I want to discuss, but I put them in the Questions section.", "was_revised": true, "final_justification": "My concerns on evaluations are largely addressed. Although the paper still has some clear limitations, as I pointed out in the discussions, I think the paper provides valuable knowledge that outweighs the limitations. \nI noticed other reviewers challenge the novelty and soundness in the LLM fine-tuning part. I'm not an expert on that so those might be valid concerns. Therefore, I list my confidence (3) to be relatively low, but I would recommend acceptance to the best of my own knowledge.", "rebuttal": "Thank you sincerely for your valuable time and insightful comments.\n\n### **W1: Human preference evaluation**\n\nThank you for the suggestion. In response, we conduct a **user study with 25 compensated volunteers**, all with relevant backgrounds in 3D vision or computer science.  Participants are asked to rate anonymized, randomly ordered layouts generated by different methods across the following three criteria:\n\nSemantic Compliance: How well the layout matches the input text description.\n\nLayout Rationality: Whether the spatial arrangement of objects is reasonable, functional, and typical of real-world scenes.\n\nPhysical Plausibility: Whether the layout is physically sound—for example, no floating or interpenetrating objects, and appropriate support relationships.\n\n| Method              |   Semantic Compliance |   Physical Plausibility |   Layout Rationality |\n|:--------------------|----------------------:|------------------------:|---------------------:|\n| LayoutGPT (coarse)  |                  3.72 |                    1.84 |                 1.72 |\n| LayoutGPT (medium)  |                  2.88 |                    1.64 |                 1.76 |\n| LayoutGPT (fine)    |                  3    |                    2.32 |                 2.04 |\n| LayoutGPT (overall) |                  3.2  |                    1.93 |                 1.84 |\n| I-Design (coarse)   |                  3.72 |                    2.72 |                 2.44 |\n| I-Design (medium)   |                  3.08 |                    3.36 |                 2.48 |\n| I-Design (fine)     |                  3.32 |                    3.72 |                 2.72 |\n| I-Design (overall)  |                  3.37 |                    3.27 |                 2.55 |\n| Holodeck (coarse)   |                  4.12 |                    3.64 |                 2.84 |\n| Holodeck (medium)   |                  2.8  |                    2.52 |                 2.48 |\n| Holodeck (fine)     |                  2.4  |                    2.6  |                 2.04 |\n| Holodeck (overall)  |                  3.11 |                    2.92 |                 2.45 |\n| Ours (coarse)       |                  4.56 |                    4.64 |                 4.52 |\n| Ours (medium)       |                  4.6  |                    4.68 |                 4.64 |\n| Ours (fine)         |                  4.84 |                    4.72 |                 4.8  |\n| Ours (overall)      |                  4.67 |                    4.68 |                 4.65 |\n\nOur method consistently received the highest average scores across all categories, showing strong agreement with human preferences. This study complements the VLM-based evaluation and confirms that our method produces layouts that are not only quantitatively strong but also preferred by human evaluators.\n\nWe also include comparison with ATISS, DiffuScene, and Infinigen:\n\n| Method     |   Physical Plausibility |   Layout Rationality |\n|:-----------|------------------------:|---------------------:|\n| ATISS      |                    3.66 |                 3.54 |\n| Infinigen  |                    4.66 |                 4.44 |\n| Diffuscene |                    3.46 |                 3.1  |\n| Ours       |                    4.62 |                 4.48 |\n\nWhile baselines perform strongly overall, our method matches or surpasses their performance on certain scene categories, and additionally supports open-vocabulary generation.\n\n### **W2: Rule-based baseline comparison (e.g., Infinigen-Indoors)**\n\nWe include **Infinigen-Indoors**, ATISS, and DiffuScene as new baselines and compared its performance on bedrooms and living rooms with our method:\n\n| Method     | Scene      | Pos   | Rot   | PSA  |\n|------------|------------|-------|-------|------|\n| ATISS      | bedroom    | 81    | 70    | 86   |\n| ATISS      | livingroom | 71.5  | 79.5  | 76   |\n| ATISS      | overall    | 76.25 | 74.75 | 81   |\n| Diffuscene | bedroom    | 66    | 63    | 70   |\n| Diffuscene | livingroom | 66.5  | 66    | 60   |\n| Diffuscene | overall    | 66.25 | 64.5  | 65   |\n| Infinigen  | bedroom    | 75.5  | 78.3  | 80   |\n| Infinigen  | livingroom | 75.5  | 75.8  | 76   |\n| Infinigen  | overall    | 75.5  | 77.05 | 78   |\n| Ours       | bedroom    | 82.5  | 75    | 96   |\n| Ours       | livingroom | 86    | 75.4  | 80   |\n| Ours       | overall    | 84.25 | 75.2  | 88   |\n\nWe also evaluate Infinigen in the **user study**. While it performs well, our method achieves similar or better results on specific scene types and supports **open-vocabulary generation**, which rule-based systems cannot easily accommodate.\n\n### **Q1: Room shape and input**\n\nThe room size is provided as input to the layout generator. In CoT Activation, we extract it from 3D-Front scene bounding boxes as part of the input in training. In DPO, the object size is inferred from the LLM during data pair generation, as shown in Appendix A.1. Specifically, the LLM predicts the appropriate size based on the scene description, and this inferred size is then used together with the scene context to construct data pairs for DPO training. Currently, we support rectangular rooms, which cover the majority of real-world layouts. Non-rectangular or complex-shaped rooms are an interesting future extension, though they are not the main focus of our method.\n\n### **Q2: Handling complex scenes**\n\nIn practice, our model performs well in scenes with up to 30 objects, which shows no significant difference compared to the baselines. This upper limit stems from the training data distribution (3D-Front rarely contains denser scenes), rather than algorithmic constraints. We believe the method is scalable, particularly through training on datasets where individual scenes contain more objects, as well as by incorporating strategies such as recursive layout decomposition, which divides complex rooms into manageable subregions. We are actively exploring these directions. We will add more discussions in the revised paper.\n\n### **Q3: Object composition and substructures**\n\nIn Figure 3 (last row), each item on the table is placed individually. Due to rebuttal formatting limits, we could not include more visualizations, but will provide extensive examples of such compositions in the next version to offer a more intuitive understanding of Iterative Asset-Layout Alignment process.\n\nWe hope our responses can address your concerns. Please feel free to inform us of any further issues."}]

Author rebuttal:

Meta-review: This paper introduces DirectLayout, a novel framework for generating 3D indoor scene layouts directly from textual descriptions. The core idea is to bypass the common paradigm of generating intermediate representations (like scene graphs) followed by constrained optimization. Instead, the authors propose a method where a Large Language Model (LLM) is trained to directly output numerical layout parameters (e.g., coordinates, dimensions, orientation). The framework decomposes the task into three stages: generating a 2D Bird's-Eye View (BEV) layout, lifting it to 3D, and an iterative refinement step to align the layout with the generated 3D assets. 

Initial reviews were mixed, but following the rebuttal and subsequent discussion, a clear consensus emerged. Two reviewers are solid "Accepts," and one "Borderline Accept" reviewer stated that their major concerns were "fully addressed." Only one reviewer remains at "Borderline Reject," still having valid reservations about the technical novelty and the ultimate reliability of LLM evaluators.  However, even this reviewer acknowledged that many of their concerns were addressed.

Given these points above, we have decided to accept the paper.
Decision: Accept (poster)