OptiScene: LLM-driven Indoor Scene Layout Generation via Scaled Human-aligned Data Synthesis and Multi-Stage Preference Optimization
Authors: Yixuan Yang
Zhen Luo
Tongsheng Ding
Junru Lu
Mingqi Gao
Jinyu Yang
Victor Sanchez
Feng Zheng
Venue: NeurIPS 2025 poster ()
Keywords: Indoor scene layout generation
LLM-based
DPO
3D
Synthetic Data
Primary area: applications

Abstract: Automatic indoor layout generation has attracted increasing attention due to its potential in interior design, virtual environment construction, and embodied AI. Existing methods fall into two categories: prompt-driven approaches that leverage proprietary LLM services (e.g., GPT APIs), and learning-based methods trained on layout data upon diffusion-based models. Prompt-driven methods often suffer from spatial inconsistency and high computational costs, while learning-based methods are typically constrained by coarse relational graphs and limited datasets, restricting their generalization to diverse room categories. In this paper, we revisit LLM-based indoor layout generation and present 3D-SynthPlace, a large-scale dataset that combines synthetic layouts generated via a `GPT synthesize, Human inspect' pipeline, upgraded from the 3D-Front dataset. 3D-SynthPlace contains nearly 17,000 scenes, covering four common room types—bedroom, living room, kitchen, and bathroom—enriched with diverse objects and high-level spatial annotations. We further introduce OptiScene, a strong open-source LLM optimized for indoor layout generation, fine-tuned based on our 3D-SynthPlace dataset through our two-stage training. For the warum-up stage I, we adopt supervised fine-tuning (SFT), which is taught to first generate high-level spatial descriptions then conditionally predict concrete object placements. For the reinforcing stage II, to better align the generated layouts with human design preferences, we apply multi-turn direct preference optimization (DPO), which significantly improving layout quality and generation success rates. Extensive experiments demonstrate that OptiScene outperforms traditional prompt-driven and learning-based baselines. Moreover, OptiScene shows promising potential in interactive tasks such as scene editing and robot navigation, highlighting its applicability beyond static layout generation.

OptiScene: LLM-driven Indoor Scene Layout Generation via Scaled Human-aligned Data Synthesis and Multi-Stage Preference Optimization
Yixuan Yang!.:. 1.3+ Zhen Luo
Tongsheng Ding Junru Lu
Mingqi Gao
Jinyu Yang*
Victor Sanchez²
Feng Zheng" 1.59
University of Warwick Tencent Youtu Lab arooldyang90@mail.com luoz2024@mail. sustech. edu. cn, 3350538951@gmal...
SUUSecc Shanghai Innovation Institute
Tapall.ai
"Spatialtemporal Al
3D syssynthPaace Dataset
OptiScene
Room Editing
Synthatic Instructions
User Insput
Open.
Layost Json
Tham bedroam with "in, the have black bad with nightrtandt....
[objecta) Bleck book 5p Bounding bee [-]:
Add deyiih cheirs
Aneb Oathurt
S
GPT-based Synthasiser
Course to fine DPO Optimiser
Poitien f-):
woudes coffee
Retation [-]
Haman Filter & Correct
J1
Joon Weapper
enafie
Rendering madule
modi
Srom
Type (edrasen)
Coarse-te torrse-to-fnne SFT
Rendering vesults
[Reem Sixll3mm'Mm [objech]
Navigation
y
Plaxse And stuel
with aned purfece
Resalts Vis.
the roem.
Rooma
Types
Performance (SR)
32-frutt 33-4ythh
B
B
Seckirom
0
50- 10-1teth.
Vitclen
Liplace
Sptiloone
Halsdark
Figure Overview of proposed 3D-SynthPlace dataset and OptiScene framework for indoor layout generation. (Left): We propose 3D- DD-SynthPlace, large-scale, high-quality indoor layout dataset. (Middle): Our open-source LLM-based generator, OptiScene, takes user instructions and produces structured layout representations through two-stage, coarse-to-fine fine optimization. (Right): OptiScene supports interactive layout editing and downstream tasks such as robotic navigation. (Bottom): Qualitative layout visualizations and quantitative comparisons show OptiScene's superior performance (sucess rate) over existing prompt-driven and learning-basee baselines.
Abstract
Automatic indoor layout generation has attracted increasing attention due its potential in interior design, virtual environment construction, and embodied AL. Existing methods fall into categories: prompt-driven approaches that leverage proprietary L.MM services (e.g., GPT APIs), and starning-based methods trained layout data upon diffusion-based models. Prompt-driven methods often suffer from spatial inconsistency and high computational costs, while lcarning- methods are typically constrained by coarse relational graphs and limited datasets, restricting their generalization to diverse room categories. In this paper, we revisit LLM-bbsed indoor layout generation and present 3D-SynthPlace, a large-scale
Equul contribution. Cooresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

dataset that combines synthetic layouts generated via 'GPT synthesize, Human inspect' pipeline, upgraded from the 3D-Front dataset. 3D-SynthPlace contains nearly 17,000 scenes, covering four common room types- beeeooed living room, kitchen, and bathroom enrhhoee-eerichedd with diverse objects and high-level spatial anno- tations. We further introduce OptiScene, strong open-source LLM optimized for indoor layout generation, fine-tuned based on our 3D-SynthPlace dataset through our two-stage training. For the warm-up stage I, we adopt supervised fine-tuning (SFT), which is taught to first generate high-level spatial descriptions then condi- tionally predict concrete object placements. For the reinforcing stage II, to better align the generated layouts with human design preferences, we apply multi multi-turm direct preference optimization (DPO), which significantly improves layout quality and generation success rates. Extensive experiments demonstrate that OptiScene outperforms traditional prompt-driven and learning-hasedd baselines. Moreover, OptiScene shows promising potential in interactive tasks such as scene editing and robot navigation, highlighting its applicability beyond static layout generation. Project page: optiscene.githuubiio
1
Introduction
Indoor Scene Layout Generation has attracted growing attention across the 3D vision fields, including virtual environment synthesis [40] 10
46
49
12 robot navigation [29] 28 50 45 digital twin
30 391, interior design automation 42 [4] [7], and numan-centrin simulation in some scenes 47
construction [5] 20
38 [6]. With the increasing need for simulating realistic virtual environments, learning to generate functionally plausible, physically feasible, and semantically meaningful spatial arrangements has become essential capability for intelligent systems.
36
Existing approaches for indoor scene layout generation fall broadly into two paradigms: prompt- driven methods that leverage large language models (LLMs) through well-defined natural language prompts, and learning-bate -baedd methods that train models on layout data to capture spatial priors. Prompt-driven methods, such as Holodeck [42]. SceneCraft [1), FlairGPT (18), SceneTelller (23], I-Design (2). Lay [32] and LayoutGPT (8) typically rely on proprictary LLM services (e.g., GPT APIs) to produce layouts based multi-turn prompting retrieval-based in-context examples. Although prompting approaches have shown competitive performance, they suffer from fundamental limitations such incorrect domain alignment, weak controllability, and most impor- tantly, superficial physical understanding. Moreover, these commercial LLM APIs are closed-source, prohibitively expensive for large-scale generation, and thoroughly static that not be modeled with new 3D spatial priors or perfectly encoded with human preferences. As a result, they are ill-suited for fine-grained layout tasks that require geometric validity, functional coherence, and task-specific adaptation. Learning-bai based methods, on the other hand, explicitly optimize local models using open- source structured layout data. Diffusion-
approaches such as LEGO-Net (37), DiffuScene (33), InstructScene [17], and PhyScene [40] refine layouts iteratively by modeling object-object relation- ships through layout graphs or bounding boxes. LLplace [41] represents another type of attempt to fine-tune LLMs instead of diffusion-based models. Most learning- baased models are trained on datasets like 3D-Front
9 which contain fewer than 7,000 usable layouts and focus predominantly on bedrooms and living/dining rooms. This limited data diversity not only restricts scene coverage but also biases the model toward overfitting on narrow spatial patterns. Beyond the dataset, while existing learning-bass based methods have shown progress in modeling spatial layouts, they essentially learning objective that aligns with human minds. Most existing approaches optimize for data reconstruction or distribution matching, rather than aligning with human notions, such as what constitutes a desirable, functional, or aesthetically pleasing layout. Without explicit reasoning of human-aligee aligned supervision, these models struggle to generalize across room types, adapt to variable orientations, or reflect higher-level spatial semantics- capabiiiiis that are essential for practical deployment in design systems or embodied environments.
lack a preference-awe
To address the fundamental obstacle naamely the lack of human-aliggedd optimization preference, insufficient spatial reasoning, and limited scene diversity- we propose 3D-SynthPlace, a hurge-scale dataset with more than 16,000 layouts that augments 3D-Front [9] with over 9,000 synthetic layouts generated via Holodeck, as well as introduce OptiScene, an LLM-basee framework that enables controllable and preference-aware indoor layout generation. Different from existing GPT-based pipelines that depend on closed APIs and prompt engineering, OptiScene is dedicated to post-tuning
2

---

open-source LLM for spatial design tasks, making it adaptable, efficient, and deployable in real-world systems. Serving as an important prerequisite, Sy-yyhhPlaaee ensures sufficient training diversity. All generated scenes within the dataset are manually filtered and rigorously corrected to ensure quality, and the dataset expands room coverage beyond the conventional bedroon/living room focus to include kitchens and bathrooms, providing a more comprehensive training foundation. Building on 3D-SynthPlace, we enhance the regular SFT tuning by introducing high-level semantic reasoning plans: instead of directly predicting placements, the model is first guided to generate natural language summary that describes spatial organization and object relationships inside the room. This step provides an interpretable outline that helps the model internalize functional zoning and spatial wreettchhllenger that graph-based or purely coordinate-die drivvn approaches often fail to address. Afterwards, the SFT model utilizes the room layout summary for robust generation. To further align the generation with human preferences, we introduce a two-stage Direct Preference Optimization (DPO) 26y reinforcing process. In the first stage of DPO, we construct preference pairs from an expert-curated subset of human-consensus layouts, as the positive preference, and contrast them against group of suboptimal model-generated variants by the SFT basis, fostering the SFT model to capture subtle stylistic and structural preferences. In the second stage, we synthetically inject spatial violations (e.g., collisions and boundary overflows) into positive scenes produce harder negatives, encouraging the model to learn robust, geometry-aware layout decisions. Through this bessoming-guidedd anfference-alignee training strategy, our OptiScene bridges the gap between static language generation and dynamic spatial intelligence, producing layouts that are physically plausible, semantically coherent, and practically usable across design and robotics workflows.
As shown in Fig. OptiScene demonstrates consistent improvements across multiple dimensions in our experiments, including success rate and layout quality. Overall, our contributions are threefold:
We construct SD-SynthPlace, a large-scale dataset with more than 16,000 scene layouts, which combine the 3D-Front dataset and over 9,000 synthetic layouts generated by Holodeck. All samples are manually filtered and corrected to ensure quality, and we expand the scene coverage to include kitchen and bathroom layouts, providing a richer and more diverse training corpus for indoor layout generation.
We propose OptiScene, human-aligned indoor layout generation framework. The frame- work introduces high-level semantic reasoning into a supervised fine-tuning (SFT) stage, encouraging the model to infer spatial intent and object relationships before predicting concrete layouts. Building on this, it further applies a two-stage Direct Preference Optimiza- tion (DPO) reinforcement align layout generation with human preferences and physical constraints.
We conduct comprehensive evaluations across multiple metries and prove that OptiScene achieves state-of-the-ae performance. Furthermore, we show its effectiveness on down- stream tasks like interactive editing and robot navigation, validating its practical applicability.
2
Related work
Methods & datasets for scene layout generation. Besides traditional methods [35 34 25 recent methods can be grouped into prompt-driven and learning- -based approaches. Prompt-driven meth- ods use LLMs via structured or free-form prompts to generate layouts, including Holodeck 42
SceneCraft 14. FlairGPT [8], Teller 23 I-Design (2] Aguina [.. LayoutVLM [32, and
LayouIGPT g These methods show language-scene grounding but often rely on proprietary LLMSs (e.g.. GPT), lack spatial fine-tuning, and offer limited controllability or offline deployment. Diffusion models have been used in widely areas [8] 13]. and are now applied in the scene layout genera- tion area, such as LEGO-Net [37]. DiffuScene [33. PhySeene [40], and InstructScene [17]. These methods train diffusion models on layout graphs bounding boxes to improve physical plausibility. They often model object-object relations coarsely, lack expressiveness for human spatial preferences, and are sensitive to room orientation. Most are trained on small-scale datasets like 3D-Front ()] limiting generalization to diverse layouts. Other existing indoor scene datasets 44 different data format. Hence they cannot be applied in the scene layout generation task.
6 27 have
Human preference learning for 2D/3D understanding. Recent LLM research has increasingly explored human preference with reinforcement learning (RL) to enhance reasoning, as seen in models
like OpenAT's ol [55] and DeepSeek-RI [I]. In vision tasks, Visual-RFT [19] improves spatial grounding via reward tuning for localization and object detection. Closer to our domain, MetaS-
3

---

SFf Proces:
lopet:
indo repert sudt aani
Selpett
The lving roomss daee 5m armresss,
Task
High-led Deieription:
obyach Ducrlption Lenefion Rebetinn Sounding her
room
Ream line and object prapertias
Base-LIM Model
armoho seating area for relaxation and conneraatinn purion centared anurd the coffee table The TVV stand bookcase, skbsboard. and cecerative displey functions, nukding the space. (8) Two floor fooriaame tames oreate sont gighing ng onnance the overall contort anderormennd
wooden coffee table with a gloso top, a low rederrgular wasdien TV alard, too single aushloned arrchaire, upright gii wooden
Male Fromp!
frem text
oljecs Beanding Bice
wooden niseboard
with di dlavers and a decoraatvv consolle table
DPO
ference Renderinn Process:
1.Corrs
SIT model dedt
Coarse Negutive Data
DPO staget
Input
Miries
Rendering Madult
optimized medel
Human creferences Leyost (Positive Data)
Optimized
2ini ppo
Ads hosis
Fine Negetive Data
DPO Staye2
Model Ontpat: objects propertias
Figure 2: Overview of the OptiScene pipeline. (1) SFT Process: The user input is transformed into structured instructions and bounding boxes to guide the LLM, which first generates a high-level semantic description before predicting final object placements. (2) Two-stage DPO: The model outputs are aligned with human preferences by progressively training on curated positive data and synthesized negative samples with increasing difficulty. (3) Inference and Rendering: The optimized model generates layouts that are visualized through a rendering module.
patial [24] applies multi-step RL to reorganize cluttered scenes by relocating objects semantically, While effecttive for spatial adjustment, it focuses on rearrangement rather than full layout generation. In contrast, we aim to generate complete layouts from scratch, optimizing both structure and usability through preference-alignee method.
Methodology: 3D-SynthPlace & OptiScene
In this section, present the 3D-Synthe 4D-SynthPlace dataset and OptiScene framework (see Fig. 2b. We begin by providing a concise formulation of our layout generation problem Section 3.1 In Section 3.2 describe the dataset construction process and highlight the criteria and techniques used to ensure high-quality and diverse scene representations. Following this, in Se.tion[.33] we detail the supervised fine-tuning (SFT) process, including the model's input/output format and the design of the meta instruction prompt. And finally in Section 3.4] we discuss how to use Direct Preference Optimization (DPO) perform iterative optimization to make the SFT scene layout result align with human preferences.
3.1 Problem formulation
We define indoor layout generation spatial reasoning task guided by structured language input. Given a user instruction U, the system first wraps the natural language instructions into the JSON- structured input with the floor dimensions F, the room type T, a of object descriptions D and their quantities Q The DI-N are applied to retrieve the relevant 3D object assets 0 = {or .2,. from large-scale 3D assets database DB (eg. Objaverse (7),, where each object is associated with corresponding 3D bounding box size. This can be formulated as (O, bbbx) I~N R(DInN),
The previously described room and object descriptions are converted into a structured JSON specifi- cation X. We then apply the meta prompt template of static generation as P, which is a fixed text wrapper for sorting room and objects descriptions into fluent text instruction, and then guiding the LLM M for effective design. The M then generates a layout from the prompt P(X), specifying 3D positions Ci-N and orientations Fi-N in JSON structure. The L combines the generated objects' properties and input room. Finally, a rendering module Rvjs visualizes the 3D scene layout V inn-- The whole pipeline is formally defined as:
V imn Vii (M (P())), O), where bbox)
(I)

---

1790
3D-SynthPlace
3D- PD-Syyt-Place Bedroom
Helodeck-Syee Livingaem. Chards Namber Aosel: 1081.
3D-Front-
Room & Quamally 2908 Fonets Numbar Canoss
1900
--- 30-Front Lovi-Livigggggtgg Rtom
Narber Aeaets Casses 74
SynthPlace Bathroom
Numbor Roone
1290
3D-9ynth Place Klchhn
100
9.0%
Molndack-yrrth, Kitchar
340
28.3%
17.1%
Number
of Actali 775. Number of Asaets Conowe
30- Lisingrsom flom Guamaliyy stch.
12. 6%
Numbar Aosets Clrses
Helodack-yyriii Bedrosrs. Boom Cuanisity
200
30-Front Bedroom: Guanda
Namber of Aosub Grsses, 91
humber Assele: 788, Number of koser Closes, 100
16 16 Number Fumburs per Room
Objects Quantality Distribution and 3D-SynthPlace
(b)
Distribution 3D-SythhPlace
Figure 3: Data distribution analysis of 3D-SynthPlace.
To support effective training of the layout generation model M, we require large-scale dataset that provides diverse scene contigurations and is aligned with human preferences.
3.2 3D-SynthPlaae dataset construction
While 3D-Frront is widely used in layout generation tasks, it suffers from limited scale and scene diversity. After filtering, only around 7,300 layouts are usable, and most belong to bedroom or living/dining room categories. This narrow distribution hinders generalization, especially for underrepresented room types like kitchens and bathrooms. Manually creating large-scale, high-quality layout datasets is prohibitively costly, making synthetic generation a practical alternative. To this end, we construct 3D-SynthPlace large-scale dataset that augments 3D-Front with over 9,000 additional synthetic layouts, enabling richer training coverage for our reasoning- and preference- learning framework.
We use Holodeck [22 the base generator for constructing the 3D-SynthPlace dataset because manually curating scene layouts is expensive and time-consuning. Holodeck offers a low-cost, casy-to-use alternative by automatically generating layouts from GPT-based prompts. However, during synthesis we observe frequent structural issues: under- under-populatee populated layouts, clustered object placements large rooms, and erroneous object counts or orientations. Approximately 55% of the scenes exhibit such flaws. We apply a strict filtering to remove invalid samples and manually correct plausible ones. We also discard scenes with fewer than 6 objects in bedrooms/living rooms, fewer than 3 in kitchens, and fewer than 2 in bathrooms, to ensure minimum complexity. However, Holodeck's 's default prompts (only consider room description) and output formats differ from our task requirements. To bridge this gap, we (1) unify scene descriptions into a structured JSON input specifying object types and quantities, (2) align asset metadata with 3D-Front conventions, and (3) supplement each layout with a high-level semantic summary describing spatial organization. These semantic descriptions are generated by rendering each layout from top-down and oblique views and prompting GPT to describe spatial relations, providing interpretable supervision for the model's reasoning stage.
The resulting 3D-SynthPlace dataset contains over 16,000 scenes across four room categories, combine with 7,306 3D-Front data samples and 9,360 Holodeck-Si -Sythh data samples, balaneing geometric coverage with semantic diversity of four room types. Figure 3 compares object counts, room type distributions, and the diversity of newly introduced Objaverse assets between 3D-Front and our dataset. Additionally, we further extend our reasoning step supervision by generating consistent semantic summaries for all data within the 3D-SSythPllace, and annotate retrieved objects with more precise category and geometry metadata. Together, these refinements make 3D- DD-SSythPllace more comprehensive and eesooning-frenndl dataset for indoor layout modeling. For further dataset details and visualizations, please refer to the supplementary material.
3.3 Warm-up stage I: supervised fine-tuning (SFT) process
After having constructed high-quality dataset, we leverage 3D-SynthPlace to activate the spatial reasoning and generation capabilities of existing LLMs. We first adapt general- purpose LLM the
5

---

layout generation task via supervised fine-tuning (SFT) on the -D-SynthPlace dataset. Each input instruction X consists of the room type T, floor size F. object descriptions DI... quantities 2 and retrieved bounding boxes bbox all wrapped into a meta prompt P to condition the model.
To improve spatial reasoning and layout quality, introduce semantic reasoning step before the final placement prediction. Instead of directly generating object positions and rotations, the model prompted produce a high-level natural language summary Ssem describing the global spatial structure (e.g., "place the sofa against the long wall with the coffee table in front"). This step serves as an coarse interpretable intermediate prior, helping the model produce fine coherent layouts and better reflect human design preferences, especially in the absence of explicit spatial constraints.
We treat the LLM as a policy ng, where denotes the model parameters. This policy generates scene layouts conditioned on input instructions X. The full generation process is:
X Seem (c, (r)
To (P Q bbox) 1 F)), [~w } no
I-N.
(2)
L- FT (D, Q, bbox, c, b2,,bbxx,,,)-100 r-w))
(3)
We minimize the discrepancy between predictions and ground-truth layouts, including both the semantic summary and structured output. The prompt includes design constraints to (1) avoid collisions, (2) maintain functional organization, and (3) favor boundary-aligned placement to enhance spatial openness. Full prompt templates are provided in the supplementary material.
3.4 Reinforcing stage II: human preference optimization
While our supervised fine-tuning (SFT) enables the model to acquire structured spatial priors, we observe that it still struggles to generate physically valid and semantically coherent layouts. In particular, generated scenes often suffer from object collisions, boundary violations, or even high- level planning inconsistencies due hallucinations in the reasoning process. We attribute these issues to the inherent limitations of LLMs in modeling 3D spatial structures, where the learned relations between objects tend to remain coarse, linguistic, and grometry-agnostic rather than grounded in physical feasibility or human design preferences. To address this gap. we propose a multi-stage optimization framework based on Direct Preference Optimization (DPO) that progressively aligns the model outputs with human preferences and improves its understanding of 3D scene plausibility.
The DPO framework is formulated as preference modeling problem, where the model learns to favor human- oueferpreferree layouts L* over less desirable alternatives L given the same input X from the 3D-SynthPlace dataset D. Formally, we optimize the policy no using the standard DPO loss:
Tgg (L.
X:) X,)
Tgg (L X:) (x))
max E X0
1X 4 logo BBoo
-B BBlog
Thef (
Jhat (
where ß is temperature hyperparameter and (()) is the sigmoid function. Due to the complexity of layout generation and the nuanced nature of human design preferences, adopt a two-stage DPO framework to progressively refine the SFT model.
DPO-stage I: human consensus alignment. Instead of relying on noisy or random model generations to create preference pairs, we take a data-first approach. We observe that curated layouts from 3D- SynthPlace inherently reflect strong human preferences in object placement and room organization. We select 2,200 of such layouts that received unanimous approval from annotators as positive samples L+ For each corresponding scene description X, we then use our SFT model to generate multiple layout candidates. These generated results, while semantically plausible, often deviate from human-intuitie arrangements, and are therefore treated as negative samples L This contrastive pairing ensures that the optimization target explicitly favours human consensus without being biased by generation-stage noise.
DPO-stage II: geometric plausibility enhancement. While the first-stage DPO improves layout semantics and global structure, it does not always eliminate low-level physical errors, such as object collisions, surface overflow, or implausible rotations. These issues arise because the model lacks direct training signals to avoid physically invalid configurations. To address this, we design a second optimization stage focused on hard negative mining: for each ground-truth layout L+, we synthesize perturbed versions L with targeted spatial violations (e.g., objects extending beyond table bounds, interpenetrations with walls or other furniture). These negatives are subtle but structurally invalid, providing sharper gradients to improve the model's sensitivity to physical realism.

---

Table I: Layout quality (FID OOR LT 3D-Front & 3D-SynthPlace.
3D-Front
3D-SynthPlace
Method
Bedroom
Living room
Bedroom
Living room
Kitchen
Bathroom
I-Design
73.56/
80.123
76.42 00.18
80.561 00.19
90.25/ 0.228
85.4 /0.16
DitfuScene 33
49.71 00.12
55.21 0..44
58.37 / 0.
61.46 0011
-/-
-/-
L.place 41
56.63 / 0.12
61.97 /0.17
63.45 / /0006
69.64 / 0.16
89,34/0.14
59.31 /.003
DiffuScene (w/ 3D-Synth) Ak]
41.23 0.06
46.64
38 18.96/000 0.06
44,25/0.12
52.63
39.76/ 0.04
LLplace (w/ 3D-Syuth) [1
38.81 0.07
45.32/0.13
37.66 /0.04
4..72/0.08
43.68 0.08
37.44/ 0.03
OptiScene
33.56/ 0.03
37.68 0.08
26.45 / 0.01
31,25 /..02
41.73/ 0.05
35.29 0.02
Table 2: Layout quality using GPT-4o[Func 1 Layout 1 /Aes. 11 on 3D-Front & 3D-SynthPlace.
3D-Front
3D-SynthPlace
Method
Bedroom
Living room
Bedroom
Living room
Kitchen
Bathroom
I-Design 2
6.7/7.0/7.3
6.5/6.777.1
6.4/6.2/74
6.1/6.0/66 7.9778/88 8.3 8.0/82/8.5
5.8/5.666.
/.0 6.9
DiffuSeene B3
7.8/8.1/8. /8.4 8.6/8.478.5 8,2/78888 8.0
7,5/7.98..
8.2/8.0778
7.4 8.0
LLplace 41
8.5/ 8.6/88
7.2/7.08.. 8.2
7./788/80
7.5/ 8.0
OptiScene
8.6/9.1/9.3
8.8/ 9.0
8.7/9.0/92
/.9/9.2/.1
8.4/8.38888
8.7/9.4/900
Through this two-stage optimization, our model OptiScene develops dual awareness of both design intent and spatial plausibility. Unlike generic LLMs or prompt- prompt-based based tools, OptiScene is explicitly trained to meet the needs of interior design tasks, producing layouts that are not only spatially valid but also aligned with real-world usage and human preferences.
4 Experiments
4.1 Experiments setup
Dataset setup. Our constructed dataset, 3D-SynthScene, contains a total of 16,666 scenes, including 7,306 original layouts from the 3D-Front dataset and 9,360 synthetic layouts generated by Holodeck- Synth pipeline. Following the evaluation protocol of LayoutGPT [8] and L.placee [41), we select 423 bedroom and 53 living room layouts from 3D-Frront as the test set. Additionally, we sample 50 layouts for each of the four room types-bedrood
living room, kitchen, and bathroom-from Holodeck-Synth as an extended test set. The remaining 15,990 scene layouts are used as the training set. As described in Section 3.4 we manually select 2,200 high-quality numan-preferee layouts from the full set of 15,990 training samples as the positive dataset for DPO.
Training setup. We adopt Qwen3-8B as the base LLM for both supervised fine-tuning (SFT) and direct preference optimization (DPO). Due to resource constraints, we apply LoRA-based parameter- efficient fine fine-tuning tuning. For SFT, we set the LoRA & to 32, rank to 16, and dropout rate to 0.05. We train the model for 10 epochs using learning rate of 5 10-6 with a cosine learning rate scheduler. For the two-stage DPO, we use the same LoRA configuration but lower the learning rate to 5 x 10-? and train for 5 epochs with the same cosine scheduler. The whole training process takes roughly 140 GPU hours.
Evaluation metries. We adopt four metrics comprehensively evaluate layout quality. (1) FID, following DiffuScene [3 measures visual realism between rendered and real scenes. (2) Object Overlap Rate (OOR) evaluates physical plausibility by computing the proportion of intersecting bounding boxes. (3) GPTT-4o Ratings assess layout quality from three dimensions
Furniture Layout, and Aesthetics scored from 0 10 via prompt- Luimensioos-Fuunctioonlll Functionaliiy, Rate (UR) reflects human acceptability: among 50 sampled scenes, each judged by 5 annotators, layout counted as usable if at least 3 consider physically and semantically valid.
evaluation. (4) Usability
4.2 Experiment results
Table 3: Layout generation success rate.
FID & OOR metrics. Table T reports lay- out quality across two metrics: FID (L) and OOR (L). Overall, models trained with the extended 3D-SynthPlace data show signifi- cant improvements over their oD-rront-only
Method
Badrom
Living rom
Kitchen
Bathmom
DilluSemme (w/ 3D 3D-Syothi 33
30%
21%
20%
25%
L. (wr 3D-Symth [4] Holodeck
40%
35%
OptiScene
75%
50%
30%
45%
7

---

3D-Fronst Bedroom
3D-Front living room
3D-SynthPlace Beolroom
3D-SynthPlace Living room
3D-SynthPlace kitchen
3D-SynthPlace Bathroom
Diffusoene
Cannot Generalize
Gannet Generalian to bathroom.
I-Design
Miss Object
Object
Miks Object
Miss Object
Mine Object
DiffuScene w/ SynthPlace
Upplace
Upplace
w/ SynthPlace
OptiScene
Figure 4: Sample layouts generated by several models on 3D-Front & 4D-SynthPlace. Common layout errors such as misaligned objects, or object collisions, are highlighted with red boxes. (Please zoom in to see the details.)
counterparts demonstrating the effectiveness of large-scale, diverse training layouts. Among all methods, OptiScene achieves the lowest FID and OOR across every room type, including newly introduced categories such kitchen and bathroom. Compared LLplace (w/ 3D-Synth) in 3D- SynthPlace, OptiScene reduces FID by 11.21 in bedrooms and 9.47 in living rooms, while also halving the OOR in several cases.
GPT-Based evaluation. We further evaluate layout quality using GPT-4o across three aspects: functionality, layout rationality, and aesthetics. As shown in Table 2 OptiScene consistently achieves the highest scores across all room types and metrics. On the 3D-SynthPlace test set, it surpasses DiffuScene and LLplace by average margin of +0.9 in functionality and +1.0 in rationality. In more challenging scenes such as kitchens and bathrooms, OptiScene reaches up to 9.4 in rationality, reflecting stronger spatial reasoning and layout usability.
Usability consistency. To evaluate the robustness and reliability of different layout generation methods, we conduct a usability consistency test in which each model generates 50 layouts per room type, and we record the proportion of layouts deemed valid (i.c., free collisions, interpenctrattons. and major structural violations). As shown in Table 3 OptiScenc significantly outperforms all baselines in both bedroom and living room categorics, achieving 75% and 50% usable layouts respectively. In contrast, DiffuScenc and LLplace achieve much lower success rates, especially on complex scenes such as kitchens and bathrooms. While Holodeck performs reasonably well in bedrooms, its outputs remain less stable across room types.
Qualitative analysis. Figure 4 shows visual comparisons across models on six room categories from 3D-Front and 3D-SSnthPlace. I-Design consistently fails to construct scenes with more than 5-6 objects, often missing essential furniture (marked in blue), highlighting its limited scalability. DiffuScene and LLplace trained only on 3D-Front struggle to generalize, especially to novel room
8

---

Hlan fortkat
Prn parkage
foo yhc
Figure 5: Generated layout for robot object-centric navigation. The figure shows three generated scenes for the instruction-based robot navigation.
Table 4: Room editing performance.
SFT
+DP01
Method OOR ~
ESR
OptiScene
0.023
$FT DPo1
OptiScene-Ediiie
0.028
90%
Table 5: Object-centric navigation performance.
LLM
SR 1
NE 1
Only SFT
GPT-4o
87.56
1.00
Qwen-3-8B
75.92
0.98
MaattGLM4-flasht
72.33
0.92
Figure 6: Progressive improvement over stages.
types like kitchens and bathrooms. DiffuScene particular fails entirely in these categories. With 3DD-SynthPlace training, both methods see noticeable improvement, yet issues like collisions and misalignments (red boxes) persist. contrast, our proposed OptiScene generates coherent, well- structured layouts across all scenes, with correct functional grouping and minimal physical violations, demonstrating superior generalization and human-aligned spatial reasoning.
4.3 Downstream tasks
Room editing is essential for interactive layout generation, yet prior methods like DiffuScene and I-Design lack such capability. Inspired by L.place, we extend 3DDSSyttPllae with an object-level editing dataset containing addition and removal instructions. We fine-tune the model with mixed editing and original data, and evaluate on 20 edit cases using object-overlap rate (OOR) and editing success rate (ESR). As shown in Table OptiScene maintains low OOR post-editing and achieves 90% ESR, with only two failed cases.
Dbject-Centric navigation evaluates whether the generated layouts support goal-directed robot exploration. Given a command like "Find a <object> in the room," " the robot succeeds the target appears within +30 of its field of view and within 2 meters, measured by success rate (SR). We also report navigation error (NE), defined as the final distance to the target. As shown in Table S our layouts support stable navigation across LLM backends, indicating strong spatial usability. We show some robot navigation results our generated 3D scenes in Fig.
4.4 Ablation study
Effect of high-level semantic reasoning. We investigate whether prompting the model generate high-level spatial semantics be-
Table 6: Ablation Study of OptiScene.
fore layout prediction improves overall qual-
Only SPT
Reasoning SPT-DPOI
SPT. DPO2 OOR.
U.R.
ity. As shown in Table adding this reason- ing step to the SFT model increases layout usability (from 28% to 33%). This demon-
0.049
0.048
33%
0.033
40%
0.023
50%
strates that explicitly guiding the model to
first consider object relationships and room intent leads to more coherent and human-alie aaman-aligned scene structures.

---

vit righ fest terantis Decotption
accent
Remase 1 wood console toble.
nang
lie febric.
Retwering
Add tell narrow weedan
ndd floor
baokeuulf
Figure 7: Results of room editing.
Figure 8: Comparison between high-level seman- tic reasoning scene graph reasoning.
Ablation study SFT and Two-Stage DPO. As shown in Table adding reasoning improves OOR and UR by encouraging coherent object planning. DtO-Stagel aligns outputs with human preferences, further reducing violations, while DPO-Stage2 introduces targeted noise correct fine-grained physical errors. Qualitative results in Fig. iillustrate progressive improvement in layout realism and usability across stages.
High-level semantic reasoning v.s. scene graph reasoning. We compare two reasoning supervi- sion strategies: high-level semantic descriptions and scene graphs. As shown in Fig. using natural language descriptions leads to significantly better layout quality, with more coherent and functional arrangements. This because I.LMs are inherently better at interpreting and generating natural language than symbolic graph structures. Scene graphs, while explicit, lack global spatial context and less aligned with the model's pretraining distribution, resulting in fragmented or rigid layouts.
Conclusion
In this paper, present OptiScene, human-alignn indoor layout generation framework based on a fine-tuned open-source LLM. To overcome limitations of existing approaches- such as poor spatial understanding. lack of control, and limited scalability construct 3D-SynthPlace, high- quality dataset that integrates manually refined layouts synthesized by Holodeck with the original 3D-Front scenes. Built on this data, we introduce novel supervised fine-tuning (SFT) strategy that incorporates a high-level semantic reasoning step before final layout prediction, enabling the model internalize human-aligned spatial priors. Furthermore, Optimization (DPO) pipeline to refine the model's 's outputs based on human preffered layouts and physical plausibility constraints. Extensive results show that OptiScene surpasses prior baselines in layout quality and usability. It also generalizes well to downstream tasks such as scene editing and robot navigation. These findings underscore the potential of aligning LLMs with structured reasoning and human preferences for controllable and deployable layout generation.
propose two-stage Direct Preference
Limitation and future work Our work has some limitations that can be improved in future work: 1. We focus on the generation of a single room layout, without considering the generation of multiple room layouts. Our future work focuses on generating the layout of an entire house or building. Our a scene dataset considers only the furniture. Our future work focuses on collecting more small objects and wall objects to build a more complete and detailed room layout, further enhancing the authenticity of the room.
10

---

Acknowledgments
We thank all anonymous reviewers for their constructive reviews, and the support from all authors' institutions.
References
[1] Rio Aguina-Kang, Maxim Gumin, Do Heon Han, Stewart Morris, Seung Jean Yoo, Aditya Ganeshan, R Kenny Jones, Qiuhong Anna Wei, Kailiang Fu, and Damiel Ritchie. Open-universe indoor scene generation using Ilm program synthesis and uncurated object databases. arXiv preprint urXiv:2403.090 09675. 2024.
[2] Ata Çelen, Guo Han, Konrad Schindler, Luc Van Gool, Iro Armeni, Anton Obukhov, and Xi Wang. I-design: Personalized Ilm interior designer. arXiv preprint ar arXiv:2404..88 02838, 2024.
[3] Angel Chang. Manolis Savva, and Christopher D Manning. Learning spatial knowledge for text to 3d scene generation. In Proceedings of the 2014 conference empirical methods in natural language processing (EMNLP), pages 2028-2038, 2014.
[4] Angel Chang, Will Monroe, Manolis Savva, Christopher Potts, and Christopher Manning. Text to 3d scene generation with rich lexical grounding. arXiv preprint arXiv:1505.06220 2015.
[5] Tianyuan Dai, Josiah Wong, Yunfan Jiang, Chen Wang, Cem Gokmen, Ruohan Zhang, Jiajun Wu, and Li Fei-Fei. Automated creation of digital cousins for robust policy learning. arXiv preprint arXiv:2410.0748 2024.
[6] Matt Deitke, Eli VanderBilt, Alvaro Herrasti, Luca Weihs, Kiana Ehsani, Jordi Salvador, Winson Han, Eric Kolve, Aniruddha Kembhavi, and Roozheh Mottaghi. Procthor: Large-scale emhodied ai using procedural generation. Advances in Neural Information Processing Systems, 35:5982-5994, 2022.
Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Farhadi. Objaverse: A universe annotated 3d objects. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13 13142-13153, 2023.
[8] Weixi Feng. Wanrong Zhu, Tsu-jui Fu, Varun Jampani, Arjun Akula, Xuehai He, Sugato Basu, Xin Eric Wang. and William Yang Wang. Layoutgpt: Compositional visual planning and generation with large language models. Advances in Neural Information Processing Systems, 36: 1118-222222222221111111111111111 2023.
[9] Huan Fu, Bowen Cai, Lin Gao, Ling- Ling-Xiao Zhang, Jiaming Wang, Cao Li, Qixun Zeng, Chengyue Sun, Rongfei Jia, Binqiang Zhao, al. 3d-front: 3d furnished rooms with layouts and semantics. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10933-10942, 2021.
[10] Gege Gao, Weiyang Liu, Anpei Chen, Andreas Geiger, and Bernhard Schölkopf. Graphdreamer: Composi- tional 3d scene synthesis from scene graphs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21295-21304, 2024.
[11] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma. Peiyi Wang. Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability llms via reinforcement learning. arXiv preprint arxiv: 12948, 2025.
[12] Jinkun Hao, Naifu Liang, Zhen Luo, Xudong Xu, Weipeng Zhong, Ran Yi, Yichen Jin, Zhaoyang Lyu, Feng Zheng, Lizhuang Ma, et al. Mesatask: Towards task-driven tabletop scene generation via 3d spatial reasoning. arXiv preprint 2509 222811 2025.
[13] Jonathan Ho, Ajay Jain, and Pieter Abhcel. Denoising diffusion probabilistic models. Advances in neural information pmoessing systems, 33:6840-6851, 2020.
[14] Ziniu Hu, Ahmet Iscen, Aashi Jain, Thomas Kipf, Yisong Yue, David A Ross, Cordelia Schmid, and Alireza Fathi. Scenecraft: An Ilm agent for synthesizing 3d scenes as blender code. In Forty-first International Conference on Machine Learning, 2024.
[15] Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El- El-Kishky, Aiden Low, Alec Hel- yar. Aleksander Madry, Alex Beutel, Alex Camey, et al. Openai ol system card, arXiv preprint arXiv:2412.16720, 2024.
11

---

[16] Rui Li. Zixuan Hu, Weaxi Qu, Jinouwen Zhang, Zhenfei Yin, Sha Zhang, Xuantuo Huang. Hanqing Wang. Tai Wang. Jiangmiao Pang, al. Labutopia: High-fidelity simulation and hierarchical benchmark for scientific embodied agents. arXiv preprint urXiv:2505.226:: 2025.
[17] Chenguo Lin and Yadong Mu. Instructscene: Instruction-driven 3d indoor scene synthesis with semantic graph prior. arXiv preprint arXiv:2402.0411. 2024.
[18] Gabrielle Littlefair, Niladri Shekhar Dutt, and Niloy J Mitra. Flairgtt: Repurposing Ilms for interior designs. Computer Graphics Forum, page e70036. Wiley Online Library, 2025.
[19] Ziyu Liu, Zeyi Sun, Yuhang Zang. Xiaoyi Dong. Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi Wang. Visual-rft: Visual reinforcement finc- ain-ttunig. arXiv preprint arXiv:2503.01785 2025.
[20] Yao Mu. Tianxing Chen, Shijia Peng. Zanxin Chen, Zeyu Gao. Yude Zou, Lunkai Lin, Zhiqiang Xie, and Ping Luo. Robotwin: Dual-arm arm robot benchmark with generative digital twins (early version). arXiv preprint or XXix:2409.022200 2024.
[21] Yinyu Nie, Angela Dai, Xiaoguang Han, and Matthias Nießner. Pose2room: understanding 3d scenes from human activities. In European Conference on Computer Vision, pages 425-443. Springer, 2022.
[22] NVIDIA.
Isaac htpp://ddeeooprrrwddaa..vvvvssse
sim 4.0 2024.
robotics simulation
and synthetic data generation.
[23] Başak Melis Öcal, Maxim Tatarchenko, Sezer Karaoğlu, and Theo Gevers. Sceneteller: Language-to-3d scene generation. In European Conference on Computer Vision, pages 362-378. Springer, 2024.
[24] Zhenyu Pan and Han Liu. Metaspatial: Reinforcing 3d spatial reasoning in vlms for the metaverse. arXiv preprint arXiv:2503. 18470, 2025.
[25] Despoina Paschalidou, Amlan Kar, Maria Shugrina, Karsten Kreis, Andreas Geiger, and Sanja Fidler. Atiss: Autoregressiie transformers for indoor scene synthesis. Advances in Neural Information Processing Systems, 34:12013- 2021.
[26] Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728--3333 537411, 2023.
[27] Santhosh K Ramakrishnan, Aaron Gokaslan, Erik Wijmans, Oleksandr Maksymets, Alex Clegg, John Turner, Eric Undersander, Wojeiech Galuba, Andrew Westbury, Angel X Chang, et al. Habitat-matterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai. arXiv preprint arXiv:2109.08238, 2021.
[28] Adarsh Jagan Sathyamoorthy, Kasun Wecrakoon, Mohamed Elnoor, Anuj Zore, Brian Ichter, Fei Xia, Jic Tan, Wenhao Yu, and Dinesh Manocha. Convoi: Context-aware navigation using vision language models in outdoor and indoor environments. In 2024 JEEE/RSJ International Conference on Intelligent Robots and Syxtems (IROS), pages 13837-13844. IEEE, 2024.
[29] Dhruv Shah, Blažej Osiński, Sergey Levine, et al. Lm-nav: Robotic navigation with large pre-trained models of language, vision, and action. In Conference on robot learning, pages 492-504. PMLR, 2023.
[30] Shang, Yuming Lin, Yu Zheng, Hangyu Fan, Jingtao Ding, Jie Feng, Jiansheng Chen, Li Tian, and Yong Li. Urbanworld: An urban world model for 3d city generation. arXiv preprint arXiv:2407. 1965, 2024.
[31] Jamming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models, arXiv preprint urXiv.2010.0200,,2 2020.
[32] Fan-Yun Yun Sun, Weiyu Liu, Siyi Gu, Dylan Lim, Goutam Bhat, Federico Tombari, Manling Li, Nick Haber, and Jiajun Wu. Layoutvlm: Differentiable optimization of 3d layout via vision-language models. arXiv preprint arXiv:2412.022 02193, 2024.
[33] Jiapeng Tang, Yinyu Nie, Lev Markhasin, Angela Dai, Justus Thies, and Mathhis Nießner. Diffuscene: Denoising diffusion models for generative indoor scene synthesis. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 20507-200 20518, 2024.
[34] Kai Wang, Manolis Savva, Angel X Chang, and Daniel Ritchie. Deep convolutional priors for indoor scene synthesis. ACM Transactions on Graphics (TOG), 37(4):1-14, 2018
[35] Xinpeng Wang. Chandan Yeshwanth, and Matthias Nießner. Sceneformer: Indoor scene generation with transformers. In 2021 International Conference 3D Vision (3DV), pages 106 115, IEEE, 2021.
12

---

[36] Zan Wang. Yixin Chen, Tengyu Liu, Yixin Zhu, Wei Liang, and Siyuan Huang. Humanise: Language- conditioned human motion generation in 3d scenes, Advances Neural Information Processing Systems, 35:14959-14977 2022.
[37] Qiuhong Anna Wei, Sijie Ding, Jeong Joon Park, Rahul Sajnani, Adrien Poulenard. Srinath Sridhar, and Leonidas Guibas. Lego-net: Learning regular rearrangements objects rooms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 19037-19047, 2023.
[38] Rongtao Xu, Han Gao, Mingming Yu, Dong An, Shunpeng Chen, Changwei Wang. Guo, Xiaodan Liang, and Shibiao Xu. mdom: Unified modal-contextun reasoning for embodied question answering. arXiv preprint arXiv:2507. 220222277..2..... 2025.
[39] Rongtao Xu, Jian Zhang, Minghao Guo, Youpeng Wen, Haoting Yang. Min Lin, Jianzheng Huang, Zhe Li, Kaidong Zhang, Liqiong Wang, et al. AO: An affordance-awart hicrarchical model for general robotic manipulation. arXiv preprint ar 25v: 12636, 2025.
[40] Yandan Yang. Baoxiong Jia. Peiyuan Zhi, and Siyuan Huang. Physcene: Physically interactable 3d scene synthesis for embodied ai. In Proceedings of the IEEF/CVF Conference Computer Vision and Pottern Recognition, pages 16262-16272, 2024.
[41] Yixuan Yang. Junru Lu, Zixiang Zhao, Zhen Luo, James JQ Yu, Victor Sanchez, and Feng Zheng. Liplace: The 3d indoor scene layout generation and editing via large language model. arXiv preprint arXiv:2406. 03866, 2024.
[42] Yue Yang, Fan- Yun Sun, Luca Weihs, Eli VanderBilt, Alvaro Herrasti, Winson Han, Jiajun Wu, Nick Haber, Ranjay Krishna, Lingjic Liu, et al. Holodock: Language guided generation of 3d embodicd ai environments, 2024.
[43] Hongwei Yi, Chun-Hao Huang, Shashank Tripathi, Lea Hering, Justus Thies, and Michael Black. Mime: Human-aware 3d scene generation. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 12965-12976, 2023.
[44] Xianggang Yu, Mutian Xu, Yidan Zhang, Haolin Liu, Chongjie Ye, Yushuang Wu, Zizheng Yan, Chenming Zhu, Zhangyang Xiong. Tianyou Liang, et al. Mvimgnet: A large-scale dataset of multi-view images. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 9150-9161, 2023.
[45] Jiazhao Zhang, Kunyu Wang, Rongtao Xu, Gengze Zhou, Yicong Hong, Xiaomeng Fang. Qi Wu, Zhizheng Zhang, and He Wang. Navid: Video-based vlm plans the next step for vision-and langguggee navigation. arXiv preprint arXiv:2402.1 158.5222,,, 2024.
[46] Songchun Zhang, Yibo Zhang, Quan Zheng, Rui Ma. Wei Hua, Hujun Bao, Weiwei Xu, and Changqing Zou. 3d=scenedreamer: Text-driven 3d-consistent scene generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10170-10180, 2024.
[47] Kaifeng Zhao, Yan Zhang, Shaofei Wang. Thabo Beeler, and Siyu Tang. Synthesizing diverse human motions in 3d indoor scenes. In Proceedings of the IEEE/CVF international conference on computer vision, 2023.
pages
[48] Zixiang Zhao. Haowen Bai, Yuanzhi Zhu, Jiangshe Zhang, Shuang Xu, Yulun Zhang, Kai Zhang, Deyu Meng, Radu Timofte, and Lue Van Gool. Ddtm: Denoising diffusion model for multi-modall image fusion, Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 8082 8093, October 2023.
[49] Zixiang Zhao, Haowen Bai, Bingxin Ke, Yukun Cui, Lilun Deng, Yulun Zhang, Kai Zhang, and Konrad Schindler. A unified solution to video fusion: From multi-frame learning to benchmarking. In Advances in Newral Information Processing Systems (NeurIPS), 2025.
[50] Kai Zhu and Tao Zhang. Deep reinforcement learning based mobile robot navigation: A review. Tsinghua Science and Techmology, 26(5):774-61, 2021. 2021.
13

---

Under-populated Layouts
Clustered object Placements
Erroneous object Orientations
ON
Figure 9: The deleted error data which generated by Holodeck.
A More details of 3D-SynthPlaae dataset
A.1 3D scene layout alignment
generation data, (0. 0) is generally used as the bottom-left corner of the floor, while the single room information in 3D-Front extracted from large room formed by collection of multiple rooms, making the floor center more uncertain. To align the centers of the rooms in both datasets, we use (0, 0) as the floor center. We calculate the geometric center of the floor based on the floor boundary coordinates of both datasets and translate the floor boundary and the coordinates of objects within the room accordingly. The offset is the difference between (0, 0) and the original geometric center of the floor. Additionally, use a y-axis coordinate system, their xz-axis rotation directions are opposite. To unify the rotation directions, standardize to the 3D-Front rotation direction by adding 180 to the rotation angle of objects the Holodeck, thus making the rotation angles consistent.
In the Holodeck 442
find that although both Holodeck and FD-Front
A.2 Deleted Holodeck generated data
As discussed in Section 3.2, the 3D scene layout rooms created by the Holodeck generation process have a low success rate. We filter the data and delete badly generated data based on three issues: (I) under-populated layouts. (2) clustered objects placed in large rooms, and (3) erroneous object counts or orientations (see Fig. 99.
Under-populated Layouts: These layouts have noticeably insufficient number of objects, making the space appear sparse and unnatural. Particularly in bathrooms and bedrooms, the lack of sufficient furniture and decorations results in lack of liveliness and functionality. Such layouts may effectively simulate real-life environments, impacting the model's training effectiveness.
Clustered Object Placed in Large Rooms: In large rooms, objects placed too closely together, leading to uneven space utilization. This can cause visual disharmony and fail to realistically reflect the reasonable distribution of furniture in real life. This issue is especially pronounced in living rooms and kitchens, affecting both functionality and aesthetics.
Erroneous Object Counts or Orientations: Errors in the number orientation of objects lead to unreasonable room layouts. For example, the orientation of furniture may not match the room's structure, or the number of objects may be too high or too low, affecting the overall harmony of the room. Such errors can mislead the model in learning spatial relationships, impacting reasoning capabilities.
14

---

(a] Number of objects in SD-Front and 3D-SynthPlace Box plot
(b) 3D-Front Dataset Room Distribution
(d) Comparison of Number of Assets
13.7
[e] Comparison of Number Clasnen
1IA
Place Distribution
Bn-Sytth.
Bedroons
Lhingroom
Bodroom
Khohen
Figure 10: Comparison of object counts, room type distributions, and asset/class statistics between the 3D-Front and 3D-SyythPllace datasets.
In addition to filtering out invalid layouts, exclude small objects produced during the Holodeck generation process, as they are not compatible with the problem definition of our room layout formulation.
A.3 Data distribution description
Figure I0 presents comprehensive distribution comparison between the 3D-Front and 3D-SynthPlace Place datasets across various dimensions:
a): The box plot illustrates the distribution of furniture counts in different room types for both datasets. The 3D-Front dataset shows higher median count in living rooms compared to bedrooms, while the AD-SyythPlaae dataset exhibits similar pattern with slightly higher counts in living rooms and kitchens.
Number of Objects, Fig. a)
Room Distribution, Fig. I & c): Pie charts depict the room distribution within each dataset. The 3D-Front dataset is predominantly composed of living rooms (66.1%), followed by bedrooms (33.9%). In contrast, the S--SythPPlace dataset has more diverse distribution, with living rooms (38%) and bedrooms (27.5 27.5%) being the most common, alongside significant portion of bathrooms (17.5%) and kitchens (17.1%).
Comparison of Number of Assets, Fig. 10 The bar chart compares the total number of assets across different room types. The 3D-Front dataset has a significantly higher total number of assets, especially in living rooms, compared to the Holodeck-Synth dataset.
Comparison Number of Classes Fig. e): The bar chart compares the number of classes available in each dataset. The 3D-Front dataset generally has more classes across different room types, with the most notable difference the living room category.
A.4 High-level semantic deseriptions and reasoning
Figure I1 shows examples of high-level semantic deseriptions and reasoning with the 3D scene layouts. Examples of generated room layouts paired with natural language descriptions. Each row shows two 3D layout visualizations of the same room from different angles, accompanied by GPT- generated description of the spatial arrangement and object relations. The descriptions emphasize functionality, object alignment, and spatial reasoning without relying on directional terms like "left" or "right".
B Instruction-fo-JSON Wrapper
We use a Qwen2.55-7B-Instmmcl L.LM as a wrapper to convert natural language user instructions into structured JSON. The process works follows:
15

---

grey and black double bed centrally positioned room, with blue nightstand with drawers each side of the bed. The wooden corner side table stands near one side of the bed, opposite the blue nightstands. The grey wardrobe with shelves and drawers is placed agsinst the wall near of the blue nightstands.
The bathtuh is pasitioned next to the storage cabinet. The toilet placed opposite the bathtuk. vanity cabinet is located adjacent to the step stool, with the step stool placed between. the vanity cabinet and the toilet.
The oven_and_ssove wnit is positioned adjacent to the kitchen, eobinet, base, which ertends untt along the same plane. The tall_pantry.cabinn cabirtt stands the nitccen_.abbnett.ee base, with the freestanding saanndingmmcroowve.cao cabinet located across an open space from both. The kitchen, islandd stonge situated opposite the hall_pantryccabien cabinet, with mebile__ktcchen cart placed near the kitchen Island wilthniddddtttttttnt sithest
The vectional sofa faces the taflee with two armchairs positioned on the opposite side of the coffer_table. A side,table and lloor are placed near one end of the sectional, sofa. while two additional side tables and another Jaop situated close to the armchatirs The sv_stand and bookshelf are against the far wall, and the sideboard placed adjacent to the bookshelf.
The sectional_sofe sofa positioned opposite the ty_stand, with the coffee table placed centrally between them. Two armchaair facing the sectionalaefa, each adjacent to the coffee_table The bookshalf and sideboard are placed along the walls, with famp positioned near the armchatirs and another floor_lamp claser to the tr.stand.
Figure 11: High-level semantic descriptions and reasoning with the corresponding 3D scene layouts.
The wrapper extracts room size, room type, object types, and object counts from the instruction.
If any field is missing, the wrapper heuristically infers it based existing information (e.g.. number and type of objects).
It then retrieves suitable 3D assets for each object. We follow a purely text- based retrieval approach, inspired by Holodeck [42]. match the objects mentioned in the user instruction to assets in our 3D object database.
The extracted values are inserted into a predefined JSON layout template, which is then used part of the prompt input to OptiScene for inference. This wrapper provides robustness by normalizing diverse instructions, modularity for easier debugging, and flexibility through noise filtering.
C More details of SFT
Section 3.3 describes SFT. Here, we discuss the details of the meta instruction prompt. First, we illustrate the prompt template in the Table 7 and Table 8 As shown in the tables, fine-tune the model for the 3D room layout generation task, we design structured meta prompt that sets the model as a skilled room layout designer. The prompt guides the model through a step-by-step reasoning and generation process, covering object extraction, spatial analysis. layout planning, and final output formatting. It explicitly incorporates design heuristics such as edge-aligned placement, alignment to walls, and functional constraints (e.g., chairs must face desks). The model is instructed to reason about object relationships without using explicit directions (e.g.. "left", "right") and to output both the reasoning process and the final layout in a well-defined JSON format. The response is enclosed in a structured JSON-like format for better consistency and parsing. The prompt also includes post-checklists to ensure validity of the generated layout (e.g.. no overlap, correct coordinate units, functional flow). This design ensures that the model not only generates spatially valid scenes but also explains its decisions in transparent and interpretable way.
16

---

Positive
Samples
Negative
samples
Add noise: out-of-boundary
Add noise: Overlapping
Figure 12: Some positive scene layouts and negative scene layouts with noise additions the DPO- stage2.
D More details of DPO
Section 3.4 introduces the two-stage DPO framework designed to align generated room layouts with human preferences. We begin by selecting 1,100 scene layouts from the Holodeck-Syme dataset and another 1,100 from the 3D-Front dataset. These layouts are characterized by reasonable, functional, and human-aligned spatial arrangements. In the second stage of DPO training (DPO-stage 2), we generate hard negative samples by adding targeted spatial violations to the positive examples. Specifically, as illustrated Fig. T2 we introduce two types noise: (1) Out-of- bouuddaby violations, where furniture items extend beyond the floor area; and (2) Object-overlap violations, where furniture pieces intersect or collide with each other. Despite improvements from SFT and DPO-stage 1, the model still occasionally produces such errors. Therefore, we further fine-tune the model using these challenging negative samples to enhance spatial reasoning and layout robustness.
More details of the experiments
E.1 Inference and rendering setup
All layout generation results are produced using the optimized model obtained after the second stage of DPO. To ensure physically plausible and visually accurate visualization of scenes, employ NVIDIA Isaac Sim 22 as the rendering and simulation backend.
F,2 Details of the evaluation metrics
Object overlap rate (OOR). This metric quantifies the spatial overlap between set of 2D bounding boxes. To calculate the OOR, we first extract the position and size information from each object, where the position is typically represented as (x.y) and the size is given by width and depth. Using this information, we create 2D bounding boxes with the center at the object's position and dimensions determined by the size. We then compute the area of each bounding box and, for each pair of bounding boxes, we calculate the area of their intersection. pairs to obtain the total intersection area, and then the areas of all objects to get the total area. The OOR is then defined as the total intersection area divided by the total area:
sum the intersection areas of all object
E =ii Area Area([ntersection((,..I
OOR
Area(B))
where B; represents the bounding box of the i-th object, Intersection B,, B,) denotes the intersection area of two bounding boxes, and Area(B:) is the area of the bounding box.
GPT-4o evaluation. Following similar approach to the evaluation protocol introduced in Design [2, we employ GPT-4o as automated evaluator assess the quality of our generated room
17

---

layouts, Specifically, GPT-4o is prompted to assign scores (ranging from 0 to 10) based on multiple criteria, including functionality, spatial arrangement, and aesthetic coherence, in alignment with the given user preferences. The detailed prompt template used for this evaluation is provided in Table 9
E.3 More qualitative results.
Figures 13 and 14 show additional qualitative comparisons between the outputs of OptiScene model (left two columns) and the ground truth (right two columns) across various room configurations. is important to emphasize that the objective of OptiScene not to converge to the ground truth layouts exactly, but rather to generate plausible and functional indoor scenes based on high-level semantic and spatial constraints. The generated layouts exhibit more efficient use of space. better object alignment, and clearer functional zoning compared to the ground truth. These examples demonstrate the model's capacity to generalize and produce reasonable, sometimes even preferable, alternative layouts that remain faithful to the intended room semantics. This highlights the potential of OptiScene to support diverse and creative room layout configurations.
E,4 More downstream fasks results.
Figures 15 illustrate more downstream tasks results with room editing. As shown in these figures, with appropriate instructions, the model can edit the room layout.
18

---

Table 7: Meta Prompt Template for generation task (Part 1).
Meta Prompt Template for generation task (Part 1)
You are skilled room layout designer. Your task is to arrange [Objects] within a given [Room Type] effectively. Follow these guidance to complete your design:
(1) Extract the [Room Type], [Room Area]. [Objects], and [Bounding Box Size] from the provided ISON data. (2) Analyze the spatial relationships among [Objects] within the specified [Room Type]. Pay special attention to **avoiding overlap** and * **oonsddrr other spatial factors like accessibility and aesthetics**
(3) Determine and design the precise location of all [Objects] ensuring that their bounding boxes do overlap and that the layout is functional and visually appealing.
(4) prefer objects to be placed at the edge (the most important constraint) of the room if possible
which makes the room look more spacious.
(5) The objects are usually aligned in some ways (parallel or perpendicular to walls).
(6) Chairs must be placed near to the table/desk and face to the table/desk.
(7) Before specifying the detailed positions of each object, first reason step-by-step about their general arrangement and relative spatial relationships: a) Which objects need the most space or have fixed positions (like beds, wardrobes) b) Which objects need to be grouped together (like nightstands with bed) c) Traffic flow and accessibility considerations. Then, clearly articulate your reasoning process. Emphasize the spatial relationships between objects without using explicit directional terms like "left," " "right," "front," " "back." " Summarize the overall arrangement in logical and natural manner, ensuring that all major objects are accounted for. (8) After presenting the thought process, report your design with detailed 3D space coordinates and rotation angles for each object in JSON format, follows:
"object": "object",
"coordinates":
"x":
"y": y,
"z":
"rotate":[
"angle": r
]
The centroid of the room is ("x": 0.00, "y": 0.00, "z": 0.00"). Important Notes about Coordinate System:
-Z-axis points upward (z=0 is floor level)
Rotation angles are in radians, measured in the XY-plane
19

---

Table 8: Meta Prompt Template for generation task (Part 2).
Meta Prompt Template for generation task (Part 2)
(9) The response should follow the following format:
<reasoning>
[Reason]
[/Reason]
</reasoning>
<answer>
[Design]
[/Design]
</answer>
First carefully read this example:
[Example Room Type]
Bedroom
[/Example Room Type]
[Example Objects and Bounding Box Size] /* A fixed example is put here to show the input format* [/Example Objects and Bounding Box Size]
[Example Reason] /* fixed example is put here to show the reason format*/ [/Example Reason]
[Example Output]
/* fixed example is put here show the output format*/ [/Example Output]
Now, please proceed with the design task as outlined and provide only the JSON formatted output of your design:
[Task Room Type] /+Input room type*/
[/Task Room Type]
[Task Objects & Bounding Box Size]
/* The JSON format input of objects description and bounding box size+/ [/Task Objects Bounding Box Size]
Note: the units for the coordinates are meters.
Before submitting your final design, please verify:
All objects are within room boundaries
No objects overlap
Sufficient clearance space exists around furniture
The layout is practical and functional
All rotations are properly specified in radians
Now, please proceed with the design task outlined and provide your thought process and the JSON formatted output of your design:
20

---

Table 9: GPT-4o Prompt Template for Room Layout Evaluation.
GPT-4o Prompt Template for Room Layout Evaluation
You are expert in interior design and human-centric spatial planning. Your task is to evaluate the quality of the following room layout renders based on how well they match the user's design preferences, which provided below (in triple backquotes).
Please assign a numerical score from **O to 10** (0 completely inconsistent, 10 perfectly aligned) considering the following three aspects:
1. Functionality and Activity-based Alignment
Does the layout support natural and efficient use of the space for daily activities (e.g., sleeping, working, relaxing, walking)?
Are key object groupings (e.g., desk and chair, bed and nightstand) placed functionally and accessibly?
there sufficient circulation space ensure human accessibility?
2. Layout and Furniture Placement
Are the furniture pieces arranged in a logical, practical way within the room boundaries?
Are objects positioned with respect to common design principles (e.g., not blocking windows doors, aligned to walls where appropriate)?
Are there overlaps unnatural collisions between objects?
Aesthetic Coherence
Is the overall layout visually balanced and spacious?
Does the arrangement exhibit good proportions, symmetry or asymmetry, and grouping where needed?
Is the furniture distribution harmonious and pleasing to the eye, according to the user's stated aesthetic preferences?
User Preferences:
[Insert user preferences here. }
After considering all the above, return your evaluation in the following JSON format:
{
"functionalityy.core" X,
"layout_score": Y,
ceaethetccssscoree Z,
M,
"cmmentts"; "Brief explanation of your judgment." " Jece
Scoring Guidelines:
Scores should be integers from 0 to 10.
The overall score can be the average or holistic assessment across the three criteria. your scores the "comments" field (I-3 sentences). Note: The goal is to measure how well the generated layout adheres to functional, spatial, and aesthetic expectations given the user's input. Be fair and critical.
Include brief justification
21

---

Front
Top
GT-Top
Figure 13: More qualitative results compared with the ground truth (Part
22

---

Front
Top
GT GT-Front
GT-Top
Figure 14: More qualitative results which are compared with the ground truth (Part 2).
23

---

Remove standard modern wall- mounted toilet.
Remove modern white batktub.
Add silver Cabin.
Remove wooden gas stove.
BPTB
EPTR
Add contemporary wooden wardrobe.
Add 1 tall floor Jamp.
Add extra tall open wooden bookshelf.
Remove 1 small round table.
Figure 15: More editing results.
24

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We have summarized our paper's contributions and scope well in the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of our method and future work in the supplementary material.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiiatinn asymptotic approximations only holding locally). ). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
25

---

Answer:
[NA]
Justification:
We do not have any theoretical results in the paper.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We thoroughly explain the process of constructing our dataset and training, and we provide our LLM prompt template for training.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
26

---

reproducible, they Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [No]
Justification: We not submit the code and data, but we will release them after the paper is accepted.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ https//ccppblcccguuuddeeeoo Cotee//mdeeeSbmmmiic ssi onPolicy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Our paper details the hyperparameters used for training and the division of the training and test sets.
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer:
[No]
Justification:
We do not provide error bars in the experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
27

---

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out of range (c.g. negative emror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We report the computational resources we used.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of http::/ppg.............../o///tt
Answer: [Yes]
Justification: Our research complies with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Our work does not have a social impact. It is just a research topic.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
28

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our work does not pose such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The code and data used have been properly cited referenced.
Guidelines:
The answer NA means that the paper does
existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
29

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
Yes we have discussed this in the paper.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our paper does not have potential risks.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
30

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [Yes]
Justification: Yes we have discussed how LLMs.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy for what should or should not be described.
31


Reviews:
[{"review_id": "K7AotE3yfZ", "reviewer": "Reviewer_qyqm", "summary": "The paper proposes to fix the gap between prompt driven LLM approaches which are costly and learning based methods which are constrained. First it proposes a new dataset-- 3D-SynthPlace to expand the 3D-Front dataset with synthetic samples generated with Holodeck and then filtered with a human in the loop system. The dataset not only doubles the size of the dataset but also increases the diversity of the scenes to include kitchens and bathrooms. The authors train an LLM on this dataset to output a structured JSON which captures the scene with scale and position of the objects. Rather than solely training the LLM using SFT, the authors choose to use RL based DPO fine-tuning for greater alignment with human preference by comparing against perturbed versions (collision) of the scene. The method is compared against 5 baselines and an ablation of the SFT vs RL is provided.", "questions": "* Does the structured JSON proposed here bring any advantage over other very similar representations proposed in related works?\n* I'm very much on the borderline, please address weaknesses", "limitations": "Door and window positions are missing in the generated layouts. It should be discussed in limitations.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "## Strengths\n* The paper is well-written and all necessary details are provided. \n* The results are quite strong, both qualitatively and quantitaively. The evaluation setup apart from LLM as a judge is robust and the ablation shows good gains in performance when tuning with RL. The additional user study paints a more complete picture.\n* The method does not need to perform constrained optimization solve like FlairGPT or Holodeck which makes it computationally more feasible. \n* Lower token cost compared to prompt driven methods & multi-agent frameworks\n* The RL based fine-tuning with petrurbed scenes is ineteresting and lowers errors and object collisions (object overlap rate)\n\n## Weaknesses\n* Related works such as Holodeck and FlairGPT also take into account the door & window positions in the room and walls + objects on walls but this method misses out on this vital detail. Small objects make the scene much richer and provides a more complete layout of the room. Given that the authors use Holodeck for creating the synthetic dataset, I find the omission of small objects a limitation. The authors do not discuss this omission from Holodeck. \n* It's well known in the community that SFT memorizes and RL generalizes [SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training, Chu et al], therefore the contribution is not particularly unique but it might be the first scene layout method to apply this (is this the case?). Nevertheless, I think the authors should mention this in their work since it's not a novel observation. \n* The LLM as a judge (GPT-4o Ratings) for layout evaluation is not robust. It's well known that LLMs fail to provide reliable ratings in a 1-10 scale, compared to their text summaries which are considerably better.  \n\n\n## Neutral\n* The dataset could have been a nice contribution given the human-in-the-loop system for filtering poor layouts. However, since the method disposes small objects, the dataset contribution becomes weaker.", "was_revised": true, "final_justification": "The authors have done a good job with the rebuttal and have adressed my main concerns. I have recommened the authors to include some of the discussions/clarifications and evaluations to be included in the final revision. I recommend the work for acceptance and would like to see some small objects in the results.", "rebuttal": "We sincerely thank you for the positive feedback. We appreciate your recognition of evaluation rigor, and the effectiveness of our RL-based fine-tuning. \nBelow we address your questions.\n\n**R1Q1: Omitted small elements (doors, windows, wall‑mounted items) not addressed despite their inclusion in Holodeck/FlairGPT.**\n\n**Response**:  \n\nFirst, our model is designed for indoor scene layout generation with a focus on placing large furniture items. Accordingly, we extended our dataset based on the 3D‑Front format. The original 3D‑Front does not include doors, windows, or wall‑mounted objects, so—for a fair comparison with prior methods—we omitted those elements during training. As in previous methods, DiffuScene, LLplace, etc., do not predict the wall-mounted objects, we follow this setting.\n\nHowever, when we generated synthetic scenes with Holodeck and applied post‑processing, we did retain the door/window and wall‑object annotations. In this setting, we then ran two new experiments: one trained solely on 3D‑SynthPlace data augmented with doors, windows, and other wall objects; and another trained on a mix of the original 3D‑Front data (without wall objects) and the 3D‑SynthPlace-with‑wall‑objects data. We then inference 100 times on four room categories. The results are demonstrated in Table 1：\n\n**Table 1: Effect of including small wall‑mounted elements on placement recall and overall layout quality.**\n|Training Data|avg. OOR Rate↓| avg. GPT4o ↑| UR ↑|\n|-|-|-|-|\n|3D‑Synth. (with wall objects)|0.037|8.7|63%|\n|Mixed 3D‑Front + 3D‑Synth. (with wall objects)|0.042|8.6|56%|\n\nWhile our method focuses on generating layouts with large furniture, we intentionally did not include small objects in our training data.\nOur primary focus is indoor scene layout generation, consistent with prior works such as **ATISS, DiffuScene, InstructScene, I-design, LLplace, LayoutVLM, and LayoutGPT**—all of which emphasize the placement of large furniture.\n The arrangement of small objects (e.g., books, cups, desk clutter) is a fine-grained subtask that we consider **out of scope** for this work and plan to explore in future extensions.\nOur goal in constructing the dataset is to improve large-object layout modeling, which is foundational to scene generation. As shown in **Table 1 of our main paper**, our dataset significantly improves the performance of strong baselines such as LLplace and DiffuScene. This aligns with our core motivation: to augment and refine the 3D-FRONT dataset to support high-quality, human-aligned layout generation.\nWhile Holodeck includes many small items, their placements are often random and unrealistic, such as scattered desktop objects or items placed directly on the floor. Including such noisy placements would distract the model from learning coherent spatial structure.  Therefore, we intentionally excluded unfiltered small-object data from Holodeck to maintain semantic focus and layout consistency. \nBut we think adding the windows/doors/wall-mounted objects can enhance scenes realistic, we will update the dataset with these and include Table 1 in the next version paper.\n\n**R1Q2. The observation that \"SFT memorizes while RL generalizes\" is not novel—does the contribution lie in applying it to scene layout generation for the first time?**\n\n**Response**:  \n\nWhile it is well known that SFT tends to memorize its training data whereas reinforcement‑style methods generalize (Chu et al.), to our knowledge, OptiScene is the first work to integrate Direct Preference Optimization (DPO) on top of SFT specifically for 3D scene layout generation. We begin by fine‑tuning the model on the annotated dataset and then perform extensive human‑in‑the‑loop curation to extract only those layouts that reflect natural, human‑preferred living environments.\n\nIn **Stage 1 DPO**, these curated positive examples teach the model to correct SFT’s memorization biases and prioritize realistic furniture arrangements—effectively aligning it with genuine human preferences. In **Stage 2 DPO**, we introduce controlled hard negatives— random translations, overlaps (e.g., a floor lamp inserted into a sofa; excluding overlaps common in normal scenes such as a chair under a desk), and minor out‑of‑bounds placements—to force the model to distinguish plausible layouts from artifacts. As reported in Table 6 of main paper, our two‑stage DPO pipeline delivers significant improvements in scene usability and substantially reduces collision and overlap errors compared to an SFT‑only baseline, demonstrating that DPO not only mitigates memorization but also produces layouts that more faithfully mirror real‑world living spaces.  \n\n**R1Q3. Using LLMs (e.g., GPT-4o) as layout evaluators via 1–10 rating scales is unreliable—why do we use this, and what can we test more?**\n\n**Response**:  \n\nAssessing the usability of generated scenes, which is closely aligned with human preference, is indeed a challenging task. In this field, leveraging GPT-based models for evaluation has become a widely adopted practice, nearly all recent works—including **LLplace, I-Design, and LayoutVLM**—have relied on GPT-4o for scoring. In our case, we do not simply feed the generated JSON layouts into GPT-4o. Instead, we render the scenes into images and then provide these images as input to GPT-4o for evaluation. \n\nWe believe that thanks to GPT-4o’s extensive prior knowledge and alignment with human preferences, it provides a reasonable proxy for assessing output quality. However, we also acknowledge its limitations. The final score can be influenced by rendering quality, lighting, and camera perspectives, which may not directly reflect the underlying model’s generation capability.\n\nTo address this, we complement GPT-4o scoring with additional metrics such as FID, OOR, and a human usability score to provide a more comprehensive evaluation of model performance. We agree on the need for a unified quality metric and are building a comprehensive benchmark, inspired by SceneEval, that uniformly renders each scene in Blender from a top‑down view and evaluates against the original JSON and user instruction. In addition to our existing OOR (object overlap rate), FID, GPT‑4o scores, and usability rate metrics, we will include:\n\n- CNT (Object Count): compare expected vs. detected object quantities by rendering each object’s front view and using Qwen2.5‑VL‑7B-Instruct to classify and count instances.  \n\n- NAV (Navigability): compute the ratio of the largest connected free‑space region in a 2D occupancy grid.  \n\n- OOB (Out‑of‑Bounds): analytic check of bounding boxes crossing room boundaries.  \n\n- ACC (Accessibility): identify each object’s main functional face via Qwen2.5‑VL‑7B-Instruct and compute the unobstructed area in front of it. \n\nThis streamlined suite combines rendering, JSON validation, and automated VLM/analytic checks to robustly assess scene completeness, correctness, and real‑world usability.  The model we tested is the version we retrained with the 3D-SynthPlace dataset:\n\n**Table 2:Comparison of DiffuScene, LLplace vs. our method across all benchmark metrics.**\n|Method| vg. OOR ↓|FID ↓|avg. GPT‑4o Score ↑| UR (%) ↑ | CNT Accuracy (%) ↑ | NAV (%) ↑ | OOB (%) ↓ | ACC (%) ↑ |\n|-|-|-|-|-|-|-|-|-|\n|**DiffuScene (w/ 3D-Synth)**|0.080|43.90|7.98|23.8|67.3|98.3|25.79|70.2|\n|**LLplace (w/ 3D-Synth)**|0.058|39.88|7.86|31.3| 97.8|98.8 |12.86|72.4|\n|**Ours**|0.025|33.68|8.90|50.0|98.6|99.6|8.58| 88.7|\n\nAs shown in Table 2, our method outperforms DiffuScene and LLplace across all key metrics, achieving lower overlap and FID, higher GPT‑4o scores and usability, as well as stronger CNT, NAV, OOB, and ACC performance. This demonstrates the effectiveness and robustness of our proposed approach. We will include all of these metrics in the appendix of the future version. However, scene generation research still lacks a clear, unified benchmark, so we are developing a new evaluation suite that builds on our current metrics by adding curated test sets with standardized rendering protocols, rigorous FID comparisons under consistent conditions, and a CLIP‑based evaluator fine‑tuned on paired scene images and text instructions—together providing a more robust and reproducible standard for future work.  \n\n**R1Q4: Does the structured JSON proposed here bring any advantage over other very similar representations proposed in related works?**\n\n**Response**:  \n\nThank you for the question. We chose a structured JSON format for representing 3D scene layouts because it offers an **effective balance between human readability and machine interpretability**. The format uses clear key-value pairs such as \"x\": 1.2 and \"rotation\": 90, which are easy to read and understand. At the same time, JSON supports schema validation, type checking, and straightforward extensibility, allowing new fields to be added without disrupting existing workflows.\n\nUnlike formats such as GLB or USD, which tightly integrate geometry, textures, and scene graphs, JSON separates spatial properties from rendering details. This separation makes it particularly suitable for large-scale layout generation and editing, where assets can be dynamically loaded and replaced in engines like Isaac Sim or Blender.\nIn contrast to CSS-style in LayoutGPT, 2D layout representations that rely on properties such as left, top, width, and height and often require complex transformation logic to describe spatial arrangements, JSON provides a more precise and semantically rich encoding of full 3D information, including position, rotation, and size. It is also inherently more extensible, allowing new spatial attributes or metadata to be added without altering the existing structure.\n\nMost importantly, JSON serves as a practical intermediate format between natural language and executable code. It enables LLM to convert user instructions into structured scene layouts that can be directly used in downstream tasks such as 3D modelling and rendering in Blender or Isaac Sim."}, {"review_id": "Mp30CJYASI", "reviewer": "Reviewer_LV9F", "summary": "This paper aims to the problem of scene layout generation from a natural-language description of a set of indoor objects. It introduces **OptiScene**, an LLM‑based framework fine-tuned through SFT and RL (DPO [22]). Additionally, the manuscript also introduces **3D-SynthPlace** a novel dataset constructed from Holodeck[35], significantly extending 3D-Front[9] dataset.", "questions": "* Q1: in L302, Object-Centric navigation experiments are presented. The experiment description defines a command \"find a <object> in the room\" to execute a robot exploration. Results are presented in Tab 5. Can the authors elaborate on how the robot/agent explores the scene to find the object? If the robot does not use the proposed **OptiScene** why this experiment is evidence that the proposed solution allows the downstream task of navigation?\n\n* Q2: The ablation study in Tab. 6 indicates that prompting with a high-level description (reasoning) significantly boosts performance (as pointed out in L314). I’m curious whether other formats of high-level input, such as a JSON file with some centroids or even visual inputs (image in BEV), would have a similar effect or not. How might the results change with these different formats?\n\n* Q3: How the proposed solution performance with more complex scene layouts? For instance 5, 10, 20, 30 objects. Is there any limitation in this matter.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "# Strengths\n* The paper is well-written and explain clearly the motivation behind the proposed solution. It offers a clear view of the current modules and contributions, and effectively evaluates the proposed method/solution in a suitable benchmark with valid baselines for scene layout generation.\n\n# Weaknesses\n* The current manuscript fails at explaining and discuss the limitation of the current solution. For instance, more complex scenes (specifically more objects) and un-seen items (objects/items no presented during training).", "was_revised": true, "final_justification": "I partially agree with the authors about OptiScene’s role in the navigation task as presented in the current manuscript. From my understanding, using OptiScene increases the amount of training data available for the navigation tasks (as the authors mentioned in their rebuttal, ' it helps expand the data space for training and testing'). However, the experiments showed in the manuscript and in the rebuttal do not convey this idea. OptiScene was mainly used for testing predefined solutions for navigation. The impact of OptiScene for training was not presented. Additionally, it remains unclear to me how OptiScene may helps for navigation task in larger environments, with multi-rooms, corridors, and multiple areas, which is the main setting for the embodied navigation tasks.\n\nNevertheless, based on the authors' response R2Q3, I want to emphasize the clear contribution of this paper. Although, the proposed solution’s performance declines as the number of objects increases (i.e., which is totally expected), it still significantly outperforms the baselines. The selected baselines are both appropriate and fair, and the current results provide strong evidence for the manuscript’s contribution.\n\nEven though, there are some minor flaws, I keep my rating as borderline accept.", "rebuttal": "We thank Reviewer 2 for the clear summary and positive evaluation. We appreciate your recognition of our motivation, modular design.\nBelow, we address your concerns and provide clarifications on the specific points you raised.\n\n**R2Q1. Clarification on Object-Centric Navigation (L302)**  \n  \n  **Response**:  \n  \nHere is our answer about navigation with LLM:\n\nWe first provide clarification on the navigation setting as follows:\n  \n  For navigation experiments, we adopt the settings in Benchmark 1: Object Loco-Navigation in InfiniteWorld [1]. Specifically, the process by which a robot/agent explores a scene to searches for a target object is as follows: First, a 2D grid representation of the environment is constructed through an occupancy map, which divides the space into free zones, obstacle zones, and unknown zones. This grid serves as the foundational spatial reference for subsequent path planning, enabling the agent to identify navigable areas and avoid obstacles. Then, natural language instructions such as \"Find an [object] in [room]\" are parsed: LLM extracts semantic information from the instruction to determine the target object (e.g., \"black bedside table\") and its approximate location clue (e.g., \"in the bedroom\"), and further queries the scene's built-in object coordinate data or real-time scene graph to obtain the specific coordinates of the target object. Subsequently, the path follower, based on the 2D grid, uses an algorithm to plan the optimal path from the agent's current position to the target coordinates. If the target coordinates fall within an obstacle zone, the nearest non-colliding point is selected as an alternative target. During movement, the agent proceeds along the planned path, with the final requirement that the target object is within its 60-degree horizontal field of view and within 2 meters. If these field-of-view conditions are not met due to occlusion or other reasons, the task fails.\n  \n  Then we discuss the connection from navigation to OptiScene:\n\n Although the navigation algorithm itself is not modified, we evaluate it on scenes generated by OptiScene. Since indoor datasets suitable for such embodied tasks are limited, our method helps expand the data space for training and testing. The fact that existing navigation algorithms can be directly applied to scenes generated by OptiScene, and achieve comparable behavior, validates that our synthetic scenes are well-aligned with real ones and usable for downstream embodied tasks such as navigation.  And also, we think navigation is one kind of way to evaluate the usability of the generated indoor scenes. As in **R1Q3**, we introduce navigability to test models' performance. Although navigability is different from the object loco-navigation results, they both show the overall spatial rationality of the scene.\n  \n  Ref:\n  \n  [1] Ren, P., Li, M., Luo, Z., Song, X., Chen, Z., Liufu, W., ... & Liang, X. (2024). Infiniteworld: A unified scalable simulation framework for general visual-language robot interaction\n  \n**R2Q2. Is the semantic reasoning step part of the input prompt or the model’s internal reasoning process? Would alternative high-level inputs (e.g., fixed positions, centroids, BEV images) also improve performance?** \n\n**Response:**\n\nThanks for your question! For the first part of the question, we apply the semantic reasoning because we believe that if we let the model try to understand the relationship between each other can help to generate more reliable 3D scenes. So we construct this data in the training dataset and **make the semantic reasoning as a step in the chain of thought before outputting the final 3D scene result**. So this is not a part of the prompt for the model; we just try to let the model learn object-object relationships of the scenes.\n\n And for the second part of the question. As we understand, the reviewer would like us to test fixing the positions of certain objects as the input. For example, when generating a bedroom, first specifying the bed’s position and then predict the positions of the remaining objects. We believe this experiment is interesting and clearly demonstrates our model’s robustness, so we performed inference on four room categories while fixing the positions of some selected objects (e.g., Bed, sofa, bathtub...). This setup also aligns closely with the downstream task of scene editing we discussed in the paper. In addition to directly performing inference with fixed object positions, we further included a fine-tuned version of our model specifically trained for editing as a baseline comparison.  And we provide the quantitative results based on key metrics, Object Overlap Rate (OOR), GPT4o, and Usability:\n\n**Table 3: Compare the Fixed Position OptiScene with other settings** \n| Condition| avg. OOR ↓| avg. GPT4o↓ | Scene Usability (%) ↑ |\n|-|-|-|-|\n| Original OptiScene | 0.023 | 8.7| 50|\n| Editing OptiScene  | 0.028 | 8.7| 50|\n| Fixed Position OptiScene | 0.045 | 8.2| 40|\n\nAs shown in Table 3, even when fixing the positions of certain objects during inference, the model maintains good performance with only a slight drop, which demonstrates our model's robustness.\n\nAdditionally, we conducted further experiments using highly specific user instructions that include object relationship descriptions (see **R3Q4** for details). The results show that, without any re-training, our model can leverage the generalization ability of the LLM to interpret such structured instructions and still generate reliable 3D scenes.\nBut our model is designed to use an LLM to generate spatial placements of indoor objects and does not currently integrate other modalities (like 2D image or 3D point cloud) or change the overall architecture to help the scene generation. We agree that introducing other modalities is an exciting direction, and we plan to explore it in future work.\n\n\n**R2Q3. How do the proposed solutions perform with more complex scene layouts? For instance, 5, 10, 20, 30 objects. Is there any limitation in this matter?**\n\n**Response:**\n  \nThank you for your insightful question. Evaluating the robustness of our model under varying scene complexity is indeed crucial. To address this, we conducted comprehensive experiments on scenes containing 5, 10, 15, 20, and 30 objects, sampling 20 living-room scenes for each scenario and evaluating performance using Object Overlap Rate (OOR), FID scores, GPT-4o ratings, and usability metrics. We compared our results against several baselines, including DiffuScene, LLPlace, and Holodeck. While Holodeck's OOR always be 0 due to its processing step that enforces, we won't compare Holodeck's OOR in the table. \nAs summarized in the table below, our method consistently outperforms baselines (DiffuScene, LLPlace, Holodeck) across most complexity levels, maintaining strong performance up to 15 objects. However, at higher complexities (20+ objects), especially at 30 objects, the model exhibits significant performance drops (e.g., OOR rises to 0.289, usability drops to 0%), clearly indicating a limitation.\nWe attribute this decline primarily to the scarcity of densely-packed scenes in our training data, resulting in an out-of-domain scenario at very high object counts. This limitation is prevalent across other state-of-the-art methods as well as Holodeck, for example, completely fails to generate coherent layouts at 30 objects, even after we change the code to enlarge room sizes to accommodate them.\n\n**Table4. Performance Across Object Counts (avg. OOR ↓ / avg. GPT‑4o ↑ / UR ↑)**\n\n|Method|5 Objects|10 Objects|15 Objects|20 Objects|30 Objects|\n|-|-|-|-|-|-|\n|**Ours**| **0.020 / 9.3 / 75** | **0.047 / 8.7 / 60** | **0.088 / 8.1 / 35** | **0.154 / 7.0 / 20** | **0.289 / 6.8 / 0** |\n|DiffuScene|0.075 / 8.3 / 45 | 0.082 / 8.0 / 45  | 0.093 / 7.6 / 30  | 0.211 / 6.5 / 10 | 0.348 / 6.1 / 0  |\n|LLplace|0.072/8.5/40| 0.085 / 7.8 / 35  | 0.129 / 7.6 / 25  | 0.186 / 6.3 / 5 | 0.292 / 5.5 / 0  |\n|Holodeck|- /8.9/65| - /8.5/60|- /7.8/30| - /7.3 / 25|- / 6.9 / 0|\n\n\nAs for the unseen objects question. We discuss the concern about how our model handles previously unseen object descriptions as follows:\n- (1) Retrieval-based Generalization:\n As discussed in **R3Q4** and **R4Q1**, our model relies on a retrieval module that maps natural language object descriptions to corresponding 3D assets. This module achieves a 98% retrieval accuracy for matching object classes, ensuring reliable selection even for unseen or varied descriptions.\n- (2) LLM Generalization + Test-time Coverage:\n The LLM demonstrates strong generalization ability in interpreting semantically novel or rare descriptions (e.g., mapping “seats” to “chair”).\n Moreover, our test set already contains many object descriptions not seen during training, and the model successfully handles them, confirming its robustness to unseen inputs.\n- (3) Focus on Large Floor-based Furniture:\n Our task targets large furniture items that are placed directly on the floor (e.g., beds, sofas, wardrobes), rather than small or decorative objects. This reduces ambiguity and improves consistency in object grounding.\n- (4) High-quality 3D Asset Pool:\n All retrieved assets come from our annotated 3D database based on Objaverse, ensuring sufficient diversity and realism to support generalization.\n\nIn summary, our model demonstrates robustness and quality in moderately complex scenes, yet encounters significant challenges at extremely dense object arrangements with an extremely large number of objects (>20). \nTo improve performance in such scenarios, expanding the training set to include richer and denser configurations, combined with targeted data augmentation strategies, will be essential future directions.\nMoreover, we believe our model is capable of handling unseen objects during inference in the context of indoor scene layout generation."}, {"review_id": "1AkYeNsq78", "reviewer": "Reviewer_xtFJ", "summary": "The paper introduces OptiScene, an LLM-based framework for preference-aware indoor layout generation. Unlike prior prompt-driven approaches that struggle with controllability and physical understanding, and learning-based approaches that suffer from scarce data sources leading to a lack of expressiveness, OptiScene addresses these concerns by post-tuning an LLM for the spatial design task using the proposed large-scale 3D-SynthPlace dataset.\n\nThis is achieved in two steps: The SFT stage introduces a high-level semantic reasoning step before layout generation, while the two-stage DPO pipeline helps align layout generation with human preferences and physical constraints.", "questions": "All of my questions are listed in the weaknesses section, and I may adjust the rating if they are well addressed. Specifically:\n\n1) Some parts of the method, particularly the SFT process, require further clarification.\n2) Additional experiments could be conducted to evaluate the method’s behavior in challenging cases (e.g., valid overlapping bounding boxes) and with complex, ambiguous free-form instructions reflecting real-world scenarios.\n3) Potential bias in the training data should be discussed, and generalization experiments should be conducted.", "limitations": "Yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1)\tThe proposed pipeline attempts to combine the strengths of prompt-driven and learning-based methods, and further extends them by aligning the generation process with human preferences.\n2)\tThe paper introduces a synthetic indoor scene dataset called 3D-SynthPlace, which covers a wide variety of layouts along with semantic summaries and scene descriptions, contributing to the field.\n3)\tThe proposed approach demonstrates potential for different applications such as scene editing and object-centric navigation.\n\nWeaknesses:\n\n1)\tMissing details for SFT: Although Section 3.3. discusses the SFT process, some crucial details are missing from the text or require further clarification:  a) L195-196: How is the supervision provided to minimize the discrepancy between predictions and ground-truth layouts, including both the semantic summary and the structured output? b) How is the user instruction U converted into the JSON specification X? c) Generally, object retrieval from the dataset is performed using the Euclidean distance between the predicted bounding boxes and the bounding boxes of the dataset assets, as well as category information. However, Equation 1 appears to use object descriptions for retrieval. How this is achieved is not clear.\n2)\tValid overlapping bounding boxes: DPO Stage 2 and the way prompts are constructed (L196–197) are designed to avoid object collisions. However, real-world scenarios often include cases where valid overlapping bounding boxes naturally exist (e.g., a chair underneath a desk). How does the method handle these cases? Is it possible to include them as challenging positive examples?\n3)\tPossible bias in the training data: a) Despite the advanced filtering, the training dataset relies on Holodeck’s predictions and may inherit its biases. b) During DPO training, negative samples are generated by the SFT model itself, which could lead to similar error patterns and limited diversity.\n4)\tLack of ablations on user instruction complexity: No evaluation is performed showing how the free-form instructions which may exhibit high complexity and ambiguity affect the generations.", "was_revised": true, "final_justification": "The authors have addressed my questions and provided further clarifications, which resolved most of my concerns. I am inclined to accept this paper. I have recommended that the authors include these discussions and experiments in the final version to strengthen the work.", "rebuttal": "We sincerely thank you for the encouraging feedback and thoughtful summary. We are pleased that you found our dataset, two-stage DPO framework, and overall method design valuable.\nBelow, we address your comments and respond to the specific concerns you raised.\n\n**R3Q1. Clarify key missing details in the SFT stage: supervision methods, instruction-to-JSON conversion, and the object retrieval process?**\n\n**Response:**\n\nWe agree these aspects were not sufficiently clear in our manuscript. We clarify each sub-question explicitly below, and will incorporate these detailed explanations into our revised version:\n\n 1) Supervision during SFT:\n\nDuring SFT, the model jointly generates (A) a semantic summary $S^{\\text{sem}}$ and (B) a structured JSON layout with 3D coordinates $c_{1\\ldots N}$ and orientations $r_{1\\ldots N}$. We serialize both outputs into token sequences and apply token-level cross-entropy loss against human-annotated ground truth. This unified supervision encourages the model to capture both global semantic intent and precise spatial configurations in a single forward pass.\n\n2) Instruction-to-JSON Wrapper:\n\nWe use a Qwen2.5-7B-Instruct LLM as a wrapper to convert natural language user instructions $\\mathcal{U}$ into structured JSON $\\mathcal{X}$. The process works as follows:\n- The wrapper extracts **room size**, **room type**, **object types**, and **object counts** from the instruction.\n- If any field is missing, the wrapper heuristically infers it based on existing information (e.g., number and type of objects).\n- It then retrieves suitable 3D assets for each object (details in part 3).\n- The extracted values are inserted into a predefined JSON layout template, which is then used as part of the prompt input to OptiScene for inference.\nThis wrapper provides robustness by normalizing diverse instructions, modularity for easier debugging, and flexibility through noise filtering. It is also easily extensible—indeed, we further extended it in R3Q4 to extract spatial relationships between objects.\n\n3) Object Retrieval Process:\n\nWe follow a purely text-based retrieval approach, inspired by HOLODECK (Yang et al., 2024c), to match the objects mentioned in the user instruction to assets in our 3D object database.\nDue to the character limitation, please refer to the **R4Q1** for more details, where we discuss the object retrieval process and show the performance test of the text-based approach. \n\n**R3Q2. How does the method handle naturally valid bounding-box overlaps? Are these explicitly modelled?**\n\n**Response:**\n\nCertain overlaps (e.g., chairs under desks) naturally occur in realistic scenes, and explicitly handle these cases as follows:\n- Dataset Preservation: We intentionally retain the valid overlapping layouts (e.g., chairs tucked beneath desks) in our synthetic dataset—474 scenes (2.8% of total), predominantly in living rooms—allowing the model to recognize natural overlap distributions without skewing its collision avoidance capability.\n- Prompt Engineering (Appendix Table 7): Our prompts explicitly instruct minimal distances for specific object pairs (e.g., chair–desk), guiding the model to correctly interpret these overlaps as intended and acceptable rather than as collisions.\n- Stage 2 DPO Noise Injection: During DPO Stage 2, we preserve chair–desk overlaps as positive examples, avoiding random perturbations on these pairs. Conversely, overlaps involving inappropriate object pairs (e.g., sofa–sofa intersections) remain labelled as negative examples.\nThus, our approach robustly differentiates between valid overlaps and unintended collisions, ensuring realistic, high-quality layout generation.\n\n**R3Q3. How does the method handle biases inherited from Holodeck and limited negative sample diversity during DPO training?**\n\n**Response:** \n\nWe clarify that our training dataset is **not directly derived from raw Holodeck outputs**. Instead, we carefully filtered and applied handcrafted refinements to ensure data quality and alignment with human preferences. Nonetheless, we acknowledge the possibility of **residual biases** from the Holodeck remaining in our data, which we addressed systematically:\n- Manual Selection of Positive Samples: During the DPO stage 1, we manually selected layouts that closely align with human preferences, ensuring positive samples specifically counteract residual biases inherited from Holodeck and original 3D-FRONT datasets.\n- Two-stage DPO Training: To enhance the diversity and effectiveness of negative samples, we introduced a DPO Stage 2. We deliberately applied controlled perturbations—including slight random displacements, overlaps, and out-of-bound placements—to relatively high-quality layouts, increasing the model’s ability to robustly differentiate valid and invalid configurations.\n- Temperature-based Diversity for Negative Samples:  Additionally, during DPO Stage 1, we generated negative samples by varying the sampling temperature from the SFT model. Consistently similar negative outputs at different temperatures indicate persistent errors that require repeated exposure, whereas varying outputs suggest multiple error patterns beneficial for robust training. Moreover, using SFT-generated negative samples for DPO training is a validated approach, as demonstrated in prior works [1,2].\n\n[1] Enhancing LLM Reasoning with Iterative DPO: A Comprehensive Empirical Investigation\n\n[2] Iterative reasoning preference optimization\n\n**R3Q4. How does instruction complexity affect generation performance?**\n\n**Response:** \n\nTo support consistent layout generation across varying instruction types, we first enhance the wrapper, which parses natural language instructions into structured JSON representations by extracting room attributes, object descriptions, and (when available) object-to-object spatial relationships.\nWe design the wrapper to handle two main cases:\n- For highly specific and directional prompts, we use the wrapper to extract spatial relations and then encode them as **[objects relationship]** tokens, which are injected at inference time to guide generation without requiring retraining.\n- For vague or underspecified prompts, where explicit object information is lacking, we rely on the wrapper to creatively infer plausible object categories and quantities from contextual cues.\n\n**Instruction Complexity Evaluation** \n\nWe first assessed the model’s robustness to instruction complexity by categorizing prompts into three levels: Simple (≤5 objects, precise object description and number), Medium (6–15 objects, moderately vague description with some object relationships), and Complex (15-20 objects, highly ambiguous). \nEach group contains 50 prompts evaluated on living room scenes. The wrapper and retrieval process achieves over 95% parsing success across all levels. However, under complex or vague instructions such as “some chairs” or “a few shelves,” the number of extracted objects tends to vary (typically between 2–4), leading to decreased layout reliability.\n\n**Table 5. Generation performance under different instruction complexity levels** \n| Instruction Level |  avg. OOR ↓ | avg.GPT-4o ↑ | UR (%) ↑ |\n|-|-|-|-|\n|Simple|0.021|8.9|70|\n|Medium| 0.087 |7.9|45 |\n|Complex| 0.186 |7.1| 30 |\n\nDespite the increased difficulty, our model maintains reasonable performance even under complex instructions, achieving an average GPT-4o score of 7.1 and a usability rate of 30%. This demonstrates the robustness and stability of both our wrapper and the generation model when handling ambiguous or underspecified inputs.\n\n**Instruction-Following with Extreme Prompts** \n\nWe further evaluate how the model handles two extreme instruction types:\n (1) Highly specific and directional instructions,\n (2) Extremely vague and underspecified instructions.\n Each of these settings was evaluated over 20 test cases across four room categories.\n\n**Directional Instructions** \n\nTo evaluate the model's ability to follow directional prompts, we measure both scene-level and instruction-level relation success rates, along with OOR and scene usability, as summarized in Table 6.\nA scene is considered successful only if all specified spatial relations are satisfied. Under this criterion:\n- 12/20 scenes met all spatial constraints (60% scene-level success)\n- 30/46 individual relations were followed (65.2% instruction-level success)\n\n**Table 6: Performance on directional prompts** \n   | Metric | Value  |    \n   |-|-|\n   | Scene-level Relation Success Rate|60%|\n   | Instruction-level Relation Success Rate | 65.2% |\n   | OOR |  0.056 |\n   | Scene Usability |  40% |\n\nWe also evaluated single-instruction examples. For instance:\n- “Place the desk parallel to the bed” → correct placement in 66.7% of 12 cases\n- “Place the chair next to the desk” → correct placement in 80% of 20 cases\n\nThese results suggest that, despite not being explicitly trained on spatial language, the model exhibits a limited but notable understanding of spatial semantics. However, the model often fails with instructions involving corner placements, likely due to the absence of explicit geometric encoding of room corners.\n\n\n**Vague Prompts**\n\nTo assess robustness under ambiguous inputs, we tested prompts containing vague quantifiers (e.g., “some chairs”) or high-level intents (e.g., “a cozy living space”). We report JSON parse success and scene usability:\n\n**Table 7:Performance on vague instructions** \n   | Metric | Value |\n   |-|-|\n   |JSON Parse Success Rate|65%|\n   |Scene Usability|35%|\n\nIn 65% of cases, the wrapper generated valid object layouts, but many still suffered from usability issues (e.g., missing object types or implausible combinations).\n The remaining failed parsing due to overly abstract input or absence of essential categories (e.g., no sofas in a living room).\nWhile the wrapper handles moderately vague prompts reasonably well, it struggles with highly underspecified instructions. We plan to explore this direction in future work."}, {"review_id": "rrMi6B7sap", "reviewer": "Reviewer_1Sit", "summary": "This paper proposes OptiScene, an indoor scene layout generation method based on the LLM. The authors start by constructing a large-scale indoor scene layout dataset, which contains about 17,000 scenes and covers four room types. Based on the dataset, they develop a two-stage training strategy, including supervised fine-tuning and multi-turn direct preference optimization, to produce structured layout representations from instructions. The results demonstrate the superiority of OptiScene over the prompt-driven and learning-based baselines. They also extend OptiScene to handle other relevant tasks, such as scene editing and robot navigation.", "questions": "1. Could the authors elaborate on the implementation details of the object retrieval module? Error accumulation maybe a key issue in current framework. That is to say, if the retrieval accuracy is not high, the final output will not conform to the user instruction.\n2. Following up on the previous question, current evaluation metrics lack the assessment of the consistency with user instructions. How well can the output layout meet the user needs?\n3. It Tab. 6, the introduction of semantic reasoning brings little performance improvement. Do the results mean that the semantic reasoning in SFT is not that necessary? Could the authors explain the results? It would be great if the authors could share the quantitative results of \"SFT+DPO1+DPO2\" without \"Reasoning\".\n4. How is the layout editing task achieved using the layout generation model? Either the dataset construction and format is not clear.\n5. The FID and GPT-4o Ratings are missing in the ablation studies.\n6. In DPO-stage II, the authors add noise to GT layouts to produce negative samples. What is the noise level used? Do the authors study the effect of different noise levels? Are the negative samples produced by large noise better than those produced by small noise? It would be great if the author could have a discussion about this.\n7. Does the current model support layout generation based on object relationships (e.g., the bed is to the east of the desk)? Users sometimes also specify the relative positions of the objects rather than just the types of the objects.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The curated large-scale dataset, 3D-SynthPlace, would be valuable for indoor layout generation.\n- Two stage training of SFT along with DPO achieves layout generation effectively. Experimental results show that OptiScene is the SOTA method in this task.\n- The introduction of the two-stage DPO framework, including human consensus alignment and geometric plausibility enhancement, makes the generated layouts closer to human preference and more geometrically valid.\n- Paper writing is clear and easy to follow.\n\nWeaknesses:\n- Some notations and equations are confusing. For example, in Eq. 2, the LLM outputs $X$, which represents the input instruction. Besides, Does the so-called \"bounding box\" refer to the component size? It seems very easy to confuse with the concept of \"layout\" (i.e., $c$).\n- The paper lacks some implementation details. For example, it is unclear how the object retrieval and layout editing (data construction, data format) are achieved.\n- Introducing high-level semantic reasoning into the model output seems not effective.\n- The ablation studies seem not thorough. Only two quantitative results are reported in this part.\n- OptiScene generates layouts based on object types, but in real scenarios, users may also have requirements for the relative relationships between objects.", "was_revised": true, "final_justification": "I am satisfied with the authors' rebuttal. In discussions with the authors, they also addressed my other concerns. I encourage the authors to include additional experiments and analyses in the final version. I will maintain my rating as borderline accept.", "rebuttal": "We are grateful to Reviewer4 for the thoughtful assessment and positive evaluation of our work. We appreciate your recognition of 3D-SynthPlace, the two-stage DPO framework, and the clarity of our presentation.\nWe now address your questions and concerns in detail.\n\n**R4Q1. How is the object retrieval module implemented, and how will it influence final scene quality?** \n  \n  **Response:** \n  \n  We follow a purely text-based retrieval approach, inspired by HOLODECK (Yang et al., 2024c), to match the objects mentioned in the user instruction to assets in our 3D object database:\n  \n1. Let $t'$ be the target object’s name from the wrapper, and $t$ be the annotation of a candidate 3D asset from the re-annotated the Objervase dataset.\n  \n2. We compute cosine similarity between their sentence embeddings using SBERT (`all-mpnet-base-v2`):\n  \n   $$\n   T(t, t') = \\cos\\big(\\mathrm{SBERT}(t),\\,\\mathrm{SBERT}(t')\\big)\n   $$\n  \n3. The asset $o$ that yields the highest similarity is selected as the retrieved item for placement.\n  \nThis ensures semantic coherence even without explicit spatial constraints, making our retrieval robust against variations in descriptive text.\n  \n**Retrieval Performance:**\n\nThis retrieval approach has been validated within both Holodeck and LLPlace, and we have conducted dedicated tests on our retrieval module:\n- Explicit type queries (98% accuracy): When the input query explicitly specifies an object category (e.g., “sofa”, “desk”), the system returns the correct category as the top result 98% of the time.\n- Rich, free‑form descriptions:  \n  - Top‑1 retrieval accuracy (86%): In 86% of cases, the single highest‑scoring asset exactly matches the ground‑truth asset.  \n  - Top‑3 retrieval accuracy (94%): In 94% of cases, the ground‑truth asset appears somewhere among the three highest‑scoring candidates, providing a small ranked shortlist for follow‑up selection. \n\nTherefore, we believe the retrieval step has minimal impact on overall performance. Our 3D asset library is based on Objaverse, with refined and precisely annotated object labels, and is easily extensible beyond the 3D-SynthPlace dataset. The wrapper filters and standardizes user instructions, extracting clear room and object specifications. Retrieval then uses these parsed descriptions to match suitable assets. As long as the object type is clearly stated, retrieval remains reliable. For details on the wrapper, please see our response to **R3Q1(2)**. \n\n**R4Q2. How is consistency between the generated layout and user instructions evaluated?**\n\n**Response:**\n\nWe fully understand your concern. In response to **R3Q1(2)**, we have provided a detailed explanation of how the Qwen2.5-7B wrapper is used to parse natural language instructions into a standardized JSON template, and extract information. This design helps eliminate redundant or irrelevant information and ensures a consistent input format. \nAs further discussed in **R4Q1**, our text-to-asset retrieval strategy reliably maintains semantic alignment between the input and the generated layout.\nWe also evaluated the model’s instruction-following ability under both **highly specific** and **vague prompts** (**see R3Q4**), assessing its robustness to varying instruction granularity and complexity.\n\n**R4Q3. Is semantic reasoning in SFT necessary, given the small performance gain shown in main paper's Table 6?**\n\n**Response:**\n\nThank you for this insightful question. It is true that the performance gap between SFT with and without semantic reasoning appears small in terms of OOR. As shown in Table 6, the object overlap rates are nearly identical, and scene usability improves by only about 5 percentage points with reasoning.\nThis is largely because, in the SFT-only setting, any object overlap renders the scene unusable. As a result, OOR dominates usability, and semantic reasoning has limited impact on the OOR metric itself.\nHowever, the reasoning step provides high-level spatial priors—descriptions of relative object positions—that guide the model toward more coherent and human-aligned layouts. While OOR may remain similar, this reasoning significantly improves overall scene structure and layout plausibility, leading to higher usability.\nTo further clarify, we include below the quantitative comparison of SFT+DPO1+DPO2 with and without semantic reasoning:\n\n**Table 8:Compare the performance between with/without semantic reasoning** \n| SFT+DPO1+DPO2 | avg. OOR ↓ | UR (%) ↑ |\n|-|-|-|\n| with semantic reasoning | 0.023| 50|\n| without semantic reasoning  | 0.029| 42|\n\nThese results show that although the OOR only slightly drops without reasoning, scene usability drops noticeably. This confirms that semantic reasoning helps the model produce more usable and human-preferred layouts, even if the overlap metric is not significantly affected.\n\n**R4Q4. How is the layout editing task implemented, and how is the dataset constructed and formatted?**\n\n**Response:**\n\nThank you for your question. To enable layout editing using the same generation model, we constructed a dialogue-style editing dataset inspired by LLplace. Each original scene with more than four objects was converted into a two-turn dialogue:\n1. In the first turn, we randomly removed a subset of objects (40% probability per object, excluding essential items like beds, tables, and sofas) to create a corrupted layout.\n2. In the second turn, we either added or removed these objects depending on the task type.\nFor the add task, we provided the corrupted scene and the list of removed objects as input (add=True), expecting the model to reconstruct the original full scene.\n For the remove task, we gave the original scene and the target objects to remove (remove=True), expecting the model to generate the edited scene.\nEach sample includes structured object attributes (category, position, rotation) and uses 
 to separate dialogue turns. In total, we created 5000 editing samples (3000 add, 2000 remove). The input format mirrors our original generation format, with added flags and object lists to indicate the editing intent.\nTo adapt the model for editing, we further trained it for 5 additional epochs on this dataset, continuing with the original model. This improves the model’s ability to handle scene edits in a conversational context.\n\n**R4Q5. More ablation study results with GPT4o and FID.**\n\n**Response:**\n\nWe provide a complete ablation study that includes FID and GPT‑4o scores to assess the contribution of each training component and the effect of high-level：\n  \n  **Table 9: More Ablation Study with FID and GPT4o Score** \n  | Only SFT | Reasoning | SFT+DPO1| SFT+DPO1+DPO2 | OOR ↓ | UR ↑| FID ↓ | GPT4o ↑ |\n|-|-|-|-|-|-|-|-|\n| ✓ | - | - | - | 0.049 | 28%  |41.84|8.1|\n| ✓ | ✓ | - | - | 0.048 | 33%  |39.61|8.3|\n| ✓ | ✓ | ✓ | - | 0.033 | 40%  |37.24|8.3|\n| ✓ | - | ✓ | ✓ | 0.029 | 42% |35.51|8.6|\n| ✓ | ✓ | ✓ | ✓ | **0.023** | **50%** |**34.32**|**8.9**|\n  \nAs shown in the updated table, after incorporating both FID and GPT‑4o scores, the model's performance continues to improve progressively across different stages, exhibiting a clear stepwise enhancement. Each component contributes meaningfully to the generation quality,  validating the overall design of our pipeline.\n\n**R4Q6. What noise levels are used in DPO Stage II, and how do different levels affect performance?**\n\n**Response:**\n\nFor this experiment, we tried three noise levels.  Our perturbations covered several types—objects placed outside the room, overlapping, and random placements.\n Before settling on the current medium setting, we evaluated a heavy setting (severe overlaps, large out‑of‑room shifts, or fully random layouts) that made the preference gap between positive and negative samples too wide. We also tried a light setting, where objects were moved only slightly; this made the positive‑negative gap very small.\nWe trained with each noise level and evaluated 100 scenes per setting with OOR and UR:\n\n**Table 10: Impact of Noise Levels on OOR and UR in Stage-II DPO**\n| Noise level | OOR ↓ | UR ↑  |\n|-|-|-|\n| Heavy       | 0.048 | 34% |\n| Medium | 0.023 | 50% |\n| Light       | 0.035 | 39% |\n\nWhile heavy noise leads to a sharp drop in OOR, it also results in the lowest usability. We believe this is because when negative samples are extremely poor, the model quickly learns to avoid only the most obvious errors. This causes it to converge prematurely, without learning to handle more nuanced spatial relationships—ultimately leading to layouts that still contain moderate overlaps or awkward placements.\nOn the other hand, light noise provides minimal supervision beyond Stage I, offering little signal for further refinement. As a result, performance shows limited improvement and may even slightly degrade.\nThe medium noise setting strikes the best balance: it introduces enough discrepancy to guide learning while avoiding extremes that cause saturation or instability. This produces the most effective training signal, yielding the best overall performance in Stage-II DPO.\n\n**R4Q7. Does the model support layout generation based on object spatial relationships?**\n\n**Response:**\n\nThank you very much for this question. It also serves as a test of the model’s instruction-following capability. When processing user instructions through our wrapper, users may wish to specify object-to-object relationships as priors for the model input prompt. We have already discussed this issue in our responses to **R3Q4**, which we kindly refer you to for more details.\n\n**R4Q8: Some notations and equations are confusing.**\n\n**Response:**\n\nThanks for the helpful suggestion. We acknowledge that some notations may be ambiguous or inconsistent. We will revise and standardize the notation in the final version to ensure clarity and avoid confusion with related concepts."}]

Author rebuttal:

Meta-review: The paper proposes OptiScene, a new system for indoor scene layout generation from natural-language description. OptiScene is an LLM-based framework that is fine-tuned through supervised fine-tuning (SFT) and RL (specifically, 2-stage Direct Preference Optimization [DPO]). It is trained on their 17,000 scene dataset called 3D-SynthPlace that is based on the 3D-Front dataset and Holodeck-generated synthetic layouts. OptiScene also has the potential to be used for scene editing and object-centric navigation.

Pros: OptiScene’s novel approach combines the strengths of prompt-driven and learning-based methods, and aligns the generation process with human preferences. The paper also introduces a new and useful dataset 3D-SynthPlace. SOTA results validate the design decisions.

Cons: Though the architecture in combination with its application space is new, the basic ideas of SFT memorizes and RL generalizes are not new. Also, using LLMs (e.g., GPT-4o) as layout evaluators via 1–10 rating scales tends to be unreliable. (This problem may be ameliorated by using additional metrics such as FID and OOR.)

There is consensus among the reviewers to accept. They like the technical contributions of OptiScene and 3D-SynthPlace, and SOTA results shown support their novel system.

The major issues raised by reviewers were addressed by the authors. Some examples: possible bias caused by using Holodeck, performance with varying scene complexity based on object count, no ablation study on GPT4o, FID, and user instruction complexity. Responses: bias is minimized via manual filtering and refinements, and results on scene complexity and the suggested additional ablation studies were given. On the novelty front, the authors reiterated that OptiScene is the first to integrate DPO on top of SFT for 3D scene layout generation. This is in response to a comment that it is well-known that SFT tends to memorize its training data while reinforcement-style methods generalize.
Decision: Accept (poster)