Scenethesis: A Language and Vision Agentic Framework for 3D Scene Generation
Authors: Lu Ling
Chen-Hsuan Lin
Tsung-Yi Lin
Yifan Ding
Yu Zeng
Yichen Sheng
Yunhao Ge
Ming-Yu Liu
Aniket Bera
Max Li
Venue: ICLR 2026 Poster (2026)
Keywords: 3D scene generation; Text-to-3D scene; Agentic framwork; Visual guidance; Physical plausibility
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Generating interactive 3D scenes from text requires not only synthesizing assets but arranging them with spatial intelligence—support, affordances, and plausibility. However, training data for interactive scenes is dominated by a few indoor datasets, so learning-based methods overfit to in-distribution layouts and struggle to compose diverse arrangements (e.g., outdoor settings and small-on-large relations). Meanwhile, LLM-based layout planners can propose diverse arrangements, but the lack of visual grounding often yields implausible placements that violate commonsense physics. We propose Scenethesis, a training-free, agentic framework that couples LLM-based scene planning with vision-guided layout refinement. Given a text prompt, Scenethesis first drafts a coarse layout with an LLM; a vision module refines the layout and extracts scene structure to capture inter-object relations. A novel optimization stage enforces pose alignment and physical plausibility, and a final judge verifies spatial coherence and triggers targeted repair when needed. Across indoor and outdoor prompts,  Scenethesis produces  realistic, relation-rich, and physically plausible 3D interactive scenes, reducing collisions and stability failures compared to SOTA methods, making it practical for virtual content creation, simulation, and embodied AI.

Published as a conference paper at ICLR 2026
Scenethesis:
A LANGUAGE AND VISION AGENTIC FRAMEWORK FOR 3D SCENE GENERATION
Lu Ling!
Chen-Hsuan Lin'
Tsung-Y Yi Lin'
Yifan Ding' Aniket Bera?*
Yu Zeng' Zhaoshuo Li
Yichen Sheng'
Yunhao Ge
Ming-Yu Yu Liu'
!NVIDIA Research
2Prrdue University
ABSTRACT
Generating interactive 3D scenes from text requires not only synthesizing assets but also arranging them with spatial intelligence-supeiee bility. However, training data for interactive scenes is dominated by a few indoor datasets, so learning-basee methods overfit to in-distribution layouts and struggle compose diverse arrangements (c.g., outdoor settings and small-on-lanee large relations). Meanwhile, LLM-based layout planners can propose diverse arrangements, but the lack of visual grounding often yields implausible placements that violate com- monsense physics. We propose Scenethesis, a training-free, agentic framework that couples LLM--bsed scene planning with vision-gus suiion- layout refinement. Given a text prompt, Scenethesis first drafts coarse layout with an LLM; a vision module refines the layout and extracts scene structure to capture inter-object relations. A novel optimization stage enforces pose alignment and physical plausibility, and final judge verifies spatial coherence and triggers targeted repair when needed. Across indoor and outdoor prompts, Scenethesis produces realistic, relation-rich. and physically plausible 3D interactive scenes, reducing collisions and stability failures compared with SOTA methods, making it practical for virtual content creation, simulation, and embodied AI.
affordances, and plausi-
INTRODUCTION
Synthesizing interactive 3D scenes from text is crucial for gaming (Hu et al., 2024), virtual content creation (Ócal al., 2024), and embodied AI (Yang et al., 2024c;b; Deitke al., 2023a; Kolve et al., 2017; Krantz et al., 2020; Nasiriany et al., 2024). Beyond asset synthesis, realistic scenes must go beyond photorealism support spatial intelligence meeliiennee-reasonnngg about support, occlusion, and affordances in editable, physically coherent environments-so objects serve functional roles and afford interaction. Traditional interactive scene generation methods, including manual design (Kolve et al., 2017; Gan et al., 2020; Li et al., 2023a) and procedural approaches (Deitke et al., 2022), are either labor intensive or overly simplified inter-object relations.
Recent learning-based approaches, including layout generation (Paschalidou et al., 2021; Yang et al., 2024b; Tang al., 2024) and multi-instance 3D scene generation (Huang et al., 2025a), learn object arrangement directly trained on CAD dataset such as 3D-FRONT (Fu et al., 2021). However, both families inherit the biases of the available indoor datasets: they are predominantly indoor, skewed toward large furniture, and even contain collisions (Yang et al., 2024b), leaving small/supporting objects and long-tail relations (on-top-of inside / behind) underrepresented. Consequently, eval- uations emphasize in-distribution synthetic splits, and generalization to dittrfffiitrribttinnOOOD layouts-outdoor scenes, rare support relations, and small objects- remains weak.
Meanwhile, LLM-based layout planners expand semantic diversity by leveraging textual common- sense ( (Yang et al., 2024c; Feng et al., 2024; Kumaran et al., 2023), but ungrounded planning often yields placements that violate object function roles and basic physical constraints (e.g., misoriented
loo- author.

---

Published as a conference paper at ICLR 2026
A living room with many reading materials
private billiard room for relax
A spacious warehouse
The miseum exhibition
A omt piano room
warm hospial ward
The pesceful beach daring
relax street with mailbxx
A quiet park bench twilight
Given text prompt. Scenethesis leverages both language and visual priors to generate realistic and physically plausible indoor and outdoor environments.
Figure 1:
seating and blocked openings). Such misplacements break functional affordances and spatial co- herence, reducing the scene a collection of objects rather than coherent layout, and thereby undermining downstream usability.
Motivated by vision foundation models that encode compact real- world spatial priors, we introduce Scenethesis, a training-free agentic framework that integrates LLM planning with vision- vision-guided spatial refinement. Given a prompt, the LLM drafts coarse layout; the vision module refines layout and extracts scene structure to capture inter-object relations. A lightweight optimization then aligns object poses and enforces physical plausibility via semantic correspondence and SDF- contact/support constraints, preventing interpenetration and instability. A final judge verifies spatial coherence and triggers re- re-planning when needed. This design preserves the open-ended diversity of language planning while injecting the visual grounding and physics awareness absent from L.LM- based pipelines without training a new scene-level generator. Figure shows examples of indoor / outdoor scenes generated by Scenethesis. Our contribution is summarized follows.
We introduce Scenethesis, a training-free, agentic framework that integrates LLM planning, vision foundation models, physics-aware optimization, and a judge module to synthesize open-ended, physically plausible interactive scenes.
We propose an efficient physically plausible optimization guided by semantic correspondences and SDF CDF--onstsaints, enforcing collision-free and stable placements without post-hoc simulation. On benchmarks spanning indoor/outdoor prompts and long-tail spatial relations (on-top-of, inside, behind), Scenethesis consistently improves layout realism and physical plausibility relative to state-of-the-an methods, producing diverse scene layouts downstream use.
2

---

Published as a conference paper at ICLR 2026
2 RELATED WORK
Indoor scene synthesis. Early work cast scene synthesis as layout prediction with relation graphs or hierarchical structures (Luo et al., 2020; Chang et al., 2014; Zhou et al., 2019; Li et al., 2019; Wang et al., 2019). Learning-basedd layout generation methods model layout via auto-regression and diffusion (Wang et al., 2021; Paschalidou et al., 2021; Tang et al., 2024; Lin & Mu, 2024; Yang et al., 2024b). Multi-instance scene generation methods leverage strong image-to-3D priors to output assets and poses in one pass (Huang et al., 2025a; Meng et al., 2025; Lin et al., 2025a) to post-optimization. Despite these advances, both families inherit biases from predominantly indoor datasets (c.g.. 3D- AD--FOOT//D-FUTUEE represent smal/supporting objects and long-tail relations (on-top-of/ / inside behind), Consequently, generalization to OOD layouts- scenes and rare support relations underexplored. The recent learning- based approach I-Scene (Ling et al., 2025) exhibits strong generalization, extending scene generation from structured indoor layouts complex outdoor environments with rich spatial relationships, while not explicitly encoding physical constraints in the training objective. In contrast, Scenethesis tackles lexi-to-scent with a training-free, physics-aware optimization loop, aiming for open-ended layout diversity and physically plausible arrangements beyond indoor distributions.
(Fu al., 2021) ): emphasis on large furniture and under-
LLM/VLM Guided 3D Scene Generation. With the advancement of LLMs/VLMs, recent meth- ods (Ócal et al., 2024; Yang et al., 2024c; Çelen et al., 2024; Kumaran et al., 2023; Zhou et al., 2024c; Wang et al., 2023; Aguina- Kang et al., 2024; Lin et al., 2024a; Sun al., 2024; Zhou et al., 2024b) use language models for: (1) spatial relation planning, (2) 3D asset retrieval from semantic descriptions or vision-language embeddings, and (3) rule-based rough collision detection. However, LLM-based planning is ungrounded: it operates in symbolic space rather than metric geometry, and thus often misorients furniture, violates support/clearance constraints, and produces floating or intersecting cesementssespeciiale for small or occluded objects. Template relations in text coarse and miss room-specific structure (Lin & Mu, 2024; Khanna et al., 2024) In contrast, Scenethesis uses LLM priors only to draft coarse layout instructions, then grounds it with vision foundation model to preserve compact spatial information, effectively capturing real-world spatial complexity.
Visual Foundation Model-Guided Scene Generation.
sonnuul-oonntiionmmmoeee pipelines often synthesize 3D scenes by optimizing a single global representation (e.g.. NeRF/3DGS) from 2D priors (Höllein et al., 2023; Fridman et al., 2024; Yu et al., 2024b;a; Zhang et al., 2024a). While photorealistic, these monolithic approaches lack instance structure, hindering per-object editing and interaction. A second line composes scenes via perception - per-object synthesis + geometric fitting (Wang et al., 2024; Zhou et al., 2024a; Dai et al., 2024; Huang et al., 2025b; Rahamim et al., 2024); CAST further adds post-hoc physics-aware pose correction reduce penetration and floating artifacts (Yao et al., 2025). Instead of a post-hoc physical correction stage, Scenethesis is text-to-scene and training-free free, enforcing contact/support constraints inside the pose-alignment loop with andge-and-repair agent-so physics shapes the layout as it is formed, not patched after the fact.
Physics-Aware Scene Generation. Physical principles are often underexplored in interactive scene generation. Most methods approximate geometry with 3D bounding boxes (Yang al., 2024b;c), a coarse proxy that breaks small-on-large relations and causes floating and interpenetration (Çelen et al., 2024). CAST adds physical correction but applies it post-hoc (Yao et al., 2025). In contrast, Scenethesis performs mesh-level collision tests by sampling points on object surfaces and injects SDF- contact/support constraints in-loop, substantially reducing collisions and instability.
3 METHOD
Scenethesis generates spatially realistic, physically plausible interactive 3D environments from user prompts. An overview of the pipeline is shown in Figure 2, consisting of four key stages: (1) an LLM module drafts a coarse scene plan, (2) a vision module refines the layout with visual guidance and structural extraction, (3) physical-aware optimization module distills priors and adjusts object placement for spatial coherence and physical plausibility, and (4) a scene judge module verifies spatial consistency. The following sections detail each module's 's role.
3

---

Published as a conference paper at ICLR 2026
User Prompt:
Vision Module: Layeur vinual refisemer
Optimization Module!
penceful
qutinization
duina
LLM Modull:: souse planning
Cerropasiacce
Beasoting pnsret....
instrction::
Scane graph
Reiriene wen eextnement map
Venibiig dbjecta...
Pasiil Vice
LLM
Selecticg controon objects
syont....
VIM
Grovsded SAM
Deysi Pro
Judge Module: cubherenc juiguwost
Object Hst
Upaampeed prompt This
Boach moxel
Tooch chiir
sowiellle place on the beads mwed... baac
J.J
Limbedla
Vollebblll
TD Geieriann
Seadhell
Eoca of
dear pariisptacd
Imagr gridance
danilase
Figure 2: Scenethesis is an agentic pipeline: an LLM drafts a coarse scene planning, a vision module grounds and refines it, a physics-aware optimizer iteratively aligns poses and enforces contact/support constraints, and judge verifies spatial coherence.
3.1 COARSE SCENE PLANNING
Scenethesis supports either simple prompt (e.g., "a peaceful beach during sunset") for flexible scene generation or detailed prompt for controllable scene generation (e.g., a scene plan describing the detailed spatial relations shown in the appendix). For a simple prompt, the LLM generates a coarse scene plan by reasoning over user input. It first interprets the prompt, reviews all object categories in the 3D database, selects commonly associated objects, and then generates up up-sampled prompt describing coarse spatial relations, as illustrated in Figure 2. When given detailed prompts, the I.M checks for the presence of all specified objects in the database, infers relevant object categories, and skips the prompt up-sampling process.
Following prior work (Yang al., 2024c), the LLM selects anchor object (the top of the scene hierarchy after the ground) and builds a coarse spatial hierarchy by placing other objects relative to this anchor. These relations are folded into an upsampled prompt. Example: cozy living room, the sofa is the anchor at the center; bookshelf sits in the background aligned to the wall; a coffee table and chairs are placed in front of or beside the sofa.
3.2 LAYOUT VISUAL REFINEMENT
Scenethesis exploits the spatial priors implicit in image generators: large-scale training imbues them with common co-occurrences and object arrangements. We use these priors refine the LLM draft via three steps: (1) Image Guidance Generates images to refine spatial relations, ensuring realism and object functionality. (2) Scene Graph Generation Segments objects, estimates depth and 3D bounding boxes (3DBB), and constructs graph encoding inter-object relationships. (3) Asset Retrieval Selects 3D assets and environment maps for final scene composition.
Image Generation. The vision module refines the upsampled prompt into visually grounded image. This generated image serves as the basis for segmentation, depth estimation, and asset retrieval.
Scene Graph Generation. Using VFMs/VLMs (e.g., GPT-5 (OpenAI, 2025), Grounded-S -SMM (Ren al., 2024), DepthPro (Bochkovskii et al., 2024)), we construct a scene graph with 3DBBs and identify structural components, including the anchor, parent, and child objects (see Figure 2). We initialize 5-DoF poses by segmenting each object, lifting with monocular depth, and back- proje-pojgnning to sparse 3D point cloud. Because occlusions, limited viewpoints, and segmentation noise can bias 3DBBs, these poses are only coarse and are adjusted in the optimization stage (Section 3.3).
The scene graph forms the basis for iterative 5DoF pose adjustments during optimization in the next stage. Since Scenethesis targets ground-level layout, mall-mmmd///bccggrouun elements are provided by the retrieved environment map. Details of the scene-graph format appear in the appendix.
Asset Retrieval. Generaaive/reconstruttio pipelines (c.g., 3DGS-bart (Liang et al., 2024; Wu al., 2024)) can be photorealistic but often introduce geometric artifacts and lack edit-ready outputs
4

---

Published as a conference paper at ICLR 2026
(manifold meshes, UVs, decomposable PBR), limiting downstream use. We therefore adopt retrieval: Scenethesis selects category/attribute-nma matcheed assets from a curated, high-quality Objaverse subset (Deitke et al., 2023b) (following (Yang et al., 2024c)) and custom environment-map set. These assets and an environment map are retrieved to assemble the scene, providing both geometric fidelity and editability. Retrieval and filtering details in the Appendix.
3.3 PHYSICS- OPTIMIZATION
Directly placing 3D assets on 3DBBs estimated from image guidance poses significant challenges: (1) Occlusions yield incomplete point clouds, corrupting orientation, scale, and position and thus degrading 3D-BB accuracy. (2) Assel-image mismatch (shape/texture differences between retrieved assets and the guidance image) hampers precise pose estimation. To overcome these issues, Scenethe- sis introduces physics-aware optimization powered robust semantic feature matching and signed-distance fields (SDFs) (Zhang et al., 2024b; Edstedt et al., 2024; Chen et al., 2024a). This optimization process iteratively refines object poses to ensure pose alignment and physical plausibility.
3.3.1 POSE ALIGNMENT
To mitigate pose errors from occlusion, segmentation noise, and asset mismatch, semantics-awand correspondences from RoMa (Edstedt et al., 2024), which are robust to partial views. Let there be N objects in the rendered image I, where object o, has 5-DoF configuration (scale, upright rotation, translation) and its counterpart in the guidance image I is ö, We extract m high-confidence correspondences between the two:
dense,
p(x, v), p(z. N" RoMa(o, 6.),
(1)
where p(z, y) and p(z, y) are corresponding points on o, and ö, respectively. We then optimize 5-DoF parameters by minimizing weighted sum of 2D reprojection and 3D consistency over these correspondences, and backpropagate gradients to refine scale, translation, and upright rotation (see Figure 2). Full details appear in the Appendix.
3.3.2 PHYSICAL PLAUSIBILITY
Real-world 3D scenes obey physical laws, ensuring objects remain stable contact surfaces and collision-free However, pose aligmment with image guidance alone does guarantee phys- ical -bbjectttt may intersect, float, or sink due imperfect scene un- derstanding. See Figure 7 (b) as an ex- ample
Collstad objact Pueat
Child
Optinized
Vg
d(Pp)
14
Parent
Lepansiatiion
lxtability
Figure 3: Collision avoidance and stability maintenance.
Prior work often approximates objects with 3D bounding boxes (3DBBs) (Yang et al., 2024c:b). This coarse proxy ignores shape, making containmennt/unde relations impossible and distorting inter-object constraints-e.. c.g., items cannot be placed inside shelf compartments because 3DBBs falsely collide (see Figure 6). We instead operate on surface geometry: for each object sample points on the mesh and use them for precise collision tests and SDF- contact/support reasoning, enabling accurate placement (inside/undee//m))
The physics-aware optimization iteratively constructs a SDF-based physical structure, follow- ing the scene graph hierarchy: processing the anchor object first to establish stable foundation, followed by parent and child objects. The physics-awane optimization incorporates collision and stability constraints. Since retrieved 3D assets are upright, their rotation is constrained to azimuthal adjustments.
Formally, given a scene graph with N objects, each object has a 5-DoF configuration defined by scale upright rotation R. and translation T = (tz, tys t.). For computational efficiency, uniformly sample n points from its triangle surface mesh as its geometric representation and compute its centroid for collision avoidance.

---

Published as a conference paper at ICLR 2026
Table I: Quantitative evaluation on text-image alignment and spatial quality (1 higher better).
Spatial quality preference measures GPT-5 and human preference for Ours over baselines.
Text-Image Alignment
Spatial Quality Preference (GPT-5 Human Evaluation)
Method
CLIPT BLIPT VQAT
Object Diversityt
Layout Coherencef
Spatial Realism
Overall Performance
MIDI
50% / 55%
60% / 65%
75% / 82%
70% / 788%
Digital Cousins
55% / 55%
80% / 195%
80% / 94%
80% / 192%
DiffuScene
23.11
48.28
0.7832
75% / 80%
80% / 90%
90% / 76%
80% / 80%
SeeneTeller
25.27
51.99
0.7999
80% / 85%
80% / 771% %
85% / 80%
80% / 74%
Holodeck
28.32
46.25
0.6815
85% 80%
83% / 78%
81% 86%
85% 85%
LPy
23.01
46.35
0.8052
85% / 85%
80% / 71%%
90% / 82%
90% / 71%
LayOutVLM
24.57
41.96
0.6365
80% / 85%
80% / 90%
80% / 85%
85% / 85%
Desigg
28.19
44.76
0.7095
60% / 65%
70% / 74%
70% / 72%
65% / 77%
Ours
30.71
77.17
0.8269
Collision Constraints. We query the scene SDFs using object surface points to detect collision states and define position collision loss Lıramslation and scale collision loss L-sale- As shown in Figure 3, the deviation caused by collisions impacts translation T as:
Liranslatinn Evevv (T, [d,1, T
(2)
where [d,], u.) T u, |d.| computes a collision-free free position by adjusting the translation along direction u, with step size |d].. Here, d, is the negative SDF value at collided point v,, which belong to the points set with negative SDF V sampled uniformly from the surface. The direction is defined from the collision point toward the model's 's centroid, guiding objects away from collisions.
Collisions also affect object scale s due to opposing forces:
(Eviev- u.) - s)², if Neluster 1, 0,
Licale
otherwise.
where g(dd,), u,) = [l..l-|dd] ddil defines the target scale to reduce collision regions. Netuser denotes the ||ul||
number of distinct clusters formed without SDF sign flipping. As shown in Figure 3, two surface points and j with d, S 0 and dj S 0 belong different clusters, thus push the object to be smaller.
Stability Constraints. Objects are dragged by gravity and rest on their bottom contacting surface. We ensure stability by enforcing contact between object's 's bottom points and its parent surface, where their SDF values should be zero, as shown in Figure 3. The stability loss is defined as:
Lstability
Eveve (1 de)
where B are the sampled points on the bottom surface of the bounding box, and de are their corresponding SDF values. Further details on collision loss optimization are provided in the Appendix.
3.4 SPATIAL COHERENCE, JUDGMENT
After pose optimization, we apply a GPT-5-based judge to compare the generated 3D scene with the guidance image and planned objects, verifying inter-object relations. To assess this alignment, we design three metrics: (1) object category accuracy, comparing the generated scene with the image guidance; (2) object orientation alignment, measuring how well object orientations match the reference layout; (3) overall spatial coherence, capturing the holistic consistency the scene layout. Each metric is normalized between 0 (lowest) and I (highest). If any metric falls below predefined threshold, the scene judge triggers a re-planning step. Further details are provided in the Appendix.
4 EXPERIMENT
Implementation. We use GPT-5 (OpenAL, 2025) as the LLM and image generation in vision module. Following Holodeck (Yang et al., 2024c), we retrieve 3D models from high-quality Objaverse

---

Published as a conference paper at ICLR 2026
Joat twee wrille
moill wit aill plan bakaa Repken- rolls
delileeeee soop da acochall...
Loeplat puikat ted uo puttore a haddhart rigntted soll uotedd
TOOONN
and aestalcen cimo modkine
Revdiag cernect bookiaal Apent fee
let, the cutiee otr clar pordd plau woloop
(her casorshacti
rag. a bactoock...
DiffuSemee
SoeneTeller
LayouIP
Masign
Helodack
Sernethesis (Ours)
Figure 4: Qualitative results of Scenethesis compared with text-to-3D scene baselines.
children playroom
A modern car in the garage
A home gym room
Figure 5: Output Diversity. Given the same text prompt, Scenethesis can generate diverse scene with various objects and different layouts. More indoor / outdoor examples can be seen in Appendix.
subset (Deitke et al., 2023b). Other module details discussed in the above section. The physics- aware optimization is implemented in Py PyTorch (Paszke et al., 2019) and Py Torch3D (Ravi et al., 2020). Experiments are run on an A100 (40G) GPU, Further module details appear in the Appendix.
Baselines. We target interactive scene generation with per-object assets; thus monolithic single- geometry methods are out of scope. For text-to-3D, we evaluate open-source implementations of DiffuScene, LayoutGPT, IDesign, Scene Teller, layoutVLM, and Holodeck (Tang et al., 2024; Feng et al., 2024; Çelen al., 2024; Ocal et al., 2024; Yang et al., 2024c: Sun et al., 2025). For image- conditioned baselines, we compare to MIDI and Digital Cousins (Huang et al., 2025a; Dai et al., 2024), using the image guidance as the reference. For fairness, all text-to-3D baselines use the same GPT version. We restrict comparisons to methods with publicly available code/models: methods without released code are excluded.
Setup.
Scenethesis generates both indoor and outdoor scenes (Figure 1). As most baselines target indoor synthesis, we report comparisons on our indoor subset and present outdoor results separately.
We evaluate diversity and realism 34 prompts (22 indoor, 12 outdoor) spanning 8 primary and 16 secondary categories from DL3DV-10K (Ling et al., 2024). Indoor: Residential (living room, playroom, garage, warehouse), Shopping (bookstore, store), Tourism (museum, piano showroom),
7

---

Published as a conference paper at ICLR 2026
Sports (gym, billiards), Medical (ward), Education (laboratory). Outdoor: Nature/streeetsaapss (square, street, beach), Parks & recreation (playground).
Since DiffuScene, LayoutGPT. and SelneTeeller are designed/trained for residential indoor scenes, we restrict quantitative comparisons to that domain. To mitigate view bias, each scene is rendered from two standardized viewpoints (68 images total). For baselines that omit backgrounds (e.g., SceneTelller, we render Scenethesis without environment maps to match their setting.
4.1 METRICS
We evaluate four aspects of scene generated methods: controllability, layout realism, physical plausibility, and interactivity. Scores are computed per rendered view and averaged per scene.
Controllability. We measure prompt adherence for text-based baselines (DiffuScene, Teller Holodeck, LayourGPT, IDesign) using: (1) CLIP Score (Radford et al., 2021) cosine similarity between CLIP image and text embeddings. (2) BLIP Score (Li et al., 2023b) - image-text matching probability from BLIP-2's ITM head. (3) VQA Score (Lin et al., 2024b) accuracy of answers to prompt-derived questions from a frozen VQA model.
Layout Realism. We assess how well scenes reflect real-world layouts using a mix of GPT-5,
human-aligned evaluator in text-to-3D tasks (OpenAI, 2025; Lin et al., 2025b) and human judgments: (I) Object Diversity number of objects and categories in the scene. (2) Layout Coherence adherence of object positions and orientations to common sense. (3) Spatial Realism - presence of
diverse spatial relations (e.g., top of. inside, under). (4) Overall Performance alignment of object categories and styles with the scene type. Evaluation details and examples are in the Appendix.
Physical Plausibility. (1) Col-O average object collision rate, (2) Col-S average scene collision rate, (3) Inst-O average object instability rate, and (4) Inst-S average scene instability rate.
Collision is tested via mesh-mesh intersections, while instability follows Atlas3I) protocol (Chen et al., 2024b): average pose change after a short quasi-static physics rollout (Macklin, 2022).
Interactivity. To ensure objects accessible and manipulable in the scene based on their functional roles, we follow evaluation metrics from PhyScene (Yang et al., 2024b): (1) Reach - average object reachability rate, and (2) Walk ratio of the largest connected walkable area over all walkable regions.
4.2 QUANTITATIVE EVALUATION
Controllability. Table presents comprehensive evaluation of text-image alignment. Among all baselines, Scenethesis achieves the highest CLIP, BLIP, and VQA scores, confirming its effectiveness in adhering to the text description and the reliability of our agentic pipeline.
Layout Realism. Table also shows visual-quality preferences from humans and GPT-5 (OpenAI, 2025; Lin et al., 2025b). Although DiffuScene, MIDI, SceneTeller, and LayoutGPT are trained on 3D-FRONT-style indoor data, the training-fie ffree Scenethesis matches or exceeds their layout realism on residential scenes. Across broader indoor categories (e.g., shopping, tourism, sports), Table 1 shows that Scenethesis significantly outperforms Digital Cousins, Holodeck, and IDesign in visual quality and spatial realism, highlighting the benefit of vision-guided refinement.
Physical Plausibility and Interactivity. As shown in Table 2, Scenethesis markedly reduces colli- sions and instability on indoor scenes. Many baselines (DiffuScene, SceneTeller, LayouIGPT, MIDI) do not optimize mesh-level contact or stability during generation, leading to frequent interpenetrations. Although LayoutVLM tries to leverage on the layout common sense knowledge from VLM, the VLM layout guidance is noisy (Sun et al., 2025), still leading to unrealistic layout pattern similar to LLM-bbased method, e.g. the bookshelf or the chair orients towards the walls (see Appendix). Besides, LayoutVI.M relies on loU-based loss to avoid collision. This inaccurate geometry approximation does not model small object stability, leading to floating/unstable small objects standing on top of big furniture. Others (Digital Cousins, IDesign, Holodeck) operate at 3DBBs/collieer granularity- post-hoc cleanups such as wall snapping. 2 de- de-penetration, and z-y nudging- -hhch remain coarse and yield overlaps and brittle child-on-paree placements (small items furniture). By contrast, Scenethesis enforces SDF-basee contact/support constraints in-loop, coupled with semantic pose alignment, driving layouts toward collision-free and support-stable configurations. Beyond physical
8

---

Published as a conference paper at ICLR 2026
Table Physical-plausibiitty and interactivity results.
Physical Plausibility
Interactivity
Method
Col-Ol
Col-SL
Inst- Ol
Inst-S,
Reachf
Walk
MIDI
15.1%
50%
51.95%
90.90%
0.75
0.63
Digital Cousins
15.2%
60%
24.0%
88.33%
0.86
0.87
DiffuScene
19.5%
55%
20.75%
83.33%
0.74
0.83
Scene Teller
35.2%
75%
41.17%
78.57%
0.75
Holodeck
6.1%
21%
7.00%
31.58%
0.90
0,96
LayoutGPT
33.3%
73%
51.2%
81.14%
0.76
0.81
LLM
12.2%
57.1%
20.3%
71.4%
0.90
0.71
IDesign
6.51%
65%
8.3%
68.88%
0.88
0.80
Ours
0.8%
6%
3.20%
16.67 %
0.94
0.96
plausibility, Scenethesis also achieves higher reachability and walkability, producing more accessible, navigable scenes aligned with object functionality.
Outdoor scenes. We evaluate out-
Table 3: Outdoor Scene Qualitative Evaluation.
door generations on text-image align- ment and physical plausibility (Table 3).
Text-Image Alignment
Physical Plausibility
We omit preference scores (no baseline explicitly targets outdoor scenes) and Ours 29.87
CLIPT BLIP VQAt Col-Ol Col-S1 Inst-OL Inst-S
71.62
0.7592
0.06%
3.3%
0.12%
6.7%
Reach/ Walk metrics (no area restricted floor plan). Performance is comparable our indoor results (Table 2), indicating robustness. De- spite the added difficulty of non con-planar verrain-objectt must align to uneven surfaces for stabil- ity Scenethesis maintains low collisions and strong stability beyond indoor settings.
4.3 QUALITATIVE EVALUATION
Text-to-3D baselines. Figure 4 shows side-by-side comparisons; image-based baselines are in the Appendix. Methods trained on 3D-FRONT-style data (LayoutGPT, SceneTeller, DiffuScene) favor large furniture, often collide, and rarely follow prompts with supporting items or fine-grained relations (on/inside/sit). IDesign and Holodeck broaden categories and follow prompts but frequently misorient key objects (e.g., chairs facing walls, shelves blocking the windows) and exhibit inter-object collisions relying on 3DBBs approximation or predefined supports (see additional examples in Appendix).
Scenethesis. In contrast, Scenethe- sis is able to place small objects on inside the supports and maintains collision-free, stable layouts. It also enables fine-grained positioning across support levels (c.g., shelves, carts, as illustrated in Figure 6), which is crucial for embodied-Al scenarios that require meaningful object manipulation (Yang et al., 2024b; Nasiriany et al., 2024). top (Holodeck).
Scenethesis
Holodeck
Figure 6: Scenethesis precisely places small objects (e.g., bag, wine bottle, shoes, vase) within shelf compartments rather than only on
LLM-only planners lack this degree of visual grounding and typically fail on such spatial realism. Figure 5 further highlights diverse outputs from the same prompt (both asset selection and layout).
To assess downstream utility, demonstrate that Scenethesis can be used as a data engine for virtual content creation by using Scenethesis to improve a feed-forward interactive scene generator in the Appendix.
4.4 ABLATION STUDY
The physics-awe aware optimization has three components: pose alignment, collision constraint, and stability constraint. We perform ablation studies to assess their effectiveness.

---

Published as a conference paper at ICLR 2026
Table 4: Ablation study on the effectiveness of adding each component in spatial and physical constrains.
Component
Pose Align. t
Collision
Instability
Raw layout
0.536
22.7%
87.3%
+Pose Alignment
0.732
10.6%
74.2%
+Collision
0.755
3.6%
69.8%
+Stability
0.836
0.8%
3.2%
(4). Image Guidance
(b). Layout
(c). Pose Alignment
(d). Collision
(e). Staility system)
(b) Raw layout: places 3D models estimated 3DBBs. (c) Pose alignment: adjusts 5DoF poses to align the pose. (d) + Collision: adding collision constraint, (e) + Stability: adding stability constraint.
Figure 7: Effects of different constraints. (a) image guidance from text input.
Metric. For each generated scene, we render the output and assess pose alignment to the image guidance with GPT-5 based on: (1) per-object orientation//saaleposiiii
and (2) global spatial coherence. The judge returns similarity score in [0,1] (higher is better). We also measure mesh
collisions and support instability using the protocol Section 4.2
Baselines, Raw Layout: Objects are placed on the locations of 3DBB Pose Aligument: Aligns object pose with image guidance via correspondence matching. Collision Constraint: Optimizes location size avoid collisions. Stubility Constraint: Optimizes location to remain stable.
Results. As shown in Table 4, pose alignment significantly improves spatial consistency, while collision and stability constraints enhance physical plausibility, making scenes simulation-ready. Figure shows qualitative visualization.
CONCLUSION AND LIMITATION
introduce Scenethesis, training-free agentic framework for generating high-fidelity interactive 3D scenes by leveraging LLM-based coarse planning, vision- guided layout refinement, and physics-aware optimization for object pose adjustment. A scene judge module ensures spatial coherence. Experimental results demonstrate that it significantly outperforms SOTA baselines in layout coherence, spatial realism, and plausibility. Our approach is limited by retrieval databases since generative 3D methods cannot yet handle articulation. Future advances in generative 3D could overcome this constraint by enabling articulated object synthesis, enhancing scene diversity.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
We provide detailed pseudo-codes in the Appendix. In addition, our code, configuration files, and asset lists will be released. Data/Assets. We use curated Objaverse subset and HDR environment maps; the exact lists and licenses will be provided with the release. Prefrained Models. Grounded SAM Ren et al. (2024) and DepthPro Yang et al. (2024a) are used off-the-shelf (versions/commits will be documented in the repo). LLM/VLM. GPT-5 is used for coarse planning, scene layout judging, and quality evaluation. All used prompts are listed in APP. B.2. Implementation. The physics-aware optimization is implemented in Py PyTorch Paszke et al. (2019) and Py PyToch311 Ravi et al. (2020). Compute. All experiments ran on single NVIDIA A100 (40G) GPU.
ETHICS STATEMENT
Generated scenes are for research/creative use and are not safety-certffied. We respect dataset licenses (Objaverse, HDRIs) and use no personal data. GPT-5 is used under human supervision (details in the Appendix).
ACKNOWLEDGEMENTS
We thanks Xuan Li for his insightful and constructive feedback method discussion.
REFERENCES
Rio Agguin-Kaagg Maxim Gumin, Do Heon Han, Stewart Morris, Seung Jean Yoo, Aditya Ganeshan, Kenny Jones, Qiuhong Anna Wei, Kailiang Fu, and Daniel Ritchie. Open-universe indoor scene generation using IIm program synthesis and uncurated object databases. arXiv preprint arXiv:2403.09675 2024. 3
Aleksei Bochkovskii, Amaël Delaunoy, Hugo Germain, Marcel Santos, Yichao Zhou, Stephan Richter, and Vladlen Koltun. Depth pro: Sharp monocular metric depth in less than second. arXiv preprint XrXiv:2410.02073, 2024.
Ata Çelen, Guo Han, Konrad Schindler, Lue Van Gool, Iro Armeni, Anton Obukhov, and Xi Wang. I-design: Personalized IIm interior designer. In European Conference on Computer Vision, pp. 217-234. Springer, 2024. 3, 7,5
Angel Chang, Manolis Savva, and Christopher D Manning. Learning spatial knowledge for text to 3d scene generation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp. 2028-2038, 2014.
Yamei Chen, Yan Di, Guangyao Zhai, Fabian Manhardt, Chenyangguang Zhang, Ruida Zhang, Federico Tombari, Nassir Navab, and Benjamin Busam. Secondpose: Se (3)-consistent dual-stream feature fusion for category-level pose estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 996999969, 2024a. 5
Yunuo Chen, Tianyi Xie, Zeshun Zong, Xuan Li, Feng Gao, Yin Yang, Ying Nian Wu, and Chenfanfu Jiang. Atlas3d: Physically constrained self-supporting text-to-3d for simulation and fabrication. arXiv preprint 2rXiv:2405.18515 2024b. 8
Tianyuan Dai, Josiah Wong, Yunfan Jiang, Chen Wang, Cem Gokmen, Ruohan Zhang, Jiajun Wu, and Li Fei-Fei. Automated creation of digital cousins for robust policy learning. arXiv preprint arXiv:2410.07408,, 2024. 3, 5,
Matt Deitke, Eli VanderBilt, Alvaro Herrasti, Luca Weihs, Kiana Ehsani, Jordi Salvador, Winson Han, Eric Kolve, Aniruddha Kembhavi, and Roozbeh Mottaghi. Procthor: Large-scale embodied ai using procedural generation. Advances Neural Information Processing Systems, 35:5982-5994
2022. 1
11

---

Published as a conference paper at ICLR 2026
Matt Deitke, Rose Hendrix, Ali Farhadi, Kiana Ehsani, and Aniruddha Kembhavi. Phone2proe: Bringing robust robots into our chaotic world. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 9665-9675, 2023a.
Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of anno- tated 3d objects. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 15142-13153 2023b. 5, 7, 6, 14
Johan Edstedt, Qiyu Sun, Georg Bökman, Marten Wadenbäck, and Michael Felsberg. Roma: Robust dense feature matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19990-1980, 2024. 5,3
Weixi Feng, Wanrong Zhu, Tsu-jui Fu, Varun Jampani, Arjun Akula, Xuehai He, Sugato Basu, Xin Eric Wang, and William Yang Wang. Layoulgpt: Compositional visual planning and generation with large language models. Advances in Neural Information Processing Systems, 36, 2024. 1, 7,5
Rafail Fridman, Amit Abecasis, Yoni Kasten, and Tali Dekel. Scenescape: Text-driven consistent scene generation. Advances in Neural Information Processing Systems, 36, 2024. 3
Huan Fu, Rongfei Jia, Lin Gao, Mingming Gong, Binqiang Zhao, Steve Maybank, and Dacheng
Tao. 3d-future: 3d furniture shape with texture. International Journal of Computer Vision, 129:
3313-3337,202 3,
Chuang Gan, Jeremy Schwartz, Seth Alter, Damian Mrowca, Martin Schrimpf, James Traer, Julian De Freitas, Jonas Kubilius, Abhishek Bhandwaldar, Nick Haber, et al. Threedworid: A platform for interactive multi-modal physical simulation. arXiv preprint 1xXiv:2007.04954 2020. 1
Lukas Höllein, Ang Cao, Andrew Owens, Justin Johnson, and Matthias Nießner. Text2room: Extracting textured 3d meshes from 2d text-to-image models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7909-7920, 2023. 3
Ziniu Hu, Ahmet Iscen, Aashi Jain, Thomas Kipf, Yisong Yue, David A Ross, Cordelia Schmid, and Alireza Fathi. Scenecraft: An Ilm agent for synthesizing 3d scenes blender code. In Forty-first International Conference on Machine Learning, 2024.
Zehuan Huang, Yuan-Chen Guo, Xingqiao An, Yunhan Yang. Yangguang Li, Zi-Xin Zou, Ding Liang, Xihui Liu, Yan-Pei Cao, and Lu Sheng. Midi: Multi-instance diffusion for single image to 3d scene generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 23646-23657, 2025a. 1,3, 7,
Zhening Huang, Xiaoyang Wu, Fangcheng Zhong, Hengshuang Zhao, Matthias Nießner, and Joan Lasenby. Litereality: Graphics-ready 3d scene reconstruction from rgb-d scans, 2025b. URL
attps://arxxt org/ abtg//bbs//e 22bb/25077 02861.8 3
Mukul Khanna, Yongsen Mao, Hanxiao Jiang, Sanjay Haresh, Brennan Shacklett, Dhruv Batra, Alexander Clegg, Eric Undersander, Angel X Chang, and Manolis Savva. Habitat synthetic scenes dataset (hssd-200): An analysis of 3d scene scale and realism tradeoffs for objectgoal navigation. In Proceedings of the IEEÉ/CVF Conference on Computer Vision and Pattern Recognition, pp.
16384-16393, 2024. 3
Eric Kolve, Roozbch Mottaghi, Winson Han, Eli VanderBilt. Luca Weihs, Alvaro Herrasti, Matt Deitke, Kiana Ehsani, Daniel Gordon, Yuke Zhu, et al. Ai2-thor: An interactive 3d environment for visual ai. arXiv preprint arXiv: 2017.
Jacob Krantz, Erik Wijmans, Arjun Majumdar, Dhruv Batra, and Stefan Lee. Beyond the nav-graph: Vision-and-led and-language navigation continuous environments. In Computer Vision ECCC 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXVIII 16, pp. 104-120. Springer, 2020. 1
Vikram Kumaran, Jonathan Rowe, Bradford Mott, and James Lester. Scenecraft: Automating interactive narrative scene generation in digital games with large language models. In Proceedings of the AAA/ Conference on Artificial Intelligence and Interactive Digital Entertainment, volume 19, pp. 86-96, 96, 2023. 1,3
12

---

Published as a conference paper at ICLR 2026
Chengshu Li, Ruohan Zhang, Josiah Wong, Cem Gokmen, Sanjana Srivastava, Roberto Martín- Martín, Chen Wang, Gabrael Levine, Michael Lingelbach, Jiankai Sun, et al. Behavior- lk: A benchmark for embodied ai with 1,000 everyday activities and realistic simulation. In Conference on Robot Learning, pp. 80-93. PMLR, 2023a.
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference machine learning, pp. 19730-19742. PMLR, 2023b. 8
Manyi Li, Akshay Gadi Patil, Kai Xu. Siddhartha Chaudhuri, Owais Khan, Ariel Shamir, Changhe
Tu, Baoquan Chen, Daniel Cohen-Or, and Hao Zhang. Grains: Generative recursive autoencoders for indoor scenes. ACM Transactions on Graphics (TOG), 38(2):1-16, 2019.
Yixun Liang, Xin Yang, Jiantao Lin, Haodong Li, Xiaogang Xu, and Yingcong Chen. Luciddreamer: Towards high-fidelity text-to-3d generation via interval score matching. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 6517-6526, 2024. 4
Chenguo Lin and Yadong Mu. Instructscene: Instruction-drrive 3d indoor scene synthesis with semantic graph prior. arXiv preprint arXiv:2402.04717, 2024.
Tsung- Yi Lin, Chen-Hsuann Lin, Yin Cui, Yunhao Ge, Seungjun Nah, Arun Mallya, Zekun Hao, Yifan Ding, Hanzi Mao, Zhaoshuo Li, et al. Genusd: 3d scene generation made easy. In ACM
SIGGRAPH 2024 Real-Time Live!, pp. 1 1-2. 2024a. 3
Yuchen Lin, Chenguo Lin, Panwang Pan, Honglei Yan, Yiqiang Feng, Yadong Mu, and Katerina Fragkiadaki. Partcrafter: Structured 3d mesh generation via compositional latent diffusion trans- formers. arXiv preprint urXiv:2506.0557: 2025a.
Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan. Evaluating text-to-visual generation with image-to-text generation. In European Conference on Computer Vision, pp. 366-384, Springer, 2024b.
Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan. Evaluating text-to-visual generation with image-to- generation. In European Conference Computer Vision, pp. 1666-88. Springer, 2025b.
Lu Ling, Yichen Sheng, Zhi Tu, Wentian Zhao, Cheng Xin, Kun Wan, Lantao Yu, Qianyu Guo, Zixun Yu, Yawen Lu, et al. D13dv-10k: A large-scale scene dataset for deep learning- based 3d vision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22160-22169, 2024. 7
basedd
Lu Ling, Yunhao Ge, Yichen Sheng, and Aniket Bera. I-scene: 3d instance models are implicit generalizable spatial learners. arXiv preprint arXiv:2512.1366 2025.
Andrew Luo, Zhoutong Zhang, Jiajun Wu, and Joshua B Tenenbaum. End-to-end optimization of scene layout. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 3754-3763, 2020. 3
Miles Macklin. Warp: A high performafee python framework for gpu simulation and graphics. https: /ttp///ttbb////////////////tbhtt wtw.tho.ooooooo...coooooooo....t com/ http://iidaaaaaaaeaa March 2022. NVIDIA GPU Technology Confer- ence (GTC). 8
Yanxu Meng. Haoning Wu, Ya Zhang, and Weidi Xie. Scenegen: Single-image 3d scene generation in one feedforward pass. arXiv preprint arXiv:2508. 15769, 2025. 3
Soroush Nasiriany, Abhiram Maddukuri. Lance Zhang, Adeet Parikh, Aaron Lo, Abhishek Joshi, Ajay Mandlekar, and Yuke Zhu. Robocasa: Large-scale simulation of everyday tasks for generalist robots. arXiv preprint ar arXiv:2406.025223 2024. 1,9
Başak Melis Öcal, Maxim Tatarchenko, Sezer Karaoğlu, and Theo Gevers. Sceneteller: Language-to- 3d scene generation. In European Conference on Computer Vision, pp. 362-378. Springer, 2024.
7.
13

---

Published as a conference paper at ICLR 2026
OpenAI. Gpt-5 system card. Technical report, OpenAI, August 2025. URL https: sttps://cdn. openai wtw/ 5- S-ssstem-mcarde pdf. Version Aug 13, 2025. 4, 6, 8,
Despoina Paschalidou, Amlan Kar, Maria Shugrina, Karsten Kreis, Andreas Geiger, and Sanja Fidler. Atiss: Autoregressive transformers for indoor scene synthesis. Advances in Neural Information Processing Systems, 34:12013-12026, 2021. 3,
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32, 2019. 7, 11
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. PmLR, 2021. 8
Ohad Rahamim, Hilit Segev, Idan Achituve, Yuval Atzmon, Yoni Kasten, and Gal Chechik. Lay-a-scene: Personalized 3d object arrangement using text-to-image priors. arXiv preprint arXiv:2406.00687. 2024. 3
Nikhila Ravi, Jeremy Reizenstein, David Novotny, Taylor Gordon, Wan- Yen Lo, Justin Johnson, and Georgia Gkioxari. Accelerating 3d deep learning with pytorch3d. arXiv preprint arXiv:2007.08500 2020. 7, 11,4
Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, al. Grounded sam: Assembling open-world models for diverse visual tasks. arXiv preprint 2rXiv:2401.14/11 2024. 4, 11,
Fan-Yun Yun Sun, Weiyu Liu, Siyi Gu, Dylan Lim, Goutam Bhat, Federico Tombari, Manling Li, Nick Haber, and Jiajun Wu. Layoutvlm: Differentiable optimization of 3d layout via vision- languae models. arXiv preprint arXiv:2412.02/99 2024.
Fan-Yun Yun Sun, Weiyu I. Liu, Siyi Gu, Dylan Lim, Goutam Bhat, Federico Tombari, Manling Li, Nick Haber, and Jiajun Wu. Layoutvlm: Differentiable optimization of 3d layout via vision-language models. In Proceedings of the Computer Vision and Pattrn Recognition Conference, pp. 29469- 29478, 2025. 5,5
Jiapeng Tang. Yinyu Nie, Lev Markhasin, Angela Dai, Justus Thies, and Matthias Nießner. Diffuscene: Denoising diffusion models for generative indoor scene synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 20507-20518. 2024. 3,7,
Kai Wang, Yu-An Lin, Ben Weissmann, Manolis Savva, Angel X Chang, and Daniel Ritchie. Planit: Planning and instantiating indoor scenes with relation graph and spatial prior networks. ACM Transactions on Graphics (TOG), 38(4):1-15, 2019. 2019,
Xinpeng Wang, Chandan Yeshwanth, and Matthias Nießner. Sceneformer: Indoor scene generation with transformers. In 2021 International Conference on 3D Vision (3DV), pp. 106-115. IEEE, 2021.
Yian Wang, Xiaowen Qiu, Jiageng Liu, Zhehuan Chen, Jiting Cai, Yufei Wang, Haun- Johnson Wang, Zhou Xian, and Chuang Gan. Architect: Generating vivid and interactive 3d scenes with hierarchical 2d inpainting. Advances in Neural Information Processing Systems, 37:67575-67603, 2024. 3
Yufei Wang, Zhou Xian, Feng Chen, Huun-Hsuan Wang, Yian Wang, Katerina Fragkiadaki, Zackory Erickson, David Held, and Chuang Gan. Robogen: Towards unleashing infinite data for automated robot learning via generative simulation. arXiv preprint arXiv:231 1.0/455, 2023.
Rundi Wu, Ben Mildenhall, Philipp Henzler, Keunhong Park, Ruiqi Gao, Daniel Watson, Pratul Srinivasan, Dor Verbin, Jonathan T Barron, Ben Poole, al. Reconfusion: 3d reconstruction with diffusion priors. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 21551-21561, 2024. 4
14

---

Published as a conference paper at ICLR 2026
Jianfeng Xiang, Zelong Lv, Sicheng Xu, Yu Deng, Ruicheng Wang, Bowen Zhang, Dong Chen, Xin Tong, and Jiaolong Yang. Structured 3d latents for scalable and versatile 3d generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 21 21469-221488 2025. 10
Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10371-10381, 2024a. 11
Yandan Yang, Baoxiong Jia, Peiyuan Zhi, and Siyuan Huang. Physcene: Physically interactable 3d scene synthesis for embodied ai. In Proceedings of the IEEF/CVE Conference on Computer Vision and Pattern Recognition, pp. 16262-1622 16272, 2024b. 5,
Yue Yang. Fan-Yun Sun, Luca Weihs, Eli VanderBilt, Alvaro Herrasti, Winson Han, Jiajun Wu, Nick Haber, Ranjay Krishna, Lingje Liu, et al. Holodeck: Language guided generation of 3d embodied ai environments. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patlern Recognition, pp. 16227-16237, 2024c. 1, 4, 5, 6, 7, 12
Kaixin Yao, Longwen Zhang, Xinhao Yan, Yan Zeng, Qixuan Zhang, Lan Xu, Wei Yang, Jiayuan Gu, and Jingyi Yu. Cast: Component-aligned scene reconstruction from an rgb image. ACM Transactions Graphics (TOG), 44(4):1-19) 2025. 3
Hlong-Xing Yu, Haoyi Duan, Charles Herrmann, William T Freeman, and Jiajun Wu. Wonderworld: Interactive 3d scene generation from a single image. arXiv preprint arXiv:2406.09399 2024a. 3
Hong-Xing Yu, Haoyi Duan, Junhwa Hur, Kyle Sargent, Michael Rubinstein, William T Freeman, Forrester Cole, Deqing Sun, Noah Snavely, Jiajun Wu, al. Wonderjourney: Going from anywher to everywhere. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6658 6667, 2024b.
Jingbo Zhang. Xiaoyu Li, Ziyu Wan, Can Wang, and Jing Liao. Text2nerf: Text-driven 3d scene generation with neural radiance fields. IEEE Transactions Visualization and Computer Graphics, 2024a. 3
Junyi Zhang, Charles Herrmann, Junhwa Hur, Eric Chen, Varun Jampani, Deqing Sun, and Ming- Hsuan Yang. Telling left from right: Identifying geometry-awan semantic correspondence. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3076-3085, 2024b.
Junsheng Zhou, Yu-Shen Liu, and Zhizhong Han. Zero-shot scene reconstruction from single images with deep prior assembly. arXiv preprint arXiv:2410.15971, 2024a. 3
Menggi Zhou, Jun Hou, Chuanchen Luo, Yuxi Wang, Zhaoxiang Zhang, and Junran Peng. Scenex: Procedural controllable large-scale scene generation via large-language models. arXiv preprint arXiv:2403. 15698, 2024b.
Xiaoyu Zhou, Xingjian Ran, Yajiao Xiong, Jinlin He, Zhiwei Lin, Yongtao Wang, Deqing Sun, and Ming-Hsuan Yang. Gala3d: Towards text-to-3d complex scene generation via layout-guided generative gaussian splatting, arXiv preprint 202iv:2402.07207, 2024c.
Yang Zhou, Zachary While, and Evangelos Kalogerakis. Scenegraphnet: Neural message passing for 3d indoor scene augmentation. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 7384-7392, 2019. 3
15

---

Published as a conference paper at ICLR 2026
A LLM USAGE
We used LLM (OpenAI, 2025) in two ways: (i) manuscript assistance for grammar check only-no no technical claims, equations, figures, code, results authored by the model; and (ii) in-method roles as coarse layout planner. All outputs were reviewed by the authors; humans made final decisions and accept full responsibility.
Model and settings. Unless otherwise stated, we used the gpt-5 API. Prompts are provided in Appendix B.2. No personal sensitive data were sent to the API.
Scope limitations. mental results/code. Its layout planning is described in Sec. and APP. B.2.1.
The LLM did not select baselines, choose hyperparameters, modify experi-
B IMPLEMENTATION DETAILS OF Scenethesis
B. ALGORITHM OVERVIEW
In this section, we provide a high-level algorithmic overview of Scenethesis, with detailed steps outlined in Algorithm 1.
B.2
МЕTHON DETAILS
B.2.1 COARSE SCENE PLANNING
Using the user's scene prompt input, the LLM (powered by GPT-5 (OpenAI, 2025)) follows six-step process:
1. Interpreting the user's scene prompt.
2. Reviewing the object categories available in the provided asset database.
3. Selecting relevant objects from the asset list.
4. Cross-checking the availability of the selected objects.
5. Planning the scene using the selected objects.
6. Generating output files according to the specified standards.
The final coarse scene planning output consists of two components: list of selected object categories commonly found the scene (defining anchor object and other common objects) and an upsampled prompt that outlines the scene's 's spatial hierarchy. The designed prompt presents in Coarse Scene Planning Instruction Prompts Section D.I and the output example is in Coarse Scene Planning Output Example Section D.2.
B.2.2 LAYOUT VISUAL REFINEMENT
Based the upsampled prompt. GPT-5 generates image to serve fine- fine-grained layout guidance. Several post-procee post-processing steps are applied to this image guidance:
graph, defining the ground as the root object, along with parent objects and their corresponding child objects. Additionally, Grounded-SAM (Ren et al., 2024) segments each object in the image to obtain masks and cropped images. These are then projected into 3D space using Depth Pro (Bochkovskii et al., 2024), allowing for the initial positioning of objects within spatial relationship graph.
Scene Graph Construction: GPT-5 (OpenAI, 2025) is used generate
Asset Retrieval. CLIP (ViT-L/14 trianed on LAION-2B) image and semantic features are employed to retrieve 3D assets that align with the image guidance. GPT-5 (OpenAI, 2025) is further utilized to select the most relevant environment map based on the upsampled prompt. It is important to note that Scenethesis focuses layout planning for objects on the ground, while background elements, such as wall decorations. lighting, or outdoor settings (e.g.. sunshine the sea), are visually determined by the environment map.

---

Published as a conference paper at ICLR 2026
Algorithm Text to 3D Interactive Scene Generation
Input: User text
2: Output: 3D interactive scene layout
3:
Stage 1: Coarse Scene Planning:
object list, upsampled prompt <- LIM(user text)
obtain the object list an upsampled prompt
5:
6:
Stage 2: Layout Visual Refinement:
img-gun guidaneee - VLM eeeeampled_promppe
generate the guidance image the reference
8:
cmapped_images + Grounded_SAM (img-guidee guidance, object_list) images
b identify each object and crop the
9:
depth map Depth.Pro (img guidance)
D generate depth map generate initial 5DoF poses generate scene graph I retrieve 3D assets
10:
SDoF poses
(cspped images, mapth scene graph + VIM (img guidance, object list, 5DoF poses)
12:
3Dasssets CLIP (cropped_image) object.list)
13:
emvironment.map <- VIM (upsampled promptt
D retrieve environment maps
14:
15:
Stage 3: Physics-aware Optimization:
16:
scene SDF <- Init Init_Scene Scene LocenessDFaaaeeeeeeeeeeeeee
SDF (anchor
D. compute SDF for each object >> iterate over all objects variables be optimized a obtain parent object's SDF
17:
for node in scene grwph.bfs.traveee......
do
18:
8, R. T node.pose
parent. SDF + mnodeparrett.]] parent.SDF
20:
for iteration = to max _tteratioss
21:
mesh + Gct Gct_ fet (eshhoodee
22:
mesh Apply Traplyy_rransfoommmmee
R, T)
coordinate alignment
23:
img rendered, depth rendered +- Render camera)
render RGB and depth
24:
correspondence < RoMa((gg_uuidane, img-rnndeeed
correspondence matching
25:
mesh_points Get Poi.toointto.. Cet..ooort...ooosddeeh.........e correspondence, camera) guided_points oeedd correspondence, camera)
26:
27:
Lpose.20 + Dist 2isst_2D(coressooddeee)
b loss computation
28:
Lpose.3D) Dist_3D(mesh.pouiii points, guided points)
29:
Leoltision Celsssiinmmssh" Scene_SDi
30:
Letability <- Stabi Stability(hotonn honeeltttyhoootoooonnsseeeee points(mesh), parent.SDD)
31:
loss + ACpose + Neattiatoo + Astabiiity Catability loss. (as.Ba))wwrrd)
32:
33:
optimizer.Stepp ()
pose optimization
34:
optimizer..err_Grrae
35:
end for
36:
scene SDF +- Update Scene SDF(scene SDE, node)
37:
end for
38:
39:
Stage 4: Scene Spatial Coherent Judgment:
40:
Multi-view images + Render Coptimized_3D.sseee Qualified <- VLM (Multi-view images) if qualified then
41:
42:
43:
goto Stage
b re-generate if current optimization fails.
44:
end if
45:
Return: Optimized 3D interactive scene
2

---

Published as a conference paper at ICLR 2026
V8
Child
Ph (Pb))
8
Va
Parent
(a). Lraaslation
(b). Lecale
(c). Catability
Figure 8: Illustration of collision avoidance and stability maintenance. The solid-line circle indicates the 3D object's s current position, while the dotted-line circle marks its anticipated position. The black dot represents the centroid of the target object, the purple dots indicate surface nodes with negative SDF values, and the red point Uy is the bottom node the object. (a). The collision pushes the circle object out of the rectangle along the direction from the sampled point to the circle's center by step [d]. (b). The collision indicates the object is too large and negative signed distance fields (SDF) points (i.e. point v. and point vb) are detected from distinct classes of the object during optimization. The collision loss shrinks the object size such that there are no different clusters of negative SDF points on the object surface can be detected. (c). The stability maintenance keeps the the child and the parent to be as close as possible with no collision.
The output of the fine-grained layout planning includes the generated image as guidance, graph with the initial poses of the objects, the retrieved assets, and the retrieved environment map. The visual details are presented in the web. Note: the algorithm only generates LLM-planned objects on the scene graph instead of generating each object in image guidance- since the image generator may hallucinate extras (e.g. tiny ones in the coffee-table at Figure 18). If an LLM-planned category is missed, the self-check agent triggers second round of planning until it is detectable. This keeps optimization tractable while guaranteeing planned objects appear in the final scene.
the 'A living room with reading materials'
B.2.3 PHYSICS-AWARY OPTIMIZATION DETAILS
The chysics-aware optimization is an iterative optimization process that consists of two key com- ponents: pose alignment optimization and physical plausibility optimization. pose aligmment optimization focuses on aligning the position, size, and orientation of 3D models with their counter- parts the image guidance ensure visual coherence for spatial relationships. Physical plausibility optimization ensures that the 3D models in the scene are free from collisions and maintain stability, contributing to a realistic and physically consistent layout.
Pose Alignment. To align the object's 's position, size, and orientation with its counterpart in image guidance, Scenethesis applies the dense semantic correspondence matching from RoMa (Edstedt et al., 2024). Unavoidable discrepancies in texture and shape between image guidance and retrieved assets are mitigated by focusing on high-level semantics over low-level details. That is, minimizing the distance between correspondence points the rendered image and the guided image î. Suppose there are N objects in the rendered image 7, each represented by and defined by 5-DoF configura- tion, which includes scale 8, upright rotation R, and translation = (ta. ty: t2). The counterpart of each object in the generated image ī is denoted as ō. The objective of ensuring visual coherence is minimize the distance between corresponding points by optimizing the 5-DoF parameters. This ensures that the spatial positions, size, and orientations of the 3D models are closely aligned with their counterparts in the guided image. The matching process is formalized as:
(p(x, v) p(x, y) = RoMa(o, O),
(5)
where p(z, y). p(x,)) y) are correspondent pair in object and ö. We select pair points in each optimization iteration with confident score higher than т. The higher confidence score indicates a higher probability of matching. We minimize the 2D pixel distance and 3D projected point clouds
3

---

Published as a conference paper at ICLR 2026
distance between the matched pair denoted as follows:
Lpsse X2aL2d + Aad
(6)
where X2d and Xad are coefficients of 2D pixel loss and 3D point cloud loss denoted Laa and Lzd-
Physical Plausibility. Physical plausibility ensure generated 3D scenes adhering to fundamental physical principles. Instead of using 3D bounding box (3DBB) as object approximation, Scenethesis accurately detects collision state from the surface points of the 3D models using signed distance field (SDF). The collision avoidance and stability maintenance as illustrated in Figure 8.
The collision avoidance affects the translation by: Liramslation [T [d| u) T 2
where f(T, d, u) T u. |d] computes collision- free position T by adjusting T along direction u with step size d. Here, d is the negative SDF value at a collided point o; such that d(u;) I 0 and [d] m0 )d(:)) is the negative SDF value d after being processed through ReLU function, meaning only collided points contribute to this collision term. The direction u is defined from the collision point toward the model's centroid C, guiding objects away from collisions.
The collision avoidance affects the scaling by detecting that object collides from at least two different directions:
Lucale Eviev- (allds). - )))
2
if Neluster
otherwise
||,.l defines the target scale to reduce collision regions. Neluser denotes the number of distinct clusters formed without SDF sign flipping. As shown in Figure 3, two surface points i and j with d; S 0 and dj < 0 belong to different clusters, and thus push the object to be smaller.
where g(dd)), u;) = ||
The stability maintenance affects the translation T by:
Sstability = -E (1-ex(((do))
(9)
v.EVB
where F the sampled points on the bottom surface of bounding box, and d, are their correspond- ing SDF values.
Method Overview Building the physics- aware optimization described above, integrate pose spatial constraints and physical constraints into the text-to-3D optimization framework. The following function defines the joint optimization of object position, orientation, and scale:
XpPpose Cirnstation Ac.shaaale > stabiility
(10)
B.3 EXPERIMENT DETAILS
Parameters. For pose alignment, we select the m 100 correspondence points with matching confidence Z 0.6 in each optimization iteration. Additionally, we uniformly select = 400 samples from the surface of 3D model accurately detect the collision and stability states in each optimization iteration.
We explored Adam and SGD as the optimizer during the optimization process. Though Adam has been widely applied for training deep neural networks, the adaptive momentum makes the optimiza- tion unstable, leading to sub-optimal. So we use SGD in our implementation. The optimization implementation is based on pytorch3D (Ravi et al., 2020) and the visualization is rendered using Blender.
We examine Scenethesis's 's runtime for scene generation using RTX-4090. The stages include planning (20s), layout optimization (90s per-object), and self-checking (4s), totaling 13 min. For reference, Digital Cousins (another compositional method) takes 18 min for the same scene.
4

---

Published as a conference paper at ICLR 2026
User specified prompt case
User specified short prompt case
Figure 9: Short prompt: a living room with reading materials; detailed long prompt: A living room that provide a neutral and cozy space with minimalist design. At the center of the scene, light beige sofa is positioned against a textured stone wall the background. wooden coffee table sits the floor, with white coffee cup placed on top. Two blue armehairs symmetrically arranged on either side of the coffee table, facing inward toward the sofa. Behind each armchair, tall white floor lamp stands, providing ambient lighting. Next the lamps, green potted plants are placed near the wall, adding natural decorative touch.
front of the sofa, a round
Prompts. Scenethesis supports both short and detailed user-specified prompts. A short prompt provides a user-friendly and flexible approach to 3D scene generation, where the LLM interprets the input, revisits the available 3D models in database, selects the common objects and anchor objects, and generates an upsampled text prompt for coarse layout planning. In contrast, a long prompt, which includes user-defined objects and inter-object relationships, enables greater user control over 3D scene generation. In this case, the LLM directly reasons over the detailed prompt, revisits available 3D models in the database, and defines the anchor object, skipping the upsampling stage. We illustrate examples of short and long prompts defining living room in Figure 9.
Baseline Comparison. Our baselines includes: the most relevant text-to-3D scene synthesis meth- ods such as Diffuscene (Tang et al., 2024), SceneTeller (Ócal et al., 2024), LayoutGPT (Feng et al., 2024), IDesign (çelen al., 2024), Layout (Sun et al., 2025), and Holodeck (Yang al., 2024c); Figure 10 demonstrates the qualitative example of 'a children playroom' for Layout Scenethexis. The results show Scenethesis achieves much better layout arrangement, physical plausibility and interactivity comparing with LayoutVLM, We also provide quantitative compar- isons with image-conditioned scene methods: Digital Cousins (Dai al., 2024), an image-based compositional approach that retrieves per-object assets and arranges them based on the image refer- ence, and MIDI (Huang et al., 2025a), an end-to-end diffusion-trann traessoomtrannommrrrr that jointly reconstructs object meshes and poses from single image (with instance masks) without retrieval or post-hoc optimization. The quantitative result is presented the Table and 2 of the main paper.
To ensure fair comparison with in-context learning based text-to-3D baselines (LayouGGPT, SceneTeller), we adopt closed-set setting: the retrieval pool is restricted 3D-FRONTT/DDFTTURR and prompts are limited to living-roon and bedroom scenes, matching the basclines' domain coverage. We follow the same evaluation protocol. Comprehensive text-image alignment and visual-coherence results reported in Table 5, and physical plausibilitynnteeaaciivy results are in Table 6.
Note, the spatial-quality preference Table columns report preference for ours over each baseline and dash in the row indicates "not applicable." For example, for 'IDesign' the object diversity metric is 60% / 65%. It indicates in 'object diversity' preference, GPT voted 60% for Scenethesis generated scenes (and 40% for IDesign generated scenes) and human voted 65% % for Scenethesis
5

---

Published as a conference paper at ICLR 2026
Table 5: Quantitative evaluation on text-image alignment and spatial quality (1 higher is better).
Spatial quality preference measures GPT-5 and human preference for Ours over baselines.
Text-Image Alignment
Spatial Quality Preference (GPT-5 Human Evaluation)
Method
CLIPT
BLIPT VQAT
Object Diversity?
Layout Coherence?
Spatial Realism
Overall
Performance
Teller
26.11
54.74
0.7801
70% / 72%
80% 75%
82% 75% 90% / 85%
80% 77%% 90% 75%
LayoutGPT
23.01
52.15
0.7982
70% / 65%
82% / 70%
Ours
32.48
82.96
0.8342
/-
---
Table 6: Plyssicl--lausiiiltty and interactivity results.
Physical Plausibility
Interactivity Reacht
Method
Col-Ol
Col-S. Inst-O. Inst-S.
Walk
SceleTeeller
33.7%
70.45%
43.7%
72.7%
0.72
0.84
LayoutGPT
34.2%
75.0%
51.2%
79.6%
0.74
0.80
Ours
0.45% %
2.27 %
1.12%
9.09% %
0.95
0.96
generated scenes (and 35% for IDssign generated scenes). That means both human and GPT prefer Scenethesis generation results over IDesign's generation results.
For visual quality assessment, we use both user study and GPT-5 as evaluation tools.
We outline the GPT-5 prompt assessment for both baseline evaluation and ablation evaluation as follows:
Comparison with baselines by GPT-5: GPT-5 is employed to evaluate the generated scenes for four metries: object diversity, layout coherence, spatial realism and complexity, and overall performance. The evaluation prompts are detailed in the Instruction Prompts for Evaluating Generated Scene Section D.3. Additionally, a comparison example of generated scenes is provided in Figure 12 with their evaluation results generated by GPT-5 detailed in Evaluation Example of Generated Scenes Section D.4.
Comparison with baselines by human preference: We applied a user study to study human preference of baseline method and our method. See Figure 11 an example. There are 69 users took our survey.
Evaluation in ablation studies: GPT-5 is also utilized assess the pose alignment metric during the ablation studies of Scenethesis's ' physics- aaaase optimization. This evaluation measures the similarity of object position, size, and orientation with their counterparts in the image guidance, as well as the overall spatial coherence of the layout. The instruc- tions for assessing pose alignment are provided the Instruction Prompts for Ablation Study Section D.5.
We present additional qualitative results.
Qualitative Results of Scenethesis's Scene: the qualitative results of Scenethesis's s scenes, as shown in Figure 15. Figure 18 presents the generated scenes by Scenethesis with their image guidance. Note that Scenethesis focuses on layout planning for ground objects. The absence of certain unique assets in Objaverse (Deitke et al., 2023b) may cause discrepancies between the generated scene and the image guidance. Future work could address this by incorporating more diverse assets.
We present different camera views to showcase
Qualitative Results of Scenethesis with image-based 3D generation /reconstruction approach. The quantitative results of image-based methods are presented in the Table 1 and 2 in 4. We provide qualitative comparison results between Scenethesis and image-based 3D scene methods in Figure 17. Digital Cousins is image-based compositional scene synthesis method by object retrieval. In our test results, it often lacks spatial coherence and physical plausibility (c.g., floating items), whereas Scenethesis produces collision-free supported placements. MIDI a learning bearning-based based approach that jointly reconstructs object meshes and poses from single image(with instance masks). Compared with Scenethesis,

---

Published as a conference paper at ICLR 2026
M10
LayoutVLM
Scenethesis
Figure 10: Qualitative example of children playroom for LayoutVI.M vs. Scenethesis
we observe: (i) hallucinated geometry under partial views, reducing per-asset fidelity (e.g.. artifacts in table, cart, and shelf Figure 17); (ii) errors long-tail relations- small objects on/inside/under larger supports, leading to heavily collision, floating (e.g., the flower vase failing sit the shelf Figure 17); and (iii) no explicit physical constraints, leading to floating and penetrations (c.g., shelf, chair, and toys intersect and float Figure 17). These behaviors are consistent with its training prior: layouts learned largely from DD-FNNNT//D- FUTURE, which emphasize large furniture and underrepresent small-object arrangements. Consequently, MIDI is less simulation-readd By contrast, Scenethesis 's retrieval-baseed pipeline provides edit-ready, high-quality meshes and, through SDF contact/support con- straints, produces diverse, fine-grained relations with physical plausibility. This supports downstream interaction and also serves as a data engine to strengthen future learning-based methods.
Qualitative and quantitative Results of Physical plausibility Comparison: The physical Plausibility quantitative comparison presented in Table Among all text-to-3D methods, LayouGGPT, Scene and Diffuscene only focus on visual quality ignoring physical plausibility. Therefore, we observe high collision rate and instability rate of these methods in the Table 1 and Figure 4 4. While Holodeck and IDesign apply physical constrains via 3DBB approximation. In particular, Holodeck applics both soft and hard constrains based on the Depth- Septh-First-See First-Search Solver and small objects placed predefined locations. These small objects may collide with each other due to the shape and size variations as shown in Figure 13.
Visual Comparison with Holodeck: addition to the quantitative comparison presented in Table of the 4, we provide visual comparison between Scenethesis and Holodeck, ofate-of-the-an LLMM-based 3D interactive scene generation method, in Figure 16. Based the four evaluation metrics detailed in 4, scenes generated by Scenethesis demonstrate greater diversity in object categories, quantities, and sizes. More importantly, Scenethesis's scenes have broader range of spatial relationships, such "on top of". "inside". and

---

Published as a conference paper at ICLR 2026
Here two views of bedroom. Rate which is better.
Please ignore aspects in environment background style such as lighting, shadow, color and decorations In the wall etc.
View 1:
View
A
B
A better
B is better
Layout coherence
O
Spatial realism
Overall consistency
Figure 11: User study example.
"under", compared to those generated by Holodeck (Yang al., 2024c), which supports only "on top of" spatial relation. Furthermore, Scenethesis' 's scenes align more faithfully with the intended scene type. i.e. when given the description "a peaceful beach during sunser". Scenethesis produces an outdoor scene with appropriate beach elements, while Holodeck incorporates beach-related objects but generates environment resembling indoor setting.
Success rate. In our quantitative evaluation, 72% of scenes could pass the judge after the first optimization round; the self-check lifts the success rate to 91% after the first round repair and 2-3 repairs for the rest scenes, confirming both the pipeline's 's robustness and the value of the self-check agent.
We observe three main failure modes: (1) LLM-planned objects that are very small in the generated image are missed by the detector; (2) Heavy occlusion degrades pose alignment. Our pose alignment optimizes scale, translation, and rotation of category-matchee mesh using
Limitations.
8

---

Published as a conference paper at ICLR 2026
Scenethesis
Holodeck
Figure 12: An example comparison of generated scenes given user prompt: "a warchouse".. Note that Scenethesis's 's scenes rendered without environment map ensure fair comparison with Holodeck's s scenes.
Holodeck
IDesign
Figure 13: An example of objects collision from Holodeck and IDesign.
semantic correspondences computed only on an object's s visible pixels. The loss in Eq. 9, Appendix. back-propagated pose until matched 3D points align with the visible region while ensuring physically plausible. This category-level matching tolerates asset discrepancy and works reliably with 2 70%. For example, partially occluded lamp the living room with reading materials' scene in Figure 18). When visibility drops further, correspondences become sparse/noisy and the optimizer can shrink misplace objects (e.g., the shelf in "A living room with reading materials' and the corner sofa in "a cute piano room" ", Figure 18). Improving correspondence robustness for tiny and heavily occluded objects an important direction for future work; (3) Retrieval database. Our approach is limited by retrieval databases since generative 3D methods cannot yet handle articulation and high-quality assets during occlusion (as see in MIDI's example in Figure 17). Future advances in generative 3D could overcome this constraint by enabling articulated object synthesis, enhancing scene diversity.
C DOWNSTREAM APPLICATION
Scenethesis as a data engine. To assess downstream utility, we demonstrate that Scenethesis can be used data engine for virtual content creation by using Scenethesis to improve feed- forward interactive scene generator (MIDI-3D). Existing feed-forward interactive scene generation method (c.g.. MIDI-3D (Huang et al., 2025a)) trained 3D-FRONT, whose supervision is dominated by large furniture and underrepresents common functional relations (e.g., small-on-large support, containment/inidee, under). Since Scenethesis produces scenes with diverse layouts and rich

---

Published as a conference paper at ICLR 2026
Method (training data)
CD-S
F-score-S
CD-O 4
F-score-O f
loU-S f
MIDI-33 (3D-FRONT)
0.0416
0.6186
0.0455
0.7466
0.5644
MIDI-3D (3D-FRONT+5K)
0.0252
0.7590
0.0316
0.7922
0.6935
Table 7: Quantitative results on BlendSwap. CD-S and CD-O refer to scene-level CD and object-level CD respectively: F-score-S and F-score-O refer to scene-level F-score and object-level F-score respectively. Lower CD is better; higher F-score/loU is better. 5K refers to 5K Scenethesis scenes.
Inpul images
(3D-FON))
MIDI (3D-FRONT 5k Scenethesis's seenes)
Figure 14: Qualitative comparison of MIDI generated scenes by training on 3D-FRONT vs, training on 3D-FRONT with additional 5K Scenethesis scenes.
spatial relations, we hypothesize that augmenting training with Scenethesis scenes will improve the generalization of feed-forward generators.
Data. We generate 5k scenes from Scenethesis with diverse functional relations (on/under//nside).
Model. MIDI-3D is trained on 3D-FRONT. As MIDI-3D does not release their training code, we re-implement MIDI by using Trellis (Xiang al., 2025) as its backbone model.
Protocol. We augment the MIDI-3D original training set (3D-FRONT) with 5k Scenethesis scenes and using the same data preparation protocol and training hyperparameters.
Evaluation. On MIDI-3D's out-of-distribution testing dataset BlendSwap. we report scene- and instance-level Chamfer Distance (CDI) and F-score (1, 7=0.1), and layout accuracy via volumetric IoU (f) between predicted and ground-truth 3D boxes.
Results. As summarized in Table 7 and Figure 14, Scenethesis augmentation yields consistent gains in geometric quality (lower CD, higher F-score) and layout accuracy (higher volumetric loU), demonstrating that physically plausible, relation-rich layouts produced by Scenethesis provide effective supervision that improves generalization of feed-forward scene generators.
10

---

Published as a conference paper at ICLR 2026
reading corner
A warehouse space
A private billiard room for relax
AREHON
[
beautiful surf-fishing beach
A relax street with mailbox
A children playground
Figure 15: Qualitative results of generated indoor and outdoor scenes by Scenethesis at different camera viewpoints
11

---

Published as a conference paper at ICLR 2026
A peacefiul beach during sunset
A cute piano rom
A living room with reading materials
warm hospital ward
children playroom
exhibition
spacious warehouse
museum exhibition
Scenethesis w.o'w. environment map
Holodeck
Scenethesis w.okww environment map
Holodeck
Figure 16: Visualization comparison of generated scenes between Scenethesis and Holodeck. The first column of images shows scenes generated by Scenethesis without an environment map, the second column displays scenes generated by Scenethesis with an environment map, and the third column presents scenes generated by Holodeck. The evaluation metrics, including object diversity, layout coherence, spatial realism, and overall performance, are detailed in the Experiment section. Scenethesis's 's scenes have wider variety of spatial relationships, such as "on top of "inside" and "under", compared to those generated by Holodeck (Yang et al., 2024c), which supports only "on top of spatial relation. addition, Holodeck lacks visual perception and usually generates misoriented objects, c.g. shelves occlude the window in children playroom and warchouse, chair orients towards the window the hospital case, hindering their functionalities.
12

---

Published as a conference paper at ICLR 2026
Image guidance mask
MIDI
Digital Cousins
Scenethesis (Ours)
Figure 17: Qualitative comparison between Scenethesis and single image based 3D scene generation (Digital cousins) /reconstruction methods (MIDI). Digital Cousins an image-based compositional scene synthesis method by object retrieval. often lacks spatial coherence and physical plausibility (e.g., floating items), whereas Scenethesis produces collision- follision-free, supported placements. MIDI is learning-bal based approach that jointly reconstructs object meshes and poses from a single image instance masks). Compared with Scenethesis, we observe: (i) hallucinated geometry under partial views, reducing per-asset fidelity (e.g., artifacts in table, cart, and shelf Figure 17); (ii) errors on long- tail relations- small objects onfinside/under larger supports, leading to heavily collision, floating (c.g.. the flower vase failing sit on the shelf Figure 17); and (iii) explicit physical constraints, leading to floating and penetrations (e.g., shelf, chair, and toys intersect and float Figure 17). These behaviors consistent with its training prior: layouts learned largely from ED-FRONT/SD-FUTUUS emphasize large furniture and underrepresent small-object arrangements. Consequently, MIDI is less simulation-ready. By contrast, Scenethesis 's retrieval-based pipeline provides edit-ready, high-quality meshes and, through SDF contact/support constraints, produces diverse, fine-grained relations with physical plausibility. This supports downstream interaction and also serves as data engine to strengthen future learning-bass based methods.
which
13

---

Published as a conference paper at ICLR 2026
A cute piano room
A warm bospital ward
living room with reading materials
laboratory scene
A penceful beach during sunset
A relax beach scene
Image guidance
Scenethesis scene w.o./w: environment map
Image guidance
Scenethesis soene f.o.fw. eavironment map
Figure 18: We provide visual illustration of the generated scenes and their corresponding image guidance. The first column displays the image guidance, while the second and third columns show the generated scenes without and with the environment map, respectively. Note that Scenethesis focuses on layout planning for floor-supported objects. Scenethesis instantiates only the categories specified by the LLM-planmn scene graph, rather than every object hinted by the image guidance-sing the image generator may hallucinate extras (e.g., tiny props on a coffee table for 'a living room with reading materials') )). If any planned category is missing from the visual refinement outputs, the self-check agent triggers another round of planning until the object becomes detectable. This design keeps the optimization tractable while ensuring that all planned objects appear in the final scene. Additionally, certain unique assets, such as beach mat, are unavailable in Objaverse (Deitke et al., 2023b), which may result in the generated scene differing from the image guidance. Future work could enhance the system by incorporating a wider range of assets.
14

---

Published as a conference paper at ICLR 2026
D PROMPTS EXAMPLES
D.1 COARSE SCENE PLANNING INSTRUCTION PROMPTS
Coarse Scene Planning Instruction Prompts
Task Deseription:
You are responsible for generating set of common objects and planning scene hased on these common objects. You will be given list that includes all available object categories and text prompt describe scene. This a hard lask, please think deeply write down your analysis following steps:
Step 1: Review All Categories
a. Begin by thoroughly reviewing the categories in the provided list. Identify potential groups or clusters objects within this that are commonly found in similar environments (e.g., furniture, electronics, household items, etc.).
Step 2: Interpret Input Prompt
a. Carefully read the input prompt. Understand the theme, primary activitics, or the setting it describes, as these will guide your object selection. i.e. if the prompt gives: children playing room, then you may think of objects like tent, toy, bear, ball, chair, etc.
Step 3: Object Selection
Based the description, select least 15 object categories from the list that match the scene.
Determine the anchor object:
Identify the anchor object among the selected objects. Consider the following factors:
A large object directly on the ground (i.e. floor, table, or shelf).
2. An object that influences where other objects placed (i.e. table in dining room, and there are cups fruits on table).
3. The object should logically anchor the seene and often defines the scene's layout orientation. i.e. the sofa in front-facing view in the scene.
Step 4: Object Cross-check
will give you $100 tips if you cross-check whether objects in the its relevant categories. i.e., if there a bookshelf in your planned scene. the bookshelf should also be found in the given list, or bookcase can be found in the list if bookshelf is not covered by the category. Otherwise, re-plan the scene. Step 5: Plan Scene with Selected Objects
be found
in the given eategory
Based on the description and selected objects, plan the scene, keeping these aspects in mind:
Choose objects that are contextually relevant to the scene (e.g.. selecting table, chair, flower vase, and utensils for dining room), but do generate any wall décor objects.
Functionality:
i. Spatial Hicrarchy:
Please have depth effect in the layout, For depth effect, scene should have some objects placed on the ground as the background, central, and in the front, resulting in depth layout, i.e. the sofa and bookshelf are the background of the table and chair set in the living room.
2. Please have a supportive item in the layout. i.e. the shoes, bag, and hat are in the display shelf in a clothes store, where the display shelf is supportive item. Balance: Ensure mix of large and small objects to avoid overcrowding or under- populating the scene. i.e. taking the table as the center, there are flower vases, fruits, and cups on the table, and chairs are on the sides.
Step Output Format:
Save the selected objects as json file follow the output format: Anchor object:
Other common objects:
b. Save scene planning txt file.
15

---

Published as a conference paper at ICLR 2026
D.2 COARSE SCENE PLANNING OUTPUT EXAMPLE
Coarse Scene Planning Output Example
User input: a gym room
Step 1:
Reviewing Object Categories for Gym-Related Items
From the provided list, here the relevant categories typically found in gym setting:
Exercise equipment: dumbhell, treadmill, weight, barbell
Furniture or storage items: bench, shelf, cabinet
Accessories smaller items: towel, water bottle, yoga mat, seale, bag
Step 2:
Seene Interpretation
The scene is gym room, the primary objects would focus equipment for strength and cardio exercises, along with supportive items like shelves organization and accessories like towels and water bottles for hydration.
Step 3: Scene Layout and Object Selection
Anchor object: Treadmill (as the main focus of cardio equipment)
Other common objects:
Dumbbell, Barbell, Shelf, Water bottle, Towel, Weight, Bag, Mat, Scale, Cabinet, Exercise ball, Fan, Mirror
Step 4: Object cross-check
Treadmill Available as "treadmill"
Dumbbell / Available as "dumbbell"
Barbell / Available as "barbell"
Bench Available "bench"
Shelf / Available "shelf"
Water bottle / Available "water bottle"
Towel - / Available "towel rack"
Weight - Available as "weight"
Bag / Available as "bag"
Mat Not explicitly listed, but a close alternative like "place mat"
Scale / Available as "scale (measuring instrument)"
Cabinet / Available "cabinet"
Exercise ball - Not explicitly listed, but close alternative like by "ball"
Fan / Available as "fan"
Mirro Available as "mirror"
Step Plan Scene with Selected Objects
Background: Shelf and cabinet are placed to the side or back corner of the room for organization, holding smaller items. Mirror mounted the back wall for checking excreise form. They serves supportive cquipments...It enhances depth and makes the room feel more spacious...
Middle Ground: Treadmill the scene. Other objects such as bench and barbell are placed around trcadmill, allowing access from multiple directions for various exercises. They provide visual focused equipment...
primary cardio equipment and anchor object. anchoring
weight to the scene and represent
Foreground: Dumbbell, water bottle and other objects set closer to the viewer, easily accessible for hand-held exercises. It introduces smaller,
detailed object in the front. They placed nearby, adding another layer of excreise options for core stability...
This hierarchy ensures balanced layout, where larger and bulkier items form the background... The middle ground includes the treadmill and training equipment, while smaller and accessible items like dumbbells occupy the foreground... This setup provides clear layout for practical use and visual depth in the gym scene...
Step 6: Output Format A json file that include the available objects and a txt file that describe the scene planning.
16

---

Published as a conference paper at ICLR 2026
D.3 INSTRUCTION FOR EVALUATING GENERATED SCENES
GPT-5 Evaluation Prompts Generated Scenes
Evaluate Generated Scenes for Scenethesis and Holodeck
This hard problem. You are supposed compare the alignment of pair of images with given text prompt that deseribe the seene. Images contain generated scenes by two different methods. Please evaluate them in the following five aspects:
Counting number of object and object types in the scene. The higher number, the hetter object diversity. i.e. there are three shelves and a box the scene. Then the ohject type is 2 and the number of counting is 3+1 =4
Object diversity:
2. Layout coherence: whether the objects position and orientation in the scene are realistic and adhere to common sense.
An ideal layout would be dependents the scene type. i.e. the objects in the garage will forms bit chaotic organization and it would be less reasonable if all objects standing against the wall in very clean order.
Objects should be placed reasonably. i.c. shelves fly in the air or hange on the wall.
3. Spatial realism complexity: whether scene contains diverse hierarchy. The measure of spatial complexity is by review scenes and carefully evaluate objects relations. The higher diverse relation indicate hetter spatial complexity. The spatial hierarchy refers the following aspects:
relations such on the top of, in side of. under the, etc.
objects in various sizes. i.c. the scene should contain large objects such as shelf and small objects such apple.
objects that are arranged in different area and form a depth effects. i.e. objects are in the background, in the front, and in the center of the space.
4. Overall performance:
assets' categorizes and styles are faithful to the scene type. ie. fruits are less likely in the warchouse, while carts, boxes are more likely in warchouse scene. Thus, it would be bad performance if find fruits in the warchouse and good performance if carts/shelf/boxes presents in the warehouse. Please carcfully evaluate the object's 's category and scene type. To see whether reasonable enough based on your common sense.
5. Please ignore aspects in environment background style such as lighting, shadow, color, and decorations in the wall This is challenging task, please think aloud and make your decisions by:
analyzing each criteria one by one for each image.
report your evaluation results based on the analysis.
Your output format should be: () better than (} in the criteria f).
17

---

Published as a conference paper at ICLR 2026
D.4 EVALUATION EXAMPLE OF GENERATED SCENES
GPT-5 Evaluation Results
Scenethesis's 's Scene
1. Object diversity:
Objects visible: Ladder, barrel, hoxes, extinguisher, table, tire, crate.
Object types: types. Total count: 13
Higher diversity due to various objects.
2. Layout coherence:
Objects dispersed naturally, suggesting activity.
Different sizes and less structured arrangement suggest more realistic warchouse scene.
3. Spatial realism complexity:
Objects have various spatial relations; ladder stands, boxes on ground, tables center...
Good diversity with mix of background and central objects.
4. Overall performance:
Varied objects like ladders, barrels, crates, and extinguishers typical in warehouse.
Realistic depiction of warchouse setting.
Holodeck's Scene
1. Object diversity:
Objects visible: Shelves, cabinets, safety vests, and workhenches.
Object types: Shelves, cabinets, vests, workbenches. Total count: 7+2-9 (shelves are considered per structure, as are others).
Diversity is somewhat low as there are few types.
2. Layout coherence:
Organization is neat; workbenches and cabinets are along walls.
Reasonable layout for tidy scene, lacks variety sizes.
3. Spatial realism & complexity:
Shelving and cabinets against walls create vertical space; workbenches on the ground.
Lacks diverse spatial relations.
4. Overall performance: Objects like shelves and workbenches fit warehouse setting, but lacks variety typical in larger-scale warehousing.
Evaluation
Object diversity: Scenethesis is better than Holodeck: Layout coherent: Scenethesis is better than
Holodeck; Spatial realism & complexity: Scenethesis is better than Holodeck; Overall performance: Scenethesis better than Holodeck.
18

---

Published as a conference paper at ICLR 2026
D.5 INSTRUCTION PROMPTS FOR ABLATION STUDY
Pose Alignment Evaluation Instruction Prompt
This task involves evaluating the pose alignment between two images in a pair. One image serves as the image guidance (GT), while the other is generated image. Your objective is to measure the pose alignment of the generated image relative to the GT image. Follow these steps for evaluation:
Review Objects the GT Image: Examine all objects in the GT image, focusing on their locations, sizes, and orientations. Understand the spatial relationships among objects, such as lop of, inside, under, etc.
Evaluate pose alignment: Assess similarity between generated image and the image based on the following three aspects:
Location and Size Similarity: Compare the location and size of objects in the generated image with those in the GT image. Assign a similarity score between indicates the highest similarity. For example:
where
an apple image it is placed on the left side the table, the similarity might be moderate (e.g.. 0.5).
GT image is placed the center table, and generated
the apple is misplaced (c.g., on the ground or missing entirely), similarity would be very low (e.g., 0.1).
Orientation Similarity: Examine the orientation of each object in generated image compared to the GT image. Pay close attention to details, noting any deviations such slight tilts (e.g., right/left, up/down) rotations that create different perspectives. Assign score from 0 1, where indicates perfect alignment and 0 indicates significant mismatch (e.g.. opposite orientation).
Overall Layout Similarity: Assess the overall visual coherence of the generated image compared to the image, including spatial relationships and hierarchical structure. Assign a similarity score between and 1, where represents a perfect match. For instance:
A perfect match occurs when the generated image maintains the same spatial relationships, relative locations, sizes, and orientations as the GT image (e.g., an apple placed the center of table in both images).
Small deviations in placement or orientation are acceptable but should result in lower score.
Exclusions:
Do not consider style, appearance, object shape, or texture in your evaluation. Focus solely on pose alignment.
4. Output Format: Clearly document your similarity scores for each aspect (Location and Size Similarity, Orientation Similarity, location and size similarity score is (). orientation similarity score is (). and overall layout similarity score is {. Please save the evaluated scores as json file.
Overall Layout Similarity) following the format:
19


Reviews:
[{"review_id": "CcLxwHqwh4", "reviewer": "Reviewer_zmuD", "summary": "This paper presents Scenethesis, a training-free, agentic framework for text-to-3D scene generation that combines an LLM-based planner with vision-based layout refinement and a physics-aware optimization loop. The system decomposes the task into four modules: (1) coarse scene planning using an LLM, (2) layout visual refinement using visual foundation models (e.g., Grounded-SAM, DepthPro), (3) physics-aware optimization enforcing SDF-based collision and stability constraints, and (4) a GPT-5-based scene judge for coherence verification and targeted repair. Scenethesis outperforms several text-to-3D baselines such as DiffuScene, LayoutGPT, Holodeck, and SceneTeller on metrics of layout realism, physical plausibility, and spatial coherence, across both indoor and outdoor prompts.", "questions": "you might want to cite \"Lay-A-Scene\" who also made a collision free and many aspects as your paper\n@article{rahamim2024lay,\n  title={Lay-a-scene: Personalized 3d object arrangement using text-to-image priors},\n  author={Rahamim, Ohad and Segev, Hilit and Achituve, Idan and Atzmon, Yuval and Kasten, Yoni and Chechik, Gal},\n  journal={arXiv preprint arXiv:2406.00687},\n  year={2024}\n}\n\n1. How many judge repair iterations are typically performed, and is convergence guaranteed?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. physics-aware optimization, the use of SDF constraints for collision avoidance and stability is more principled than bounding-box–based heuristics in prior work (e.g., LayoutGPT, Holodeck).\n2. generality, unlike most prior works limited to indoor scenes, Scenethesis generalizes to outdoor prompts and long-tail object relations (on-top-of, inside, behind).\n3. despite not requiring new model training, Scenethesis achieves or exceeds the quality of trained baselines, highlighting the power of coordination across existing large models.", "weaknesses": "1. most image generation has problems with creating several objects, but i cant find how did you challenge it?\n2. many qualitative examples lack strict spatial or functional order. For instance, in “A living room with many reading materials”, books are scattered arbitrarily on the floor — any random placement would satisfy the prompt. In another case, the bookshelf is positioned behind the sofa, making it unreachable and violating functional affordance. This suggests that while Scenethesis achieves physical plausibility, its understanding of semantic utility and human-centered spatial reasoning remains limited.\n3. several components (LLM-based coarse planning, VLM grounding, Collision avoidance) closely follow prior work like Holodeck, Lay-A-Scene and LayoutGPT. The main new element, in-loop SDF-based optimization, while effective, is an incremental rather than novel.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ifkW3y7Oxe", "reviewer": "Reviewer_qQ1g", "summary": "This paper proposes Scenethesis, an agentic framework for generating physically consistent, interactive 3D scenes from textual prompts.\n\nThe authors design the following procedures to achieve a physically realistic 3D scene:\n- Coarse scene planning: using an LLM that first generates a rough layout plan based on the text prompt, objects, and spatial relationships.\n- Layout visual refinement: using Image-guidance, Sceen-Graph generation and asset retrieval for to handle the uncommon co-occurrence and physical inconsistencies\n- Optimization module: First align the pose and then refine(optimize) the pose with self-defined physical constrraints.\n- Judgment with VLMs, and then decide whether to replan or not.\n\nIn summary, the framework is training-free, using different existing and pretrained models for a physically plausible 3D scene generation.", "questions": "- I am wondering why the visualization results are much better than previous works. In this work, there are certain illumination conditions during rendering, while in previous works, they are weak. Though the layouts of this work is better, are there any other rendering setings that make the visualization better?\n- How much time does it take to synthesize one 3D scene?\n- What is the success rate of this work? Since the integration of different modules may result in an accumulative error. Some of the modules are not that stable, e.g., Image generation and Grounded SAM.\n- When the LLM plans the layout, does it explicitly model hierarchical or relational structures (e.g., “a cup on a table next to a book”)? How does it ensure that such relations are spatially consistent?\n- In my understanding, this work relies on the generated image, can this work support some methods to extrapolate the range of the scene?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper pipeline is intuitive, reasonable, and easy to understand. The whole paper is well-written in presenting the methodology.\n- The visualization results are realistic and visually plausible\n- The quantitative results seem to be much better", "weaknesses": "- Some failure cases can better show the limitations of this work,\n- This work seems to be relying heavily on the existing modules and database. The contribution may be limited. For example, both [1], [2], and [3] mentioned similar physical constraints. Prior works also used LLMs as planners and judges.\n\n[1]: Cast: Component-aligned 3d scene reconstruction from an rgb image\n[2]: WorldCraft: Photo-realistic 3D world creation and customization via LLM agents\n[3]: Layout-your-3d: Controllable and precise 3d generation with 2d blueprint", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "omcRvYakme", "reviewer": "Reviewer_v9QQ", "summary": "This paper proposes Scenethesis, an agentic framework that can generate 3D interactive scenes from language inputs. Scenethesis is a multi-stage framework that first leverages language models to plan the overall design and potential objects, which will be used for generating a 2D image as guidance. Based on the image guidance, the vision module will generate the scene graph and retrieve 3D assets.  Then, the optimization module will adjust the placement of objects to ensure pose alignment and physical plausibility. Finally, the judge module uses GPT-5 to evaluate the quality of the generated scene. The experimental results demonstrate Scenethesis outperforms previous methods both qualitatively and quantitatively.", "questions": "1. How often does the Judge Module get triggered, and how much performance gain can it bring? How long will the re-planning step take, and will it significantly affect efficiency?\n2. Have you quantified the benefits of the environment map? This may affect the CLIP/BLIP scores a lot. It is better to do some ablations to disentangle the contributions of layout, assets, and map.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The proposed approach is technically sound, and the qualitative examples clearly show that Scenethesis can generate more realistic scenes than previous methods, especially for outdoor scenes and small objects placement.\n2. The ablation studies show that each component in spatial and physical constrain can improve the performance.", "weaknesses": "1. My major concern is the lack of evidence on downstream applications. Although this work demonstrates better realism and diversity in generating scenes, it is unclear whether such improvement can transfer to downstream tasks like embodied AI, robot navigation/manipulation, or enhance the spatial understanding of vision-language models, etc. Including such downstream experiments will enhance this paper's contributions and validate the effectiveness of the generated scenes.\n2. The rendering qualities vary a lot across baselines and the proposed methods, which may affect human judgments. Controlling this variance can help us understand whether the improvement comes from better layout or just higher fidelity.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "q0kL7JlyMI", "reviewer": "Reviewer_NE9e", "summary": "The paper introduces Scenethesis, a framework that couples LLM-based scene planning with vision-guided spatial refinement and physics-aware optimization for the task of 3D scene generation. Its physics-aware optimization for mesh-level contact and stability improves physical plausibility.\n\nScenethesis first drafts a coarse layout via an LLM; a vision module then produces a guidance image, constructs a scene graph with 3D bounding boxes, and retrieves assets. A physics-aware optimization module refines object poses using semantic correspondences and SDF-based contact/support constraints, and a final scene judge verifies spatial consistency.", "questions": "All of my questions are listed in the weaknesses section, and I may adjust the rating if they are well addressed.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1) The pipeline couples LLM-based scene planning with vision-guided spatial refinement to capture real-world spatial complexity.\n2) The paper introduces a physics-based optimization that further adjusts asset placement via semantic feature matching and SDF-constraints, which is novel and completes the full scene-generation cycle.\n3) The quantitative and qualitative results are strong.\n4) The paper is written in an easy-to-follow manner.", "weaknesses": "1) Missing results: Table 1 lacks the spatial quality preference results for Scenethesis.\n2) Comparison with baselines: \n\n     a. L395-397: SceneTeller and LayoutGPT are training-free in-context learning methods.\n\n     b. Comparing against these methods on 3D-FRONT, which covers primarily bedrooms and living rooms with corresponding assets, is not fair, since many prompt objects are absent. In these baselines, retrieval is typically constrained by predicted category and then nearest dimensions (3DBBs). It is unclear how missing objects in 3D-FRONT are handled in Table 1’s qualitative examples (e.g., retrieving a “double bed” in place of a “piano,” or what substitutes for a “wheelchair”).\n\n     c. To fairly compare with training-free in-context learning methods, the baselines should also be evaluated using Objaverse. As not having the right assets could significantly affect metrics like CLIP score. A small set of sample layouts can be curated using Objaverse and provided to the baselines as in-context samples during evaluation.\n\n3) Lack of comparison with recent baselines (e.g., LayoutVLM [1])\n\n[1] Sun, F. Y., Liu, W., Gu, S., Lim, D., Bhat, G., Tombari, F., ... & Wu, J. (2025). Layoutvlm: Differentiable optimization of 3d layout via vision-language models. In Proceedings of the Computer Vision and Pattern Recognition Conference (pp. 29469-29478).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)