DeBaRA: Denoising-Based 3D Room Arrangement Generation
Authors: Léopold Maillard
Nicolas Sereyjol-Garros
Tom Durand
Maks Ovsjanikov
Venue: NeurIPS 2024 poster (2024)
Keywords: Indoor 3D Scene Synthesis
Layout Generation
Score-based Generative Models
Diffusion Models
Conditional Generation
Primary area: machine_vision
TL;DR: We propose DeBaRA, a conditional score-based generative model that performs state-of-the art arrangement generation in bounded indoor scenes and several downstream applications by solely learning 3D object spatial features.

Abstract: Generating realistic and diverse layouts of furnished indoor 3D scenes unlocks multiple interactive applications impacting a wide range of industries. The inherent complexity of object interactions, the limited amount of available data and the requirement to fulfill spatial constraints all make generative modeling for 3D scene synthesis and arrangement challenging. Current methods address these challenges autoregressively or by using off-the-shelf diffusion objectives by simultaneously predicting all attributes without 3D reasoning considerations. In this paper, we introduce DeBaRA, a score-based model specifically tailored for precise, controllable and flexible arrangement generation in a bounded environment. We argue that the most critical component of a scene synthesis system is to accurately establish the size and position of various objects within a restricted area. Based on this insight, we propose a lightweight conditional score-based model designed with 3D spatial awareness at its core. We demonstrate that by focusing on spatial attributes of objects, a single trained DeBaRA model can be leveraged at test time to perform several downstream applications such as scene synthesis, completion and re-arrangement. Further, we introduce a novel Self Score Evaluation procedure so it can be optimally employed alongside external LLM models. We evaluate our approach through extensive experiments and demonstrate significant improvement upon state-of-the-art approaches in a range of scenarios.

DeBaRA: Denoising-Based 3D Room Arrangement Generation
Léopold Maillard 122 Nicolas Sereyjol-Garros"
Tom Durand?
Maks Ovsjanikov
LIX, École Polytechnique, IP Paris
Dassault Systèmes
(aaillard,maks))01 www.:l/www..llttttnnii....eeee
(firsname. 1 lastnname)
Abstract
Generating realistic and diverse layouts of furnished indoor 3D scenes unlocks multiple interactive applications impacting wide range of industries. The inherent complexity of object interactions, the limited amount of available data and the requirement fulfill spatial constraints all make generative modeling for 3D scene synthesis and arrangement challenging. Current methods address these challenges autoregressively by using off-the-shelf diffusion objectives by simultaneously predicting all attributes without 3D reasoning considerations. In this paper, we introduce DeBaRA, a score- based model specifically tailored for precise. control- lable and flexible arrangement generation in a bounded environment. We argue that the most critical component of a scene synthesis system to accurately es- tablish the size and position of various objects within a restricted area. Based on this insight, we propose a lightweight conditional score-based model designed with 3D spatial awareness at its core. We demonstrate that by focusing on spatial attributes of objects, single trained DeBaRA model can be leveraged at test time to perform several downstream applications such as scene synthesis, completion and re-arrangement. Further, we introduce a novel Self Score Evaluation procedure so it can be optimally employed alongside external LLM models. We evaluate our approach through extensive experiments and demonstrate significant improvement upon state-of- the-art approaches in range of scenarios.
Introduction
Systems capable of generating realistic environments comprising multiple interacting objects would impact several industries including video games, roboties, augmented and virtual reality (AR/VR) and computer-aided interior design. As result and in tandem with the growing availability of synthetic datasets of indoor layouts [10, 44, 42, 63, 7], which can be populated with high-quality 3D assets [11, 63, 1]. data-driven approaches for automatically generating and arranging 3D scenes have been actively investigated by the computer vision community. Notably, the ongoing success of deep generative models for controllable content creation in the text and image domains has recently been extended to scene synthesis, allowing users to craft realistic indoor environments from a set of multimodal constraints [38, 37, 55, 54, 27,35]
Challenges associated with 3D indoor scene generation are numerous as the intricate nature of multi- object interactions is difficult to capture and model precisely. Items should be placed, potentially resized and oriented relative to one another, in way that is both plausible and aligned with subjec- tive and context-depender Additionally, objects should fit within bounded, restricted area, and a subtle mismatch can break the perceived validity of the synthesized environment (e.g., overlapping, floating or out-of- boundss
priors such as style, well as ergonomic and functional preferences.
Work done during internship at Dassault Systèmes.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

3D Layout Generation
Scene Ro-Arrangemmaa
Optimal Retrieval
amadad
Donshing
Dessiiing Sky 777 1)
ootte
Otject Adtooal
Scene Synthesis
Completion
Objeca Seauatile Gencration
Denciang Stop x(T 1)
LLMs
[illi dinine 16014]
Terplaxe
[ceting lamp)
Figure 1: Application scenarios overview. Besides generating diverse and realistic 3D indoor layouts, a single trained DeBaRA model can be employed to execute several related tasks by tweaking the initial sampling noise level max and/or performing object or attribute-level layout inpainting. Our novel SSE procedure enables 3D Scene Synthesis capabilities by efficiently selecting conditioning semantics from external sources using density estimates provided by the pretrained model.
objects, inaccessible areas). Finally, the limited availability of high-quality data [10, 42] requires learning-based approaches to make careful design choices and trade-offs.
Early data-driven approaches often rely on intermediate hand-crafted representations [43, 54, 36, 62] that are closely related to the considered dataset, which introduces significant biases. Concurrently, popular methods have been adopting autoregressive architectures that treat scene synthesis as generation task [55, 38, 27, 21, 37] by sequentially adding individual objects. More recently, score- score-based generative models (also known as denoising diffusion models) have shown promising capabilities various 3D scene understanding applications [18, 58] including controllable scene synthesis [51, 62, 60] and re-arrangement [57]. In contrast to previous methods, denoising- based approaches enable a stable and scalable training phase and can output scene attributes simulta- neously. The iterative sampling framework brings an improved consideration for the conditioning information and an attractive balance between generation quality and variety. However, current methods leveraging score-based generative models try to model all attributes (both categorical and spatial) within a single framework, which, as we demonstrate below, less data-efficient and leads to suboptimal solutions.
In this context, our work aims to establish principled and robust capabilities for generating accurate and diverse 3D layouts. Specifically, our key contributions are threefold:
1. We propose a score-based conditional objective and architecture designed effectively learn spatial attributes of interacting 3D objects in constrained indoor environment. In contrast previous approaches [5 38], we disentangle the design space and reduce the model's prediction to minimal representation consisting solely of oriented 3D bounding boxes, taking as conditioning input the room's floor plan and list of object semantic categories.
2. We propose set of approaches which allows model trained following our method to be flexibly employed at test time to perform several user-driven tasks enabling object or attribute-level control. In particular, we demonstrate strong capabilities on controllable scenarios such as scene re-arrangement or room completion, from a single trained network.
3. Finally, we introduce novel Self Score Evaluation (SSE) procedure, which enables 3I) scene synthesis by selecting the set of inputs provided by external sources, such as a L.M,, that lead to the most realistic layouts.
We exhibit our model's 's capabilities across a wide range of experimental scenarios and report state-of- the art 3D layout generation and scene synthesis performance.
2 Related Work
Score-based Generative Models By smoothly perturbing training examples with noise, Diffusion Models map complex data distribution to known Gaussian prior from which they sample back via

---

iterative denoising using neural network trained over multiple noise levels. This family of generative models has been motivated by several theoretical foundations over the past years: DDPMs [16. 33] parameterize the diffusion process as a discrete-time Markov chain, as opposed to continuous-time approaches [49, 48]. The seminal EDM [19, 20] training and sampling settings later unified previous methods into an improved ideal framework defined by set of interpretable parameters. Originally motivated by image generation, diffusion models have demonstrated impressive capabilities on various conditional tasks such text-to-image synthesis [34, 45], image-to-image generation from various 2D input modalities [45, 64, 56], text-to-3D asset creation [40, 23] or environment-awarr human motion synthesis [18, 24]. Relevant our work, diffusion models have been applied to the generation of point clouds [31, 52] and other geometric representations involving 3D coordinates [59].
Lifting Pretrained Diffusion Models Knowledge of trained diffusion models can be leveraged in various settings including content inpainting[30, 18], score distillation [40], exact likelihood computation [49, 19] or teacher-student distillation [47, 32]. More relevant to our work, image- domain diffusion priors have demonstrated compelling performance discriminative tasks including zero-shot image classification [22, 6, 5] and segmentation [4]. More precisely, Diffusion Classifiers assign a label, from finite set of possible classes {c,) to observed sample Do by computing cass-conditional density estimates from a pretrained diffusion model under the assumption of a uniform prior p (c,) 1/N. In practice, this is done by, for each class, iteratively adding noise the
observed sample Do and computing a Monte Carlo estimate of the expected reconstruction loss using the caas-conndi model.
Controllable 3D Scene Synthesis Synthesizing indoor 3D layouts from a partial set of information constraints has come in various settings depending on provided vs. predicted entities and enabled control granularity. A prolific line of research has been adopting intermediate 3D scene representations such as graphs [25, 43, 54, 36, 62, 13, 26]. furniture matrices [65] or multi-view images [35]. Autoregressive furnishing approaches [55, 38] have been supplemented by object attribute-level conditioning [37, 27] and additional ergonomic constraints [21]. However, their object time strategy does not comprehensively capture complex relationships between all the interacting elements and is known easily fall into local minima in which new items fail to be accurately inserted to the current configuration. Lately, methods have unfolded LLMs double-edged capabilities in this area [9, 61] as they excel generating sensible furniture descriptions while struggling in accurately arranging them in the 3D space, which [12] addresses by introducing a costly refinement stage. In the light of that, LLMs appear to be ideal candidates to supplement specialized 3D layout generation model.
Denoising Indoor Scenes Previous methods have explored diffusion-based approaches in the context of 3D scene synthesis. Pioneering their usage, LEGO-Net [57] performs scene re-arrangement (i.e., recovering clean object layout from noisy one) in the 2D space using transformer backbone that is not noise-conditioned, which we argue is the root cause of its main limitations. PhyScene augment diffusion- 3D scene synthesis with additional physic-based guidance enable practical embodied agent applications. Most relevant to our work, DiffuScene [51] achieves 3D scene synthesis by fitting a DDPM [16] on stacked 3D object features, resulting distribution that is hard to learn and interpret. It does not enforce spatial configurations over other predicted features. More importantly, its generative process is not conditioned on the room's floor plan (i.e., bounds) that constrains objects to be placed within a restricted area.
nigh-dimensional composite
3 Method
3.1 3D Scene Representation
Our method is based on encoding the state of a 3D indoor scene S that is defined by a floor plan (i.e., bounds) F and unordered set of N objects O = {o1 ...
ooN) )N each being modeled by its typed 3D bounding box o = {æ;, ) where e, E (0, 1} is the one-hot encoding of the semantic category among l classes and æ;; = (p;, r;, d:) E R comprises 3D spatial attributes. More specifically,
p, E R3 denotes the object's 's center coordinate position, r; = (cos 02, sin 0,) E R22 is continuous
encoding of the rotation of angle 0 around the scene's 's vertical axis [66] and d, E R23 is the dimension.
3

---

Ness-Awwre Encoler
Shared Spotial
Trush Scasz
(rn(O,0)
Fau
MIP
Mat
Add Noise or to Obtjers
x
Tranferns Eaaski
Floor
Scete
Panurhed Stone
MLP
Thirshle Modile
-0 Conditionnng Dripcut
Nosse Ensoter
Neise Levele
Encrding
Figure 2: DeBaRA architecture and training overview. At each iteration, 3D bounding boxes parameters (p, d) of indoor scene's objects O are perturbed with Gaussian noise de. The floor plan F, noise level a and resulting objects Og are processed by respective encoders form an unordered set of representations T fed as input to a transformer encoder. Novel object embeddings To are finally decoded back to their predicted clean spatial configuration (f, P, d). Trainable modules are optimized by minimizing a semantic-aware Chamfer loss. Input object categories c are randomly dropped to model both the class-conditional and unconditional 3D layout distributions.
3.2 Diffusion Framework and Architecture
We describe in this section our score-based layout generation framework, relevant design choices and network architecture, that are summarized Figure 2. Remarkably, unlike previous approaches [51, 38, 37] that output a range of attributes lying in different spaces, we focus accurately modeling 3D spatial layouts of bounded indoor scenes from a set of input object categories.
Learning 3D spatial configurations from object semantics We adopt diffusion- based approach to yield conditional generation model that outputs 3D object spatial features {æ;} from input floor plan and set of semantic categories y (F,c) with 3D spatial attributes are perturbed with Gaussian noise ~ N(0. I) at various noise levels (i.e., magnitudes) A trainable noise-conditioned denoiser model Do y. maps noisy spatial attributes I, = x + ae to their clean counterparts ă = æ E R
{c) During training,
We notice that each object spatial attribute has an individual real-world interpretation (e.g, and d can be expressed in meters, r in degrees). To preserve their measurable nature at intermediate perturbed configurations E.,, want our diffusion parameterization support a continuous range of noise levels, correlated to the scale of the input signal. This will be particularly convenient at test time (see Section 3.5). To guarantec both properties, context, this formulation is more natural than the DDPM framework employed by previous work [51]. The latter is based on discretizing noise levels and does not offer straightforward interpretability of the scene's 's state at arbitrary timesteps. Our parameterization is further detailed in Appendix A.
adapt the bbsed EDM [19] framework. In
Estimating the unconditional layout density Inspired by classifier-free guidance [17] in the image domain, we model both the class-conditional density p9 ( [z| c) and the unconditional density (=(FF 0) by single network of parameters 0. At each training iteration, we perform conditioning dropout on the set of semantic categories, by setting = 0 with probability Pamp else {c:) h= We found that this mechanism helps reduce overfitting of the training layouts Paata (a)aa())) and enables novel capabilities that we introduce in Section 3.4.
Denoiser Network Architecture Our lightweight architecture is inspired by previous work [57] to which we make key changes. Similar to [38] and [51], we use a shared object encoder in order to obtain per-object token To, as a concatenation of the object o; attributes embedded by sinusoidal positional encoding and linear layers. Following [57]. we uniformly sample P points on the edges of the floor plan and feed them into PointNet [41] model, resulting floor token Tr. This choice of feature extractor backbone is natural as it allows to maintain all the input scene's 's spatial features in

---

common 3D space. Importantly, noise token To is computed from the current noise level a, making our architecture aawree i.e., able to denoise layouts 2, any perturbation magnitude.
All the previously encoded tokens form sequence T= (Tr, To: .o....., Toy } from which a global scene encoder To computes rich representations F. We design the method without any token ordering and use padding mask for scene with fewer objects than the transformer capabilities. final shared decoder ML.P takes as input object tokens { and returns denoised spatial attribute values Φ [Pr,, Pú,
We provide implementation details on the denoiser in Appendix B.1. J
3.3 3D Spatial Objective
Our coise-conditioned model Dg is optimized towards novel semantic-aware Chamfer Distance objective that does not penalize permutation of 3D bounding boxes sharing the same semantic category between the predicted scene objects layout Ô and the ground truth one O:
(co( 1
Σ min l(o 0) + Σ min ((, DEO
(1)
2N
AEO
OEO
where ((ō, = - x x< (0(ō,
(2)
Here, is large value so that significant penalty is applied to objects that do not share the same semantic category c, preventing them to returned by the min operator.
We can finally rewrite the usual score-based training objective [49, 19] as:
(x)) LCD((() + oe; y, o), m)]
(3)
where ((r) is noise-dependent loss weighting function.
3.4 Self Score Evaluation
While specifying complete conditioning information such as the set of object semantics e could be tedious, be provided by either L.LM separately trained sequence generation model. However, using independent models inherently suboptimal since it does not guarantee that the generated conditioning input will be aligned with the score model knowledge. As a result, we propose a novel method to select conditioning inputs that are attuned with the model's capabilities.
More specifically, evaluate finite set of C object semantic categories candidates, where each candidate associated to 3D spatial layout sampled from the learned conditional density, i.e.,
candidates = (( (0,
j=1
Then, the optimal conditioning candidate e* derived from density estimate of corresponding 3D spatial layout æ" provided by the unconditional network:
x arg min Be,- [COD (Dx oc;
Algorithm 1 Self Score Evaluation
Require: a diffusion prior Dg trained with conditioning dropout and by optimizing LCD Input: conditioning candidates {c,} number of score evaluation trials T'se
1: sample 2; ~ (x [e| FF, ey) for each candidate ej using iterative sampling
2: initialize scores|c, = list() for each C₁
3: for trial 1, 1.... Tise do
4: sample σ ~ e ~~ (0,
5: for candidate Ct: sample 2 do
6: scores (ilaapend(CcppDaan + oc;; F,0. o), xxn))
7: end for
8: end for
9: return arg mitle mean(scores[c))
5

---

(a) ATISS [38]
(h) DiffuScene [51]
(c) DeBaRA
Figure 3: We compare method with established baselines for generating 3D layout from floor plan and set of object categories. DeBaRA produces less failure cases while consistently generating regular arrangements within the room's bounds.
In practice, we compute an unbiased Monte Carlo estimate of each candidate expectation using Tee fixed (a, e) pairs. Although similar in some aspects, SSE fundamentally differs from diffusion classifiers [22] as in our case, the uniform assumption over conditioning probabilities does not hold. Indeed, in our setting some input signals cannot lead to a plausible arrangement at all. As a result, density estimates of observed samples generated by the class-conditioned model are computed using the unconditional one, while diffusion classifiers compute density estimates of a single observed sample using the class-conditioned model. The SSE procedure is detailed in Algorithm I. It is further illustrated and discussed in Appendix C.
3.5 Application Scenarios
As shown in Figure 1, single trained DeBaRA model can be used at test time to perform multiple downstream interactive applications. Usual generation procedures, such EDM 2nd order stochastic sampler [19] can be applied using our trained denoiser to generate novel 3D layouts via T-step iterative denoising at discretized noise levels do
Fmax > ... OT 0.
In particular, several applications can be performed by inpainting [30], i... predicting missing spatial features from those specified (i.e., fixed) in the input layout E N To do so, we introduce binary mask m E {0, 1| N88 specifying values to retain from the input. The predicted layout at any sampling iteration can be expressed as:
io, = C.. ② (1 m) + Co. m
3D Layout Generation Novel and diverse 3D layouts be generated from input set of semantic categories e and floor plan F by sampling from a high initial noise level Fmax >> d arbitrarily initialized 3D spatial features Vo, and with m ONx8.
3D Scene Synthesis DeBaRA can perform 3D scene synthesis via 3D layout generation from semantic categories provided by external sources such as LLM [9]. Input conditioning candidates can further be optimally selected using the Self Score Evaluation procedure.
Scene Completion Additional objects o, can be inserted to an existing scene partially furnished with & objects o.. To do so, their 3D spatial attributes a. are inpainted from the existing ones we with Dg conditioned on the updated set of semantic categories c = Cellea using mi.jj = 1 ((SA))
Re-arrangement In the context of scene synthesis. re-arrangement [57] consists in recovering the closest clean spatial configuration of existing objects from messy one, which has practical applications in robotics [2]. DeBaR A can perform re-arrangement by sampling from an initial noise Fmax that depends on the scene perturbation magnitude. During denoising, object positions and rotations (p. r) E R are inpainted from the known object dimensions using m(i.)) 1 (z>5}-

---

(a) Ground Truth
(b) Noisy
(c) Re-arranged
(d) Partial
(e) Completed
Figure 4:
Qualitative results on scene re-arrangement (left) and completion (right). DeBaRA A able to recover plausible layout from a messy one, and to finely take into account initial configurations.
Optimal Object Retrieval 3D scene synthesis systems depend external 3D asset databases for furnishing rooms. For each object semantic class c, a textured furniture is retrieved by minimizing the mismatch with the generated dimension d This is inherently suboptimal scene quality is limited by the size of the external database. To overcome this issue, we introduce a post-retrieval refinement stage by performing additional re-arrangement steps starting from a noise level max derived from the mismatch between generated and retrieved object dimensions.
the resulting
Generation from Coarse Specifications We propose time-dependent masking approach to synthesize layouts from rough input spatial features (i.e., instead of exact ones), that are adjusted in the late denoising iterations. To indicate approximate e.g., object dimensions, we set, any sampling step f, m(i.)) j) = رsss and t<T;) )- The denoising step T, from which m is relaxed (i.e., set to 0) can be derived from its corresponding noise level ør, and the precision of specified input features.
4 Experiments
In this section, we provide comprehensive experimental evaluation of DeBaRA that we compare with established baselines from different model families, We also demonstrate the capabilities of our approach in various practical scenarios, enabling a wide range applications.
Datasets Our experiments are conducted on the 3D-FRONT [10] synthetic indoor layouts, furnished with assets from 3D-FUTURE [11] that we use as the object retrieval database. Out of the available room types in the dataset, we independently consider living rooms and dining rooms which are more densely furnished and feature complex floor plans. We follow the preprocessing from ATISS [38], leading respectively to 2338/587 and 2071/516 train/test splits..
Baselines We compare DeBaRA with ATISS [38] autoregressive transformer and DiffuScene [51] denoising network. To ensure a fair comparison with our method, we retrained both models with floor plan conditioning on each 3D-FRONT subset using their official implementations. To perform 3D arrangement generation with DiffuScene, we implemented DDPM inpainting [30] of object spatial features from their known semantic categories. Additionally, we report experimental results obtained by LayoutGPT [9] that we implemented with a Llama-3-8B backbone [8] that we also use to provide semantic categories in scene synthesis scenarios. Following the paper, we perform prompting with supporting examples: for each test scene, we retrieve top-k samples from the training set that have the most similar floor plan and include their spatial configuration as few-shot exemplars. Note that LayourGPT adopts training-free approach and is therefore not directly comparable to our method. However, we show how can be used alongside a specialized model such as DeBaRA, Full implementation details and LLM prompting strategies are reported in Appendix B.3.
Evaluation Metries We follow previous work [43, 38, 51, 9, 55] and evaluate the realism and diversity of generated arrangements by reporting the 256 Fréchet Inception Distance (FID) [15], Kernel Inception Distance (KID x1, 000) [3] and Scene Classification Accuracy (SCA, values closer to 50% are better) computed on top-down orthographic renderings. Resulting projections feature the scene's floor plan and objects colored according to their semantic class [51]. The generation spatial validity is further assessed by reporting the cumulated out of bounds objects area (OBA, in m² Related indicators are provided and discussed in Appendix D.I. Metrics are computed across each test subset. for which we generate the same number of scenes the number of real ones.
7

---

4.1 3D Layout Generation
The primary task of DeBaRA is generate diverse and valid 3D layouts within given floor plan and a list of object semantics. We showcase qualitative generation results and comparisons in Figure 3. As highlighted by previous work [57, 51], denoising-bare based methods better capture the interplay between interacting objects. We also observe that DeBaRA A largely outperforms baselines at respecting the scene's bounds while consistently producing more natural arrangements. These observations are quantitatively verified in Table and visualized in Figure 3.
Table 1: Quantitative experiment results on bounded 3D layout generation (providing floor plan and a list of object semantic categories). We compare our method against other learning- based approaches and additionally indicate results obtained from a training-free LayoutGPT.
Living Rooms
Dining Rooms
Methods
FID (1)
KID (J)
SCA (%)
OBA (L)
FID (J)
KID (1)
SCA (%)
OBA ())
LayoutGPT
35.53
13.69
72.8
2913.6
32.80
8.99
67.6
2447.4
ATISS [38]
25.67
8.91
71.8
857.3
28.05
9.26
63.2
702.4
DiffuScene [51]
21.54
6.40
69.7
341.1
23.06
5.35
57.7
266.4
DeBaRA (ours)
18.89
3.57
68.3
167.8
22.04
4.41
52.4
132,8
4.2 3D Scene Synthesis and Self Score Evaluation
We demonstrate competitive or state-of-the-art capabilities on 3D scene synthesis against methods that have been specifically trained for this task. We consider several settings depending on the source of input object categories and report our results in Table 2. First, we observe that randomly picking input semantics from the training set (Dataset Random) or taking the set c generated by LayourGPT [9] outperform baselines by significant margin on the 3D- 3D-FRONT living rooms test set. Then, to measure the individual impact of SSE, we compare a setup in which input semanties are selected from set of LLM-ggnnrated ones, either randomly (LLM) by applying SSE As LLMs often hallucinate or produce out-of -iit----iitribttionn sets, our procedure consistently improves realism and validity of the synthesized indoor scenes, which can also be qualitatively observed in Figure 5. These results further validate our choice to focus solely on 3D spatial features of objects.
Table 2: Quantitative experiment results on 3D scene synthesis. DeBaRA is evaluated in various settings based on the source of object semantic categories Precise settings are detailed and discussed in Appendix B.4. DeBaRA outperforms established baselines on most evaluation metrics.
Living Rooms
Dining Rooms
Methods
FID (L)
KID (L)
SCA (%)
OBA ())
FID ())
KID (L)
SCA (%)
OBA (J)
LayourCPP [9]
34.26
10.17
72.1
2902.7
37.78
11.31
60.2
1982.1
ATTSS [38]
27.02
10.99
73.0
848.4
28:26
9.28
58.2
759.1
DiffuScene [51]
21.64
5.94
66.0
323.1
23.85
5.66
54.6
289.8
LayouGPPT
20.97
3.53
69.8
193.0
26.67
7.14
56.6
151.8
DeBaRA
Dataset Random
19.52
3.53
67.6
159.0
25.45
5.11
52.5
139.5
LLM LLM +
21.58
3.53
72.4
154.3
27.09
7.38
60.5
140.4 44
DeBaRA
20.59
3.47
70.7
152.0
24.50
5.34
54.0
134.4
0,059
0,060
0,0723
0,078
0,0790
0,080
0,094
0,124
0,147
0,175
Figure 5: Top-down views of scenes generated by DeBaRA from several conditioning candidates provided by a LLM and their associated SSE values. We qualitatively observe that lower scores (green) corresponds to more natural layouts while higher scores (red) can be filtered out.

---

4.3 Other Application Scenarios
We present DeBaRA's capabilitiess at performing additional controllable tasks. Notably, we include quantitative (Table 3) and qualitative (Figure 7) experimental evaluations against LEGO-Net [57] on rerraneearnn Results highlight that our method is able to recover more realistic arrangements, while being closer to their initial, messy configurations, This is remarkable the LEGO-Net baseline has been specifically trained to perform this task.
Table 3: Quantitative evaluation on scene re- arrangement. DeBaRA is able recover more realistic arrangements, closer to their initial noisy configurations.
(a) Ground Truth
(b) Noisy
Distance
Method
FID (J)
KID (L)
Moved (1)
(c) LEGO-Net [57]
(d) DeBaRA
LEGO-Net [57] grad w/o noise
26.81
13.18
0.094
Figure 7: Qualitative comparison against LEGO-
DeBaRA
24.92
9.47
0.082
Net [57] on scene re-arrangement.
We also provide additional re r--arranemeet results and showease DeBaRA's scene completion capa- bilities, by inserting objects from a list of additional semantics, in Figure 4.
4.4 Ablations
We evaluate the individual contributions of some of our framework's 's key components on the base 3D layout generation task. Notably, results reported in Table 4 highlight the advantage of our novel objective (Section 3.3) over common formulations well as the benefits of modeling both the unconditional and class-conditional densities 3D layouts during training (Section 3.2).
Table 4: Ablation study DeBaRA training setup. We evaluate the individual impact, 3D layout generation, of different learning objectives & and of applying conditioning dropout with rate Pdrop- Notably, the of novel Chamfer distance results in significant performance increase.
Ablation Setting
Living Rooms
Dining
C(Ó, O)
Pdoop
FID (L)
(1)
SCA (%)
OBA (1)
(1)
KID (
SCA (%)
OBA (1)
MSE
0.0
21.66 21.66
6.55
70.9
237.0
23.89
5.51
136.5
CD standard
0.0
21.76
7.05
71.7
225.1
25.21
6.75
59.4
294.7
CD gemantic-aware (ours)
0.0
19.89
4.82
63.5
220.0
22.60
53.4
CD semantic-gware (ours)
0.2
18.89
3,57
68.3
167.8
22.04
4.41
$2.4
1328
4.5 Additional Results
Complex Floor Plans We notice that the 3D-FRONT dataset mostly contains simple floor maps (i.e.. single room, squared, rectangular) both for training and evaluation. As a result, we manually designed irregular floor shapes and report DeBaRA's generation in Figure 8, which further highlights the robustness of our method and consideration for the conditioning input.
Generated Floer Plan
Figure 8: Generated layouts from given set of objects and complex floor plans, selected from
the 3D-FRONT test set or handcrafted to irregular, out-t--distiibution shapes. While challenging. DeBaRA is able to output plausible layouts in which objects are scattered across the input floor plans.

---

Iterative Sampling We provide a visualization of the iterative denoising process over time when generating 3D layout from arbitrarily initialized object bounding boxes in Figure
Layout Timedepi
(init)
5
10
15
20
30
40
50
Result
Figure Visualization of intermediate layouts throughout the DeBaRA denoising process. Coarse object attributes (positions, rotations and dimensions) are determined in the early steps, and then refined the late iterations.
Generation Variety and Validity We also perform scene completion by adding bookshelf and coffee table, repeat the experiment ten times and report in Figure 10 the denoising object trajectories, intermediate and final positions (colored and black dots respectively). This allows observe the variety of predicted layouts. Notably, we can see that the bookshelf ends in various different positions, always next wall, while the coffee table, which functionally constrained by its surrounding objects, is placed at similar valid locations.
Table 5: Generation times are averaged on the 3D- FRONT [10] living room test subset. DeBaRA is im- plemented with T 50 sampling steps and Tse 100 Self Score Evaluation trials.
Network Parameters (10%)
Generation Time (s)
Method
ATISS [38]
0.160
Figure 10: Visualization of sampling trajec- tories and final positions of a pair of objects, inserted into scene across ten trials.
DiffuScene [51]
89.7
32.796
DeBaRA
12.2
0.488
DeBaRA SSE
12.2
0.894
Network Efficiency Finally, we compare the number of parameters as well as the sampling (i.e.,
generation) time, measured on the 3D layout generation task, of our DeBaRA backbone with those of other recent data-driven approaches in Table 5. We can see that our lightweight architecture is bridging the gap with autoregressive methods in terms of inference efficiency.
Conclusion, Limitations and Future Work
In this paper proposed DeBaRA, novel basee framework, which achieves state-of-the art results 3D layout generation. Our approach is distinctive in its design choices, which both favor data-efficiency with enhanced spatial reasoning, while, the same time, enabling range of applications such as scene re-arrangement and completion. Furthermore, we introduce a novel Self Score Evaluation procedure, which allows to leverage trained model to select the conditioning signals, which lead to the most plausible results. Overall, our work the first to unify the conditioning and prediction spaces of score-based models within the context of 3D generative layout.
While powerful, our method currently does not enforce physical constraints between interacting objects, which can lead to collisions. We also assume that object semantic classes are selected among a finite set of predefined categories. Finally, we do not enforce style consistency between objects, which can, nevertheless, be performed at retrieval time.
We believe that our approach can enhance other generative models (e.g., architectural layouts, images) by both evaluating the quality and by promoting more plausible 3D layout designs. Furthermore, will be interesting combine our approach with encoders from other modalities for unified multi-modal layout generation.
10

---

Acknowledgments and Disclosure of Funding
We thank Despoina Paschalidou for the preprocessing code base and Jiapeng Tang for the mesh rendering pipeline. We also acknowledge the anonymous reviewers for their insightful suggestions. This work was supported by Dassault Systèmes SE. The views and conclusions contained in the paper are those of the authors and should not be interpreted as representing official policies, either expressed or implied, of the company.
References
[1] Armen Avetisyan, Manuel Dahnert, Angela Dai, Manolis Savva, Angel X Chang, and Matthias Nießner. Scan2cad: Learning cad model alignment rgb-d seans. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2614-2623, 2019.
[2] Dhruv Batra, Angel Chang, Sonia Chernova, Andrew Davison, Jia Deng, Vladlen Koltun, Sergey Levine, Jitendra Malik, Igor Mordatch, Ronzheh Mottaghi, et al. Rearrangement: A challenge for embodied ai, arXiv preprint Xrv: 1.01975, 2020.
[3] Mikotaj Binkowski, Danica Sutherland, Michael Arbel, and Arthur Gretton. Demystifying mmd gans. arXiv preprint 180i 01 40100.10100000000 2018.
Ryan Burgert, Kanchana Ranasinghe, Xiang Li, and Michael S Ryoo. Peekaboo: Text image diffusion models are zero-shot segmentors. arXiv preprint arXiv:2211. 132.22444 2022.
[5] Huanran Chen, Yinpeng Dong, Shitong Shao, Zhongkai Hao, Xiao Yang. Hang Su, and Jun Zhu. Your diffusson model is secretly certifiably robust classifier. arXiv preprint arXiv:2402.0233 2024.
[6] Kevin Clark and Priyank Jaini. Text-to-image diffusion models are zero shot classifiers. Advances in Neural Information Processing Systems (NeurIPS), 36, 2023.
[7] Angela Dai, Angel Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-an annotaad 3d reconstructions of indoor scenes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5828-5839, 2017.
[8] Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandcy, Abhishek Kadian, Ahmad Al- Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.2178. 2024.
[9] Weixi Feng, Wanrong Zhu, Tsu-jui Fu, Varun Jampani, Arjun Akula, Xuehai He, Sugato Basu, Xin Eric Wang, and William Yang Wang. Layoutgpt: Compositional visual planning and generation with large language models. Advances in Neural Information Processing Systems (NeurlPS), 36, 2023.
[10] Huan Fu, Bowen Cai, Lin Gao, Ling-Xiac Zhang. Jiaming Wang, Cao Li, Qixun Zeng. Chengyue Sun, Rongfei Jia. Binqiang Zhao, et al. 3d-front: 3d furnished rooms with layouts and semantics. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 10933-10942. 2021.
[11] Huan Fu, Rongfei Jia, Lin Gao, Mingming Gong, Binqiang Zhao, Steve Maybank, and Dacheng Tao. 3d-future: 3d furniture shape with texture. International Journal of Computer Vision (IJCV), pages 1-25, 2021.
[12] Rao Fu, Zehao Wen, Zichen Liu, and Srinath Sridhar. Anyhome: Open-vocabulary generation of structured and textured 3d homes. In European Conference on Computer Vision (ECCV), pages 52-70. Springer, 2024.
[13] Lin Gao, Jia-Mu Sun, Kaichun Mo, Yu-Kun Lai, Lconidas Guibas, and Jie Yang. Scenehgn: Hierarchical graph networks for 3d indoor scene generation with fine-grained geometry. IEEE Transactions on Puttern Analysis and Machine Intelligence (TPAMI), 45(7):8902-8919, 2023.
[14] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770-778, 2016.
[15] Martin Hcusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to local nash equilibrium. Advances in Neural Information Processing Systems (NeurIPS), 30, 2017.
[16] Jonathan Ho, Ajay Jain, and Pieter Abbeel, Denoising diffusion probabilistic models. Advances in neural information processing systems (NeurIPS), (116640-651.. 2020.

---

[17] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:220.112598 2022.
[18] Siyuan Huang. Zan Wang, Puhao Li, Baoxiong Jia, Tengyu Liu, Yixin Zhu, Wei Liang. and Song-Chun 7hu. Diffusion-hased generation, optimization, and planning in 3d scenes. In Proceedings the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16750-16761, 2023.
[19] Tero Karras, Miik Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. Advances in Neural Information Processing Systems (NeurIPS), 35:26565 35226565-26677,, 2022.
[20] Tero Karras, Miika Aittala, Jaakko Lehtinen, Janne Hellsten, Timo Aila, and Samuli Laine. Analyzing and improving the training dynamics of diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 24174-24184, 2024.
[21] Kurt Leimer, Paul Guerrero, Tomer Weiss, and Przemyslaw Musialski. Layoutenhancer: Generating good indoor layouts from imperfect data, SIGGRAPH Asia 2022 Conference Papers, pages 1 1-8, 2022.
[22] Alexander C Li, Mihir Prabhudesai, Shivam Duggal, Ellis Brown, and Deepak Pathak. Your diffusion model is secretly zero-shot classifier. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 2206-2217, 2023.
[23] Jiahao Li, Hao Tan, Kai Zhang, Zexiang Xu, Fujun Luan, Yinghao Xu, Yicong Hong. Kalyan Sunkavalli, Greg Shakhnarovich, and Sai Bi. Instant3d: Fast text-to-3d with sparse-view generation and large reconstruction model. In The Twelfth International Conference on Learning Representations (ICIR), 2024.
[24] Jiaman Li. Alexander Clegg, Roozbeh Mottaghi, Jiajun Wu, Xavier Puig, and C. Karen Liu. Controllable human-object interaction synthesis. In European Conference Computer Vision (ECCV), 2024.
[25] Manyi Li, Akshay Gadi Patil, Kai Xu, Siddhartha Chaudhuri, Owais Khan, Ariel Shamir, Changhe Tu, Baoquan Chen, Daniel Cohen-Or, and Hao Zhang. Grains: Generative recursive autoencoders for indoor scenes. ACM Transactions on Graphics (TOG), 38(2):1-1,, 2019.
[26] Chenguo Lin and MU Yadong. Instructscene: Instruction-driven indoor scene synthesis with semantic graph prior. In The Twelfth International Conference on Learning Representations (ICLR), 2024.
[27] Jingyu Liu, Wenhan Xiong, Ian Jones, Yixin Nie, Anchit Gupta, and Barlas Oguz. Clip-layout: Style- consistent indoor seene synthesis with semantic furniture emhedding. arXiv preprint arX5v:2303.03555 2023.
[28] Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. In International Conference on Learning Representations (ICLR), 2017.
[29] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Reprexentations (ICLR), 2018
[30] Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11461-11471, 2022.
[31] Shitong Luo and Wei Hu. Difffusion probabilistic models for point cloud generation. Proceedings of the IELE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2837-2845, 2021.
[32] Chenlin Meng. Robin Rombach, Ruiqi Gao, Diederik Kingma, Stefano Ermon, Jonathan Ho, and Tim Salimans. On distillation of guided diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 14297-14306, 2023.
[33] Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffuusion probabilistic models. In International Conference on Machine Learning (ICML), pages 8162-8171. 1....I1..P.LRR.. 2021.
[34] Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob Megrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorcalistic image generation and editing with text-guided diffusion models. In International Conference on Machine Learning (ICML), pages 16784-16804. PMLR, 2022.
[35] Yinyu Nie, Angela Dai. Xiaoguang Han, and Matthias Nießner. Learning 3d scene priors with 2d supervision. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patiern Recognition (CVPR), pages 792 802, 2023.
12

---

[36] Wamiq Para, Paul Guerrero, Tom Kelly, Leonidas J Guibas, and Peter Wonka. Generative layout modeling using constraint graphs. In Proceedings of the IEEE/CVF International Conference Computer Vision (ICCV), pages 6690-6700, 2021.
[37] Wamiq Reyaz Para, Paul Guerrero, Niloy Mitra, and Peter Wonka. Cofs: Controllable furniture layout synthesis. In ACM SIGGRAPH 2023 Conference Proceedings, pages 1-11, 2023.
[38] Despoina Paschalidou, Amlan Kar. Maria Shugrina, Karsten Kreis, Andreas Geiger, and Sanja Fidler. Atiss: Autoregressive transformers for indoor scene synthesis. Advances Neural Information Processing Systems (NeurIPS), 34:12013-12022 2021.
[39] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems (NeurIPS), 32, 2019.
[40] Ben Poole, Ajay Jain, Jonathan Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. In The Eleventh International Conference on Learning Reprexentations (ICLR), 2022.
[41] Charles R Qi. Hao Su. Kaichun Mo. and Leonidas J Guibas. Pointnet: Deep learning point sets for 3d classification and segmentation. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), pages 652-66626666 660, 2017.
[42] Santhosh Kumar Ramakrishnan, Aaron Gokaslan, Erik Wijmans, Oleksandr Maksymets, Alexander Clegg, John M Turner, Eric Undersander, Wojciech Galuba, Andrew Westbury, Angel X Chang, Manolis Savva, Yili Zhao, and Dhruv Batra. Habitat-matterport 3d dataset (HM3d): 1000 large-scale 3d environments for embodied AL In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2021.
[43] Daniel Ritchie, Kai Wang, and Yu-an Lin. Fast and flexible indoor scene synthesis via deep convolu- tional generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6182-6190, 2019.
[44] Mike Roberts, Jason Ramapuram, Anurag Ranjan, Atulit Kumar, Miguel Angel Bautista, Nathan Paczan, Russ Webb, and Joshua M. Susskind. Hypersim: A photorealistic synthetic dataset for holistic indoor scene understanding. In International Conference on Computer Vision (ICCV), 2021.
[45] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10684-10695, 2022.
[46] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to- image diffuusion models with deep language understanding. Advances in Neural Information Processing Systems (NeurIPS), 35:36479-36494, 2022.
[47] Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. In International Conference on Learning Representations (ICLR), 2022.
[48] Yang Song. Conor Durkan, lain Murray, and Stefano Ermon. Maximum likelihood training of score-based diffusion models. Advances in Neural Information Processing Systems (NeurIPS), 34:1415-1428, 2021.
[49] Yang Song, Jascha Sohl-Dicksttin, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations (ICLR), 2021.
[50] Christian Szegedy, Vincent Vanhoucke, Sergey loffe, Jon Shlens, and Zhigniew Wojna. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2818-2828 2016.
[51] Jiapeng Tang, Yinyu Nie, Lev Markhasin, Angela Dai, Justus Thies, and Matthias NieBner. Diffussene: Denoising difffusion models for generative indoor scene synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 20507 215180058 2024.
[52] Arash Vahdat, Francis Williams, Zan Gojeic, Or Litany, Sanja Fidler, Karsten Kreis, et al. Lion: Latent point diffusion models for 3d shape generation. Advances in Neural Information Processing Systems (NeurIPS), 15:1002111000000000 2022.
13

---

[53] Ashish Vaswani, Noam Shazeer. Niki Parmar, Jakob Uszkoreit, Llion Jones. Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in Neural Information Processing Systems (NeurIPS), 30, 2017.
[54] Kai Wang, Yu-An Lin, Ben Weissmann, Manolis Savva, Angel X Chang. and Daniel Ritchie. Planit: Planning and instantiating indoor scenes with relation graph and spatial prior networks. ACM Transactions on Graphics (TOG), 38(4):1-15, 2019.
[55] Xinpeng Wang. Chandan Yeshwanth, and Matthias Nießner. Sceneformer: Indoor scene generation with transformers. In International Conference on 3D Vision (3DV), pages 106-115. IEEE, 2021.
[56] Daniel Watson, William Chan, Ricardo Martin Brualla, Jonathan Ho. Andrea Tagliasacchi. and Mohammad Norouzi. Novel view synthesis with diffusion models. In The Eleventh International Conference on Learning Representations (ICLR), 2022.
[57] Qiuhong Anna Wei, Sijie Ding. Jeong Joon Park, Rahul Sajnani, Adrien Poulenard. Srinath Sridhar, and Leonidas Guibas. Lego-net: Learning regular rearrangements of objects in rooms. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition (CVPR), pages 19037-19047, 2023.
[58] Zhennan Wu, Yang Li, Han Yan, Taizhang Shang. Weixuan Sun, Senbo Wang, Ruikai Cui, Weizhe Liu, Hiroyuki Sato, Hongdong Li, et al. Blockfusion: Expandable 3d scene generation using latent tri trlplane extrapolation. ACM Transactions on Graphics (TOG), 43(4): 17, 2024.
[59] Xiang Xu, Joseph Lamboume,, Pradcep Jayaraman, Zhengging Wang, Karl Willis, and Yasutaka Furukawa. Brepgen: A b-rep generative diffusion model with structured latent geometry. ACM Transactions on Graphics (TOG), 43(4):1 2024.
[60] Yandan Yang. Baoxiong Jia, Peiyuan Zhi, and Siyuan Huang. Physcene: Physically interactable 3d scene synthesis for embodied ai. Proceedings of the IEEF/CVF Conference on Computer Vision and Pottern Recognition (CVPR), pages 16262-16272, 2024.
[61] Yue Yang. Fan- Yun Sun, Luca Weihs, Eli VanderBilt, Alvaro Herrasti, Winson Han, Jiajun Wu, Nick Haber, Ranjay Krishna, Lingjie Liu, et al. Holodeck: Language guided generation of 3d embodied ai environments. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16227-16237, 2024.
[62] Guangyao Zhai, Evin Pınar Ömek, Shun-Cheng Wu, Yan Di, Federico Tombari, Nassir Navab, and Benjamin Busam. Commonscenes: Generating commonsense 3d indoor scenes with scene graphs. Advances in Neural Information Processing Systems (NeurlPS), 36, 2023.
[63] Genghao Zhang, Yuxi Wang, Chuanchen Luo, Shibiao Xu, Junran Peng. Zhaoxiang Zhang, and Man Zhang. Furniscene: A large-scale 3d room dataset with intricate furnishing scenes. arXiv preprint
urXiv:2401.03470, 2024.
[64] Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEEJCVP International Conference on Computer Vision (ICCV), pages 2023.
[65] Zaiwei Zhang, Zhenpei Yang, Chongyang Ma, Linjie Luo, Alexander Huth, Etienne Vouga, and Qixing Huang. Deep generative modeling for scene synthesis via hybrid representations. ACM Transactions on Graphics (TOG), 39(2):1 (112):1220 21, 2020.
[66] Yi Zhou, Connelly Barnes. Jingwan Lu, Jimei Yang. and Hao Li, On the continuity of rotation representa- tions in neural networks. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5745-5753, 2019.
14

---

Appendix
A Score-based Framework
In this section, we give additional details on the score-based parameterization that we adopt to learn the distribution of 3D layouts Pdota (x) and the sampling strategy used generate new samples from the resulting trained denoiser Dg.
A.1 Training
Score-based approaches model the score (i.e., the gradient of log probability density w.r.t the data) of marginal distributions Pa (z) obtained by perturbing the data with Gaussian noise e ~ (V I) at magnitudes In practice, the score can be effectively approximated by noise-conditioned denoiser =0/
that outputs clean samples from noisy ones, then, V log = (Do(æ:
Parameterizing the denoiser to output æ from its corrupted version directly is not ideal as the input magnitude varies greatly depending on the current noise level. Instead, Karras et al. [19] propose in their EDM diffusion framework preconditioning of the denoiser whose output is now derived from a trainable network Fg that either predicts the clean signal æ, the noise e or something in between, depending the value of o.. More formally, it can be expressed
(x y, σ) = Cakip (a) a + Com ()) Fo (C)((o) æ; y, Croise
The preconditioning function Cskip amplifies the network error little as possible while cin and Cout scale respectively the input and output to have unit variance. Following [19], we set:
a data
data
In(o)
Cskip
aamn + 02
Gin
V
o²
Cout
V + o2
Cuoise
4
(8)
Note that in our case, the value of ddaia should preferably be computed channel-wise for each attribute of x (p. r, d) RR as object positions, rotations and dimensions typically have different standard deviations. In practice, we compute & and od from the training data and arbitrarily set Jata (0.5, 0.5). During training, noise values a are drawn from a centered normal distribution of variance 0.25, which concentrates training on medium noise levels. Spatial values p and d of each training layout & are normalized based on the maximum extent of the scene's 's floor plan F. which ensures that all the network's inputs and outputs are scaled in [-1,1]. To model both the class-conditional and unconditional layout densities, we perform conditioning dropout on object categories e with a rate Phrop 0.2.
Finally, the training objective can be expressed by introducing our comantic-aware Chamfer recon- struction loss following Equation 3. As in EDM, \(o) 1/c to get uniform weighting across noise levels.
A.2 Sampling
At test time, the reverse SDE [19, 49] associated to the continuous-timg diffusion process is used to generate novel samples from a standard normal distribution using numerical solvers. It depends on the score approximated during training:
da: [fxx () - V log (e)) di + g(()dw
where [(; Rd + Rdd and ((-) IR + IR are respectively the drift and diffusion coefficients and w standard stochastic Wiener process.
In practice we use EDM [19] 2nd order Runge- Kutta stochastic sampler (see Algorithm 2), that resembles the predictor/correctorr framework from Song al. [49] and provides good trade-off between generation quality and number of function evaluations (NFE).
It is based T- T-sepp discretization of the reverse SDE [19, 49], with timesteps tie(O....TTII) decreasing from mmux (1 = 0) to Omin (i = T 1). Following [19], we use:
i - (amin m : )))² tr =
t = [
(10)
15

---

Algorithm 2 EDM Stochastic Sampler [19]
1: procedure LAYOUTSAMPLER
vo, (E(.....TT11) (e. -11}: Smoise)
2: io {O...., - do
3:
sample E, NO, S muise
4:
4 + 4 7:4
5:
ē; x; + v -
6:
9, - (a: (0, y. f.)) /
7:
2+41 f + (t++1 - tg)g
8:
f+++11 7 then
9:
9: t (ri+1 - Do (0i++11:30 ti+1)) / /141
10:
24+1 + i; + 2 (t++1 - tg; + g()
11: end if
12: end for
13:
end procedure
Note that and noise level can be used interchangeably. The parameter is tuned to dedicate more steps the denoising process to smaller or larger noise levels. The min value should be small enough that the model estimates the best approximation of the score and sample precise layout. On the other hand, max should be large enough to sample various layouts. The amount of fresh noise injected at the beginning of each denoising step is defined by Tie (O...... N-1-. Similar to Wei et al. [57] and Karras et al. [19], we qualitatively observed that adding noise in the final timesteps, i.e., when the layout is close to its final configuration, leads to less precise results. As result, additional Smin parameter is set so that 7/- = 0 when i; < Smin-
For 3D layout generation, we use T 50 timesteps and set max 1.0, Omin 0.005, p 7, and Smin = 0.005. Note that, although we perform conditioning dropout during training, we didn't 't find the need to amplify the strength of the input categories using any classifier-free guidance [17] scale at sampling time.
Additionally, Table 6 shows that sampling from DeBaRA using EDM [19] 2nd order stochastic procedure (Algorithm 2) outperforms ancestral DDPM [16] sampling using fraction of the denoising steps. This, combined with our lightweight architecture, enables real-time (<ls) generation.
Table 6: Ablation study on DeBaRA sampling strategy. generation task.
Metries are computed on the 3D layout
Sampler
Living Rooms
Dining Rooms
Alg.
Steps
FID (L)
KID (1)
(CE)
OBA (1)
Time (s)
FID (()
KID (J)
(%)
OBA (1)
Time (s)
1000
5.65
5.78
EDM
25
19.53
3.95
69.4
159.5
0.247
21.95
4.26
54.7
140.5
0.248
3.57
4.41
B
Implementation
We provide in this section additional implementation details on our model architecture and training configurations, illustrated in Figure 2. We also detail how baselines have been retrained and used test time to ensure fair and relevant comparison with our approach.
B.1 Network Architecture
Shared Object Encoder The shared object encoder embeds each object o, from its input 3D spatial values e; and semantic category c; Triplets of sealar values of the object's position p, and dimension d, are encoded with fixed sinusoidal positional encoding of 32 frequencies following [57]:
(E(s) = {sin (128)/311
//11 s), cos(128/)))) /31 122///2/1111
31 } E R64
(11)
Applying this module projects P, and r; in 192 is similarly applied r; = (cos (0,), sin():)) to get a feature in I 128 that is additionally fed to linear layer to obtain 192-dimensional attribute.
16

---

The object semantic class C; represented as a one-hot vector among k classes is encoded in 1 by an MLP with 2 a hidden layer of 128 units and LeakyReLU activation. Respective object spatial and semantic encodings are then concatenated to form an object token To, of dimension 4 x 192 768.
Floor Encoder The scene's 's conditioning floor plan F is embedded by a PointNet [41] module, similar to [57]. To do so, we first extract the floor's 2D polygon using the output of ATISS [38] preprocessing and sample P = 100 evenly spaced points on its contour. The PointNet backbone produces 1024-dimensional feature, that is further passed to a linear layer get the appropriate
floor token Ty E 768
Noise Level Encoder We encode the noise level a as a token To E applying (o( and a linear layer with LeakyReLUU activation.
IR 768 obtained by subsequently
Transformer Encoder Our transformer encoder that computes new representations F is composed of multi-head head self-attention and feedforward layers. following the original paper [53] and implemen- tation from the PyTorch [39] API. Importantly, we don't 't enforce ordering of any input token and pass an additional padding mask to handle sequences of different lengths. We stack 3 encoder layers, each having 4 attention heads and a feedforward hidden dimension of 512.
Shared Object Decoder The final shared object decoder produces the network's 's predicted spatial values at, IR& for each of the N objects from their respective To, embeddings. It is implemented as an MLP with three layers of 512, 128, and 8 units, using LeakyReLU activations and a dropout rate of 0.1.
B.2 Training Protocol
During training, the network is optimized towards our semantic-awar Chamfer loss, that can be efficiently implemented with appropriate broadcasting. We trained our models separately on the 3D-FRONT [10] living room and dining room subsets for 3000 epochs, with a batch size of 32 and monitor the validation loss to avoid overfitting the training set in the late iterations. We use the AdamW [29] optimizer with its Py PyTorch default parameters and learning rate 17 = 10 scheduled with a linear warmup phase for the first 50 epochs, starting at r x 0.01. Following this, a cosine anncaling schedule [28] reduces 17 to minimum of 10 over 2200 epochs. Finally, we randomly apply rotations to the training scenes as data augmentation.
B.3 Baselines
ATISS ATISS [38] is an autoregressive, permutation-invaaie invariant transformer that treats 3D scene synthesis as an unordered set generation task. The model is natively conditioned on the room's 's floor plan, from which it extracts features using ResNet-18 [14] applied on a top-down binary projection. The model predicts the semantic class, location, rotation and dimension of the next object to be inserted to the current layout configuration. As our method, it also supports inserting objects from their semantic categories given as input, which is the setting that we used to report experimental results on the 3D layout generation task (see Table I and Figure 3). We retrained the model on each 3D-FRONT subset using the authors' implementation."
DiffuScene DiffuScene [51] employs a DDPM to perform 3D scene synthesis, by learning to denoise unordered sets of objects that are each represented by all their attributes, i.e., location, size, orientation, semantic category and shape code. Although the paper specifically mentions not being conditioned on the room's 's bounds, we found out the official implementation supports this feature that we enabled to retrain the model on the 3D-FRONT subsets, with other settings set to those of the authors. In practice and similar to ATISS [38]. ResNet-18 backbone is used extract features from the floor plan's 's projection mask. The resulting encoding is passed to an MLP whose output is added to the diffusion timestep embedding, as in [46]. To assess the effectiveness of this conditioning mechanism and validate the relevance of this baseline, we report metrics obtained for the 3D layout
http:///ggicaabboooo///aa/////// pytorch
hhttps://githu/
http:://ggthhuubooooooeeeeeee htt//nv-tlabs/ATIl ATISS
17

---

generation task, using both floor-conditioeed and an unconditional (vanilla) trained DiffuScene model on the living room subset in Table Note that, top-down view of scenes, that we use to evaluate e.g., FID and KID scores, feature the floor plan, which penalizes generated configurations that don't properly take into account.
Table 7:
Quantitative impact of DiffuScene [51] floor plan conditioning on 3D layout generation. Results indicate that the additional input has successfully been learned, resulting in solid baseline.
Model
FID (1)
KID (J)
OBA (m=
DiffuScene vanilla
41.30
22.92
1621.5
DiffuScene floor
21.54
6.40
341.1
To perform 3D layout generation from input semantic categories using DiffuScene (Table Figure 3), we implemented DDPM impainting of the object spatial features from their categories as an additional method within the official implementation" and using default sampling settings with 1000 timesteps.
LayoutGPT LayoutGPT [9] is a training-free free approach that utilizes Large Language Models to generate layouts both in the image and the 3D scene domains, demonstrating competitive performance with learning-based approaches on 3D scene synthesis. To do so, the method consists in prompting LLM with specific instructions and by adding supporting examples from the training set, i.e., few-shot exemplars expected, valid layouts. These examples are retrieved from the train set based floor plan similarity computed from the binary masks with test sample. We reimplemented LPy using the official implementation" for exact prompt and supporting examples retrieval, but using Meta Llama-3-8B backbone instead of ChatGPT variants for local execution and better reproducibility.
To perform 3D layout generation, include in the prompt the list of object semantic categories. Here a typical LayoutGPT prompt for this task on the living room subset:
Instruction: synthesize the 3n layout of an indcor scene. The generated 3D layout should follow the cSS style, where each 1ine starts with the furniture category and is followed by the 3D size, orientation and absolute position. Formally, each 1ine should follow the template:
FURNITURE (length: 2m: width: ?m; height: 3my left:
3my top: ?m; depth: 2m/
orientation: ? degrees;
All values rues
meters the orientation angle is in degrees.
Condition:
Room Type: livingroom
Room Size: max length 1.00m, max width 1.53m
Furniture categories: console_tate
sofa
Serner_side__ table
corner side sidletable saffee_table tv_stand pendant_lamp
Layout:
console, table (length: 0.28m; height: 0.31m; width: 0. 0.14m; orientation: -90
degrees; left: 0.93m; top: 0. 0.41m; depth: 0.15m/
[..]]
[OTHER SUPPORTING EXAMPLES]
Condition:
Room Type: livingroom
Room Size: Tength 1.00m, max width 1.62m
Furniture categories: tv sv_stand corner corner_side side table coffee table dining_taile table dining chair dining chair dining_ohie chair dining_chali armchair pendant_ 1amp pendant lamp multi seat seat_sofa sofa
Layout:
For 3D scene synthesis, we follow the paper and include the training set's object frequencies in the prompt. We set the LLM sampling temperature to 0.7 and maximum output tokens to 1024. We
wtwp:hhwww..ooeenciiieeeeeeeeeee 'nttpe://hugiingface co/neta- Laaettaallaaa/Mttte Maaaaa//eata---a--a--aa Llama-3 3-8B
18

---

User Inpue
LLM
Template
a~N00.0
XT
Caregories Conditioning
hco
4 (desi, arnchoii dresser.... ... bed]
(friee, nug, co stand
DaBaRA (0.F)
jable, kemp, dryer, stonge)
Lco
Candidates Ranking I
Soene Conditionng
:
G
Acco
SSE allows to officiently select a conditioning afigned the knowledge pretrained diffiasiom model.
Iterttive condirional
One-slop unconditional decoising at level
sampling
Evalustion
Figure 11: Self Score Evaluation (SSE) Pipeline. The method allows to leverage the knowledge of model trained following our method to select valid sets object categories.
report LayoutGPT performance on this task in Table 2, where we also indicate metrics obtained by DeBaRA when using conditioning input the same semantic set the generated by LayoutGPT for the corresponding test scene. this setup. we largely outperform the baseline and even report state-of-the-art FID, KID and OBA scores 3D-FRONT living room.
LEGO-Net The LEGO-Net [57] model is specifically designed to perform 2D scene re-arrangement, ie.. recover close clean layout configuration from messy, perturbed one. It is trained using regression loss on object position and rotation values, and proposes a Langevin dynamics-like iterative sampling procedure. In order produce the results reported in Table implementation. in the grad without noise setting (which the best performing in the original paper), on the 3D-FRONT living room test subset and with scene perturbation level of 0.25.
used authors'
B.4 LLM Prompting
As mentioned in the main submission, we perform 3D scene synthesis using DeBaRA conditioned on LLM-generated sets of object semantics, that we optionally select via Self Score Evaluation (Table 2, Figure 5). practice, the generated categories have been obtained using Llama-3-83 [8] following the LayoutGPT [9] prompting strategy. We experienced asking the language model to generate solely lists of object semanties using a few supporting examples and providing the dataset statistics, but we noticed that it was more prone to hallucinate and drift towards inconsistent generations than when generating complete layout configurations (i.e., including object position and orientation values).
The DeBaRA LLM reported in Table corresponds to the setup where we filter from the LLM- generated sets of categories those having the same number objects the considered test scene and randomly select one to condition DeBaRA. The DeBaRA LLM + SSE setting is similar, but instead of picking set randomly, the selection is performed by applying the SSE procedure.
B.5 Computational Requirements
All the training and evaluation experiments as well as the computation of generation times reported in Table 5 have been performed on single NVIDIA RTX A6000 GPU, When comparing number of network parameters and generation times with those of ATISS [38] and DiffuScene [51], J, we notice that DeBaRA is bridging the gap with autoregressive methods in terms of inference efficiency. This made possible by our restricted output space that requires more lightweight backbone as well as our choice of sampling procedure that leads favorable NFE/ generation quality tradeoff.
Phttps: peattp://githhn htt.aa/..o.ooiitoommmmiieeeeeeee
19

---

C
Self Score Evaluation
In this section, provide additional content regarding our SSE procedure (Section 3.4), by illustrat- ing and by further assessing its expressive power.
C.1 Pipeline
The SSE formulation is expressed by Equations 4 5. It follows the procedure outlined in Algorithm 1. We additionally illustrate the SSE pipeline in Figure 11. In our experiments, SSE is implemented using se 100 trials, with noise levels drawn training.
C.2 Additional Evaluation
We additionally evaluate the expressive power of the SSE procedure in toy binary classification task: for each scene of the test set, we create a corrupted version by replacing a proportion Prand of the scene's object categories by random ones. We report the binary classification score obtained by SSE when asked to discriminate the corrupted set of semantics from the ground truth one. We perform the experiments 10 times account for the inherent stochasticity of the experimental setup and report the results for several values of Prand in Table 8.
Table 8: SSE performance at discriminating ground truth sets of categories from perturbed ones.
Perturbation
None
Single
Prond 0.35
Prand 0.50
Prand 0.75
All
Accuracy (%)
50.6 + 4.39 71.7 + 2.20 73.8 + 3.38
78.5 + 1.84 80.9 + 2.62 84.0 + 2.90
We observe a significant gap accuracy between the control experiment (none object is perturbed, meaning that the sets equals) and the setting where only single object has been swapped. It means that SSE is able to identify subtle misalignments between the network's 's knowledge and the provided conditioning candidates.
D Additional Results
In this section, we provide supplementary quantitative indicators measuring the validity of generated layouts with respect to the conditioning floor plan. We also showcase additional qualitative results and comparisons in multiple application scenarios.
D.1 Bounding Metrics
In addition the cumulated of bounds objects area (OBA, in m²) reported in Table 1, we indicate the rate of scenes having at least one object out of its bounds (OBR), and the cumulated number of boo-nds- objects in the generated layouts (OBN) in Table For OBR and OBN, we consider an object to be out-of- bounds if at least 20% of its 2D bounding box surface is outside the floor's 's limits.
Table 9:
Quantitative bounding metrics on the 3D layout generation task. We observe that DeBaRA consistently better at respecting the indoor floor plan by significant margin.
Living Rooms
Dining Rooms
Methods
OBA (L)
OBR (L)
OBN (L)
OBA (L)
OBR (L)
OBN (L)
LayoutGPT [9]
2913.6
0.695
2119
2447.4
0.659
1720
ATISS [38]
857.3
0.744
1195
702.4
0.891
1603
DiffuSeenc [51]
341.1
0.652
742
266.4
0.628
640
DeBaRA (ours)
167.8
0.390
497
132.8
0.403
401
20

---

D.2 Qualitative Results
(a) ATISS [38]
(b) DiffuScene [51]
(c) DeBaRA
Figure 12: Additional 3D layout generation results. We compare our method by generating layouts from list of object semantic categories and floor plan. DeBaRA consistently produces more realistic arrangements while respecting the room's outline.
21

---

(a) ATISS [38]
(b) DiffuScene [51]
(c) DeBaRA Dataset Random
(d) DeBaRA LayouGPT
(e) DeBaRA SSE
Figure 13: Additional scene synthesis results. We compare DeBaRA with state-of-the-art ap- proaches in various settings. Since object semantics are not part of our output space, they are randomly drawn from the training dataset (Dataset Random), generated by an external LLM (LayourGPT) or selected from LLM-ggnerated sets by our density estimate procedure (SSE), which helps to get more natural layouts for the considered floor configuration.
22

---

(a) Ground Truth
(b) Perturbed
(c) Re-arranged
Figure 14: Additional re-arrangement results. Our method can recover a plausible arrangement from a noisy one using iterative sampling with fixed dimension attributes and low initial noise level.
Partial
Completed
Figure 15: Additional scene completion results. From list of additional objects semantics,
DeBaRA is able to finely introduce the relevant items into an existing layout.
D.3
Statistical Analysis
We can observe in our qualitative results, e.g., in Figure 3, that objects are most the time parallel or perpendicular to walls, meaning that their rotation attributes r mainly take cardinal angle values, i.e., 0%, 90". 180 or 270".
Radial histograms of object rotation values reported in Figure 16 indicate that 3D-FRONT [10] subsets contain rooms in which objects, in their vast majority, exhibit this property and don't have much exotic angles, i.e., different from cardinal ones. These restricted rotation distributions are consequently learned and reflected in the outputs of generative models at sampling time.
23

---

Living Rooms
Dining Rooms
15W
(a) Dataset
(b) DeBaRA
(c) Dataset
(d) DeBaRA
Figure 16:
Distributions object rotation values extracted from the dataset or generated following method, reported for both test subsets. We can that the original data predominantly features objects rotated by 0%, 90º, 180 or 270 around the scene's s vertical axis, which, as a result, is also the tendency induced by the generative model.
E
Societal Impact
We believe that our method will predominantly yield positive societal impact by enhancing accessi- bility, controllability and efficiency in 3D indoor design and creation, benefiting both non-experts and professionals in several industries.
Nevertheless, it raises common concerns associated with deep generative models. Specifically, users should be warned about using the model for furnishing real-world rooms as generated layouts may result in unsafe designs. Additionally, privacy safeguards should systematically be implemented along our method in order to prevent unauthorized replication of personal spaces.
Licenses
In this section, we list licenses of datasets, open-source code artifacts and pretrained models that were used in the context of our experiments.
Datasets
3D-FRONT [10]: 3D-FUTURE [1]]:
CC BY-NC-SA 4.0 License CCBY-NC-SA 4.0 License
Code
ATISS [38]:
NVIDIA Source Code License for ATISS
DiffuScene [51]:
Sony Group Corporation License for DiffuScene
LayoutGPT [9]:
MIT License
LEGO-Net [57]:
MIT License
PointNet [41]:
MIT License
Models
Llama-3-8B [8]:
Meta Llama 3 Community License Agreement
Inception-v3 [50]:
Apache 2.0 License
"attpe://huggingffce.co http:///nnnnnnccceeeaaaaaaaaaaaa /tter///////ggaaoooooooeoeaee/aa L1ama- 3 wtt/////maainlL/aaaaaa.

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Claims made in the abstract are all developed throughout the paper, notably in the method Section 3.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
We discuss the paper's main limitations in the last Section 5, along with the conclusion.
Justification:
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model cell-speciiiiatinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
25

---

Answer: [Yes]
Justification: For our main theoretical contribution, Self Score Evaluation (Section 3.4), we clearly state that it is made possible under the assumption that the denoiser models both the unconditional and the class-conditional data distributions.
Guidelines:
The answer NA means that the paper does not include theoretical results, All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We comprehensively detail our experimental setup, both for our method and the baselines in the main paper, Section 4, and in Appendix and B.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
26

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [No]
Unfortunately, the code for DeBaRA cannot be disclosed due author affiliation. However, we comprehensively describe our settings and implementation details in Appendix A and and the 3D-FRONT [10] dataset is openly available.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: CooeeSummii / //nips... //nips.cc/pppppp wttpccccpppcpcppeeeeeeecccc httpuuiciguuddesssppeeppppeeaail cttpguuudde/ooeeeubiiiiiiii issionni ssssssolicyy for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: All the relevant details given in the paper, notably in Section 4, well as in Appendix B.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification:
Unfortunately, we couldn't 't perform such analysis due to limited computational
resources.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
27

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
Question:
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: We indicate our GPU setting in Appendix B.5 and report network execution time in the main submission (Table 5).
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https http//neriip..// http://wtrrs...pppueeeeeeeeeeeee cc/ppblic/Ethicssuueeeee
Question:
Answer: [Yes]
Justification: The research conducted in the paper conforms, in every respect, with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [Yes]
Justification: Potential societal impacts, both positive or negative, are discussed in dedi- cated section in Appendix F. We also discuss possible ways to mitigate potential negative socictal impacts.
28

---

Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (c.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., denloymemt of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Creators owners of assets either credited via citation of their work (e.g., 3D-FRONT [10] and 3D-FUTURE [11] datasets) or direct link to their code or model (throughout Appendix B). A dedicated section in Appendix F explicitly mentions the relevant licenses, which have been properly respected to conduct our work.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
29

---

The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi wwwrsswithhoeee thcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's ' creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the datasct/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing research with human subjects.
Guidelines:
30

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
31


Reviews:
[{"review_id": "hEgQAXz0QW", "reviewer": "Reviewer_xNz4", "summary": "This paper proposes a method for room layout generation given objects and a floor plan using score-based EDM. First, three encoders are used to encode objects, the floor plan and the noise respectively. These encoded latents are then given as input to a noise based scene encoder which decodes into the output latents. The output latents are then finally decoded to their respective categories and the decoded output is pushed to be closer to the input as measures by a proposed semantic aware Chamfer distance loss. Once trained, the model is able to generate novel layouts and scene in addition to being able to perform completion, rearrangement and retrieval", "questions": "N/A", "limitations": "Yes", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1) Qualitatively, the layouts generated seem far more coherent and spatially aligned than prior work.\n\n2) Quantitative results provide further evidence that this model performs well.\n\n3) The paper is well written and easy to follow.", "weaknesses": "1) While using a permutation invariant Chamfer loss is intuitive, I believe it should still be ablated w.r.t just the standard chamfer loss", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank reviewer `xNz4` for their time and feedback. We address the reported weakness in the following response:\n\n> While using a permutation invariant Chamfer loss is intuitive, I believe it should still be ablated w.r.t just the standard chamfer loss\n\nEvaluating the impact of our semantic-aware objective against a standard Chamfer loss is a relevant suggestion. Advantage of our novel formulation is quantitatively verified in **Table 1** of the rebuttal `PDF`. It is also evaluated against a *simple* MSE objective.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ToWc3yTWgW", "reviewer": "Reviewer_Fsdh", "summary": "This paper studies 3D room arrangement/layout generation. It proposes DeBaRA, a diffusion-based generative model, which can generate layouts given the list of furniture and the floor map of the room. The proposed method provides good results and is able to work in various scenarios: layout generation, LLM-guided text-to-scene generation, scene completion, etc.", "questions": "Please see \"Weaknesses\".", "limitations": "The authors mentioned some limitations in the paper and did not indicate the societal impacts.\n\nOne possible societal impact might be that \"the generated layouts may lead to unsafe constructions, and therefore the model should given warnings about actually using it for constructing real-word rooms\".", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The proposed method is a novel diffusion-based generative model, which is effective in the aimed task.\n- The results generated are high-quality both qualitatively and quantitatively.\n- An LLM-guided alternative pipeline is provided to simplify the input format, which adds more functions to the model.", "weaknesses": "- There is no ablation study of the design choices. It is unclear whether and how each design choice and component helps the results.\n    - It would be better if the ablation results could be provided for the following designs: EDM v.s. DDPM/DDIM, designed 3D spatial objective v.s. simple MSE, different classifier-free guidance scales, with v.s. without SSE.\n- Some of the designs are not clearly described in the paper. Besides, some of the assumptions are not very clear.\n    - I wonder whether adding noise to the scene will cause the noisy layout to contain overlapped objects, objects of unreasonable sizes/rotations, etc. I did not see any of these in Fig.1. It might be better if a visualization of the process of scene being denoised could be provided.\n    - I wonder how the size is determined by the generative model. For example, a sofa can be long or short, a table can be large or small, and a cabinet can be as tall as the wall or as small as a bedside table. How will the model decide the sizes of these objects? Can the user indicate the rough size of the object?\n    - I wonder if it is assumed that most objects should be located in a rigid way that each edge is parallel to one wall? These are observed in most of the results of the proposed model in Figs. 3~6. If so, will these limit the diversity of generation results?\n- It seems that all the floor maps are relatively simple. I wonder how the model can work in more complicated floor maps, e.g., a floor with many rooms, a round room, or a triangle room.\n- In contrast to the claim in Q4 of the Checklist, many implementation details are not revealed, e.g., training settings (e.g., lr and iterations) and classifier-free guidance scales.\n- (Minor) The reason why this diffusion model can work might be that it can be regarded as an extension from some point cloud diffusion models, which also directly apply denoising on coordinates or some explicit presentations. However, this direction was not discussed in the related work.\n- (Minor) There are some minor math presentation issues. For example, multiple $min$s should be corrected to $\\min$s (e.g., L166, L178).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank reviewer `Fsdh` for their time and positive feedback. We address the reported concerns in the following response:\n\n> It would be better if the ablation results could be provided for the following designs: EDM v.s. DDPM/DDIM, designed 3D spatial objective v.s. simple MSE, different CFG scales\n\nWe provide an **ablation study** of these design choices in **Table 1** and **Table 2** of the rebuttal `PDF`. Note that as we don't apply *CFG* at sampling time, we chose to ablate the use of *conditioning dropout* on the input object categories during training.\n\n> Some of the designs are not clearly described in the paper\n\nOne general note to clarify some of the following concerns is that unlike previous work, **we don't prevent unwanted behaviors** such as object collisions, out-of-bounds or misaligned elements **using additional loss terms** [1] or **rigid / hard-coded rules**. Instead, we adopt a purely *data-driven* approach to capture complex patterns solely from the training layouts. Remarkably, results reported in **Table 5** of our submission indicate that DeBaRA largely outperforms previous methods at respecting the indoor floor plan (i.e., keeping objects within its *bounds*).\n\n > I wonder if it is assumed that most objects should be located in a rigid way that each edge is parallel to one wall?\n\nIt is not. The 3D-FRONT dataset that we use for training predominantly contains rooms in which objects are aligned with walls and don't have much *exotic* angles (i.e., different from 0° or 90°). We will be happy to include a **statistical analysis** of the training and generated data in the revision of our manuscript to quantitatively support our observations. Additionally, on **Figure 6** (right), we perform scene completion by adding a *bookshelf* (pink) and a *coffee table* (blue). We repeat the experiment ten times and report the denoising object trajectories, intermediate and final positions (colored and black dots respectively). This allows to observe the variety of predicted layouts. Notably, we can see that the *bookshelf* ends up in various different positions, always next to a wall.\n\n> I wonder how the size is determined by the generative model. [...] How will the model decide the sizes of these objects?\n\nDuring the denoising process, **coarse spatial attributes** are typically determined during the **early iterations** (i.e., high noise levels, injection of *fresh* noise), while **precise / fine-grained features** are set in the **late time steps** (low noise levels, no injection of *fresh* noise). Notably, the added stochasticity in the early denoising steps helps better explore the space of possible features. These general assessments on diffusion sampling have been explored by other work in the context of image generation [2]. They can be qualitatively observed in **Figure 3** of the rebuttal `PDF`.\n\n> Can the user indicate the rough size of the object?\n\n**Yes**, DeBaRA can be used to sample layouts from specified (i.e., fixed) spatial features such as object dimensions, as described in Section 3.5, using the binary mask $\\mathbf{m}$ of L209. If users want to input the *rough* size of objects (i.e., instead of exact one), $\\mathbf{m}$ can be *relaxed* (i.e., set to $\\mathbf{0}$) in the late iterations to let the model adjust fine grained dimensions. Denoising time step from which $\\mathbf{m}$ is relaxed can be set depending on the precision of user-defined sizes. **We find this reviewer suggestion to be a very practical and intuitive use of our method that further highlights its versatility and we will be happy to include this in the final version**.\n\n> It might be better if a visualization of the process of scene being denoised could be provided.\n\nInjecting noise may produce such invalid intermediate 3D configurations in early time steps (i.e., when spatial features are far from their final values). However, these phenomena will tend to diminish during the denoising  process, thanks to the decreasing noise schedule. These can be better observed in **Figure 3** of the rebuttal `PDF`.\n\n> I wonder how the model can work in more complicated floor maps\n\nThe 3D-FRONT dataset mostly contains relatively *simple* (i.e., single room, square, rectangular) floor maps both for training and evaluation. As a result, we haven't encountered any *round* floor map in our exploration of the test set. Consequently, we manually designed such out-of-distribution floor shapes (round, triangular and report DeBaRA's generation in **Figure 2** of the rebuttal `PDF`. This demonstrates the robustness of our method to unseen rooms.\n\n> Many implementation details are not revealed, e.g., training settings (e.g., lr and iterations) and CFG scales.\n\nPlease note that these details are extensively **described throughout the submitted appendix**. For instance, we can read in Section B.2, L520-522 that we trained our models for 3000 epochs, with a batch size of 128 using the AdamW optimizer and learning rate $1e^{-4}$. During training, we use a conditioning dropout rate of 0.2 but didn't find the need to amplify the strength of the conditioning using any classifier-free guidance scale during sampling.\n\n> it can be regarded as an extension from some point cloud diffusion models\n\nThank you, methods leveraging diffusion models to generate point clouds [3] or other geometric representation involving 3D coordinates [4] are a relevant addition to our manuscript's references.\n\nWe thank again reviewer `Fsdh` for their detailed and constructive feedback as well as for their insightful suggestions that will improve the quality of our paper.\n\n\n[1] DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesis, Tang et al., 2024.\n\n[1] Exploring Diffusion Time-steps for Unsupervised Representation Learning, Yue et al., 2024.\n\n[2] Diffusion Probabilistic Models for 3D Point Cloud Generation, Luo et al., 2021.\n\n[3] BrepGen: A B-rep Generative Diffusion Model with Structured Latent Geometry, Xu et al., 2024.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sjSmklP9ZG", "reviewer": "Reviewer_sfxx", "summary": "The authors present DeBaRa, a diffusion-based framework for indoor scene generation, and a self-score evaluation strategy to select conditioning input. They demonstrate the effectiveness of their approach on scene synthesis and several downstream tasks.", "questions": "+ The inference time is reported to be significantly faster than DiffuScene (about 100 times faster). Are these advantages due to the use of EDM over DDPM? Please elaborate on how the choice of EDM contributes to the improved inference time.", "limitations": "N/A", "rating": 3, "confidence": 5, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "+ The empirical results are promising.", "weaknesses": "+ Insignificant Architectural Difference: The architectural differences between DeBaRa and other diffusion-based baselines (e.g., DiffuScene) are not substantial. The results benefited architecture difference is not sufficiently supported by experimental evidence. To be honest, we **do not** care about final performance, but the impact of the differences.\n+ Lack of Implementation Details and Ablation Studies: The paper does not provide sufficient implementation details, making it difficult to reproduce the results. Additionally, ablation studies are missing, which are crucial to understanding the contributions of individual components of the model.\n+ Missing Experiments: The paper claims advantages in downstream tasks like rearrangement and completion but lacks comparative experiments with baseline methods (ATISS, LEGO-Net, and DiffuScene) both qualitatively and quantitatively.\n+ Unclear Writing and Missing Sections: The writing is unclear in several parts of the paper. Additionally, **Section 4.4 (\"Additional Results\") is missing**, which further reduces the clarity and completeness of the work.\n+ Typing Errors: There are numerous typing errors throughout the paper, such as \"biaises\" instead of \"biases\" in Line 39. These errors affect the readability and professionalism of the manuscript.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank reviewer `sfxx` for their time and feedback. We address the reported weaknesses and questions in the following response:\n\n> Insignificant Architectural Difference: The architectural differences between DeBaRa and other diffusion-based baselines (e.g., DiffuScene) are not substantial.\n\nFirst, we would like to emphasize that although we do not consider our neural network architecture to be a key contribution, there are some significant differences between ours and that of DiffuScene [1]. Specifically, unlike DeBaRa, [1] adopts a U-Net backbone with 1D convolutions. It is not conditioned on the floor plan of the room and therefore performs *unbounded scene synthesis*. Our architecture features fixed positional encoding modules, linear layers, a Transformer encoder as well as a PointNet feature extractor and is therefore close the one of LEGO-Net [2] (which is not a Diffusion model), as deliberately stated in our main submission (L146).\n\nMore fundamentally, however, and as mentioned in the summary above as well as in the paper, **we consider our key contributions to lie in**: \n1. a continuous-time score-based model for indoor layout generation, which output domain has been simplified to learn unconditional and class-conditional densities of object bounding boxes, expressed in a common 3D coordinate space (**Section 3.2**). It is trained following\n2. a novel Chamfer objective that is permutation-invariant by design (**Section 3.3**).\n3. an original Self Score Evaluation (SSE) procedure to optimally select conditioning inputs from external sources, leveraging density estimates provided by the pretrained model, allowing our method to be the first to unify the use of a LLM and of a specialized diffusion model in the context of 3D scene synthesis (**Section 3.4**).\n\nThese are the key novelties of our work, which do not exist in DiffuScene or any other prior approach, and which allow us to achieve state-of-the-art capabilities in 3D layout generation (**Table 1**), 3D scene synthesis (**Table 2**) and scene re-arrangement (`PDF` **Table 3**), while adopting a backbone that is significantly more lightweight than previous methods, which further enables real-time (< 1s) efficient sampling (**Table 3**). \n\n> The paper does not provide sufficient implementation details, making it difficult to reproduce the results.\n\nImplementation details are extensively described throughout the paper and submitted appendix, notably:\n\n- **Section A.1**. Denoiser training parameterization.\n- **Section A.2**. EDM sampling procedure (Algorithm 2) and hyper parameters (L487-488).\n- **Section B.1**. Network architecture with positional encoding formula and number of frequencies, number of layers of linear modules, their activations, dropout rate, output dimensions as well as details on the Transformer implementation with number of encoder layers, their hidden dimensions, number of heads and token masking strategy.\n- **Section B.2**. Details on the training protocol, including the number of epochs, batch size, optimizer, learning rate, learning rate schedulers, and data augmentation routine.\n- **Section B.3**. Details on the (re)implementation of baseline methods.\n- **Section B.4**. Link to the model and implemented prompting strategy in our LLM-guided scene synthesis pipeline.\n- **Section 3.4**. Pseudo-code for SSE (Algorithm 1).\n- We follow the data preprocessing from ATISS [3], as stated L224.\n\nWe also include additional implementation details in our general response, but would be happy to answer implementation-related questions if remains any.\n\n> The paper claims advantages in downstream tasks like rearrangement and completion but lacks comparative experiments with baseline methods\n\nIncluding additional experimental results against established baselines on downstream task is a valuable suggestion. We include **quantitative and qualitative experiment results against LEGO-Net**, which itself outperforms ATISS [3] on the scene re-arrangement task in **Table 3** and **Figure 1** of the rebuttal `PDF`, as stated in our general response.\n\n> Unclear writing and typing errors\n\nWe thank reviewer `sfxx` for reporting a typographical error and missing reference to the table and figure of section 4.4 (making the dedicated section appear empty in the paper). This issue and other minor typos, have been **corrected in the current version of the manuscript**. We will also make sure to incorporate **every** additional comments in the final version.\n\n> Please elaborate on how the choice of EDM contributes to the improved inference time.\n\nAs stated in our manuscript (L474-476), EDM proposes a 2nd order Runge-Kutta stochastic sampler (Algorithm 2) that provides a **favorable trade-off** between generation quality and number of function evaluations (NFE), as verified throughout the quantitative evaluations of the seminal paper [4]. Our implementation uses 50 sampling steps, which results in a NFE of 101. On the other hand, DiffuScene uses ancestral DDPM sampling with 1000 steps / function evaluations. Additionally, **our original design choices enable our transformer-based backbone to feature around 7 times fewer parameters** (Table 3), which has a direct impact on inference time.\n\n\n[1] DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesis, Tang et al., 2024.\n\n[2] Lego-net: Learning regular rearrangements of objects in rooms, Wei et al., 2023\n\n[3] ATISS: Autoregressive Transformers for Indoor Scene Synthesis, Paschalidou et al., 2021\n\n[4] Elucidating the Design Space of Diffusion-Based Generative Models, Karras et al., 2022", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
# Response to all reviewers 
We would like to thank reviewers for their time and insightful feedbacks and are pleased that they recognized our submission to propose a **"novel diffusion-based generative model, which is effective in the aimed task**" with an "**alternative pipeline which adds more functions to the model**" (`Fsdh`), in a way that is **"well written and easy to follow"** (`xNz4`). Our experimental evaluations have also been appreciated, as we read that **"results generated are high-quality both qualitatively and quantitatively / seem far more coherent and spatially aligned than prior work"** (`Fsdh`, `xNz4`).

Before addressing reviewers' common concerns and questions, we wish to briefly highlight our main contributions:
1. a lightweight score-based model trained to learn the class-conditional and unconditional densities of 3D layouts in bounded indoor scenes using a novel 3D spatial objective.
2. a novel Self Score Evaluation (SSE) procedure to optimally select conditioning inputs from external sources using density estimates provided by the pretrained model.
3. a flexible sampling method to perform multiple downstream tasks from partial features (e.g., scene completion) and/or intermediate noise levels (e.g., scene re-arrangement, object retrieval).

All these contributions are key to achieving the state-of-the-art performance exhibited by our framework. Our method is also the first to unify the use of a specialized diffusion model and a separately trained LLM in the context of 3D scene synthesis.
## 1. Ablation study
`sfxx`, `Fsdh`, `xNz4`: Reviewers unanimously suggested that additional ablations of our design choices would further clarify our contributions. 

Please note that in **Table 2** of our main submission and as described L259, we compare a scene synthesis set-up in which the input object semantics are selected from a set of LLM-generated ones, **either randomly** (*LLM*) or **by applying SSE** (*LLM + SSE*). Additionally, **Table 3** reports the impact of applying SSE on the generation time. **These results quantitatively measure the individual impact of SSE in our scene synthesis pipeline**.

We provide a study to evaluate the role of **other individual components** in the attached `PDF`:
- `Fsdh`: Our 3D spatial objective v.s. simple MSE (`PDF` Tab 1.)
- `xNz4`: Our 3D spatial objective v.s. standard Chamfer loss (`PDF` Tab 1.) 
- `Fsdh`: Use of conditioning dropout during training (`PDF` Tab 1.)
- `Fsdh`, `sfxx`: Different sampling strategies (DDPM, EDM) (`PDF` Tab 2.)
## 2. Additional experimental results
We follow the reviewers' suggestions and provide new results to support the performance and versatility of our method.
- `sfxx`: We include quantitative (`PDF` Tab 3) and qualitative (`PDF` Fig 1) experimental evaluation against LEGO-Net [1] on **scene re-arrangement**, i.e., recovering a close *clean* layout configuration from a *messy* / perturbed one. We use authors implementation [2], in the *grad without noise* setting (which is the best performing in the original paper), on the 3D-FRONT *living rooms* test subset and with a perturbation level of $0.25$. Results highlights that **our method is able to recover more realistic arrangements**, while being **closer to their initial configurations**. This is remarkable as the LEGO-Net baseline has been specifically trained to perform this task. 
- `Fsdh`: Additional qualitative results on **complex floor plans** can be observed on (`PDF` Fig 2), further highlighting the robustness of our method and its consideration for the conditioning input.
- `Fsdh`: As suggested, we provide additional visualization of the iterative denoising process over time (`PDF` Fig 3). 
## 3. Implementation details
`sfxx`, `Fsdh`: We would like to emphasize that the vast majority of our implementation details are provided in the **supplementary materials** of our submission, notably in **Section A** (training parameterization and sampling hyper parameters) and **Section B** (network architecture, training protocol, baselines (re)implementation and LLM prompting).

Minor details which were omitted are as follows:
- Our *Shared Object Decoder* linear layers have respective output dimensions 512, 128 and 8.
- We use a popular PyTorch implementation [3] of PointNet for our floor plan feature extractor. 
- Our linear learning rate warmup uses a start factor of 0.1 and is active for the first 50 iterations. Then, the cosine annealing schedule is set to reach a minimum learning rate value of $1e^{-8}$ after 2200 epochs.
- We implemented SSE using $T=100$ trials and with noise levels sampled as in training (L459-460).
## 4. Text body and typos
- `Fsdh`: We find reviewer proposal mentioning potential **societal impacts** of our method to be an excellent addition. We will also add that robust privacy measures should systematically be implemented along our method in order to avoid any unauthorized replication of personal spaces. 
- `sfxx`: As reported, missing references to **Table 3** and **Figure 6** in the text body make **Section 4.4** appear empty in the submitted paper. We will clarify our manuscript by adding a concise textual introduction and analysis of these results. Notably, we can see in **Table 3** that our lightweight architecture is bridging the gap with autoregressive methods in terms of inference efficiency. **Figure 6** shows the denoising process of predicted bounding boxes being progressively *unshadowed* (left). It also allows to observe the variety of predicted layouts by plotting intermediate and final positions (colored and black dots respectively) of objects that are added to a scene over multiple trials (right).
- `sfxx`: We fixed the mentioned **typo** along with other ones identified during post-submission readings of our manuscript.

[1] Lego-net: Learning regular rearrangements of objects in rooms, Wei et al., 2023

[2] https://github.com/QiuhongAnnaWei/LEGO-Net

[3] https://github.com/fxia22/pointnet.pytorch

Meta-review: The original ratings for this paper were borderline accept (xNz4), weak accept (Fsdh), and strong reject (sfxx). All reviewers appreciated the evaluation results. The common concerns were the lack of ablation studies, additional experimental results, and implementation details. Additionally, reviewer sfxx uniquely raised concerns about the novelty and readability, which were contrary to the opinions of reviewers Fsdh and xNz4. After the rebuttal, all the previous concerns were well addressed. All reviewers increased their ratings to weak accept (xNz4), accept (Fsdh), and reject (sfxx). Reviewer sfxx once again uniquely raised additional concerns that were not mentioned in their initial review or prompted by the authors' rebuttal. The authors still adequately addressed these additional concerns in their response, but reviewer sfxx did not have the opportunity to increase their rating. For the reasons above, the AC recommend accepting this paper on the condition that all revisions from the first rebuttal are included in the camera-ready version.
Decision: Accept (poster)