SceneCraft: Layout-Guided 3D Scene Generation

Authors: xiuyu  yang
yunze  man
jun-kun  chen
yu-xiong  wang
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 01:05:05

Abstract: The creation of complex 3D scenes tailored to user specifications has been a tedious and challenging task with traditional 3D modeling tools. Although some pioneering methods have achieved automatic text-to-3D generation, they are generally limited to small-scale scenes with restricted control over the shape and texture. We introduce SceneCraft, a novel method for generating detailed indoor scenes that adhere to textual descriptions and spatial layout preferences provided by users. Central to our method is a rendering-based technique, which converts 3D semantic layouts into multi-view 2D proxy maps. Furthermore, we design a semantic and depth conditioned diffusion model to generate multi-view images, which are used to learn a neural radiance field (NeRF) as the final scene representation. Without the constraints of panorama image generation, we surpass previous methods in supporting complicated indoor space generation beyond a single room, even as complicated as a whole multi-bedroom apartment with irregular shapes and layouts. Through experimental analysis, we demonstrate that our method significantly outperforms existing approaches in complex indoor scene generation with diverse textures, consistent geometry, and realistic visual quality.

Main paper: 
 introduction: 
 The generation of diverse and complex 3D scenes plays a critical role in enhancing virtual and augmented reality (VR/AR) experiences, video game development, and the advancement of humancentric embodied AI. However, manually creating these complex 3D scenes is a tedious procedure that requires extensive knowledge and proficiency in 3D modeling tools [13, 14] . The recent success of 2D generative models [23, 50, 55] fuels the development of a line of text-to-3D work [31, 46, 63, 66] . Although these methods have achieved impressive object generation performance, scaling from objectlevel to scene-level generation presents significant challenges. It involves managing a considerably larger space with complicated semantics while ensuring 3D consistency (in terms of shape, texture, occlusion, etc.) across various camera perspectives.Recent advances in scene-level 3D generation [17, 24, 37, 60, 75] have opened new pathways for creating larger-scale virtual environments. Most work leverages image inpainting [17, 24, 75] or multi-view diffusion methods [37, 60] to optimize a text-guided 3D scene. While generating locally convincing textured meshes, these methods share two common drawbacks: (1) Focusing on local coherence, they often struggle to accurately depict geometrically consistent rooms with plausible layouts and rich semantic details. (2) Conditioned only on textual prompts, these methods fall short in terms of offering precise control over the entire scene's composition and arrangement. Although some concurrent research [16, 45, 53] has explored the generation of an indoor environment conditioned on user-defined 3D layouts, it is restricted to creating small-scale compositions involving multiple Figure 1 : Our novel method generates complex and detailed indoor scenes from 3D spatial layouts and textual descriptions. Given user-specified layouts represented as a "Bounding Box Scene (BBS)," our method renders batches of 2D layouts and coarse depth maps and then transforms them into high-quality 3D scenes.objects [12, 45] , or lacks the ability to generate multiple rooms with complex layouts, shapes, and free camera viewpoints [16, 53] due to the use of panoramic representation.In this paper, we introduce SceneCraft, a novel method designed to generate high-quality indoor scenes conditioned on user-specified free-form layouts. A high-level illustration of our work is shown in Figure 1 . Our method features two key innovative designs:User-Friendly Semantic-Aware Layout Control. Central to our approach is the utilization of 3D bounding boxes to guide the layouts of the target space, namely a "bounding-box scene (BBS)," which allows users to design complex and free-form room arrangements with simple bounding boxes. With this layout format, users can easily define both the spatial arrangement and the placement of objects within a room, as constructing a building in the Minecraft game. And SceneCraft leverages this preliminary design to generate a detailed and realistic scene. We surpass previous methods in supporting complicated indoor layouts beyond a single room, even as complicated as a whole three-story house with multiple layers and irregular rooms.High-Quality Complex Scene Generation with a 2D Diffusion Model. Our framework excels in creating 3D scenes by leveraging the advanced generation capabilities of our pretrained 2D diffusion model, SceneCraft2D. SceneCraft2D takes the "bounding-box images (BBI)" rendered from BBS as a condition through ControlNets [76] to generate high-fidelity views of the room that follow the given simple prompt like "This is one view of a [style description] room." By obtaining high-quality multiview images through SceneCraft2D, we successfully distill a high-resolution 3D representation [56] of the generated indoor scene.Trained with multi-view indoor scene datasets [49, 72] , our work achieves state-of-the-art 3D indoor scene generation performance, both quantitatively and qualitatively. We present the first effective framework to generate complex text-and layout-guided 3D-consistent scenes with free camera trajectories and diverse semantics. In summary, our technical contributions are threefold:• We propose a novel layout-guided 3D scene generation framework to create complicated indoor scenes adhering to user specifications, being the first to operate on free multi-view trajectories and free from the constraints of using panoramas. • We introduce the "bounding-box scene" as a user-friendly format to scratch a desired room as easy as building homes in the Minecraft game, which provides accurate geometry control. • We design a high-quality 2D diffusion model, SceneCraft2D, to generate high-fidelity and high-quality rooms following the rendered "bounding-box image" from the "bounding-box scene," and to support the generation of various styles via text conditioning.With all these contributions, our SceneCraft achieves high-quality generation of various fine-grained and complicated indoor scenes that have not been supported by previous work. 
related work: 
 Learnable Scene Representation. Traditional scene representations [1, 11, 22, 28, 40, 44, 54, 64] directly model the 3D geometry information of the scene and thus suffer from limited flexibility or low rendering quality. The neural radiance field (NeRF) [41] pioneers a neural network-based scene representation, providing the ability to reconstruct a complete and precise 3D scene from only multi-view images and corresponding camera parameters. Follow-up variants of NeRF [2, 6, 7, 20, 56, 62, 65, 67, 69, 73, 74] aim either to improve the original framework in different aspects, e.g., rendering quality and training efficiency, or to support additional tasks, e.g., relighting ability and editing ability. Recently, 3D Gaussian Splatting [27] outperforms the NeRF-family representation with high rendering quality and efficiency. In our work, we use a learnable scene representation as the backbone to model the output. As any representation can be used in our framework, we choose Nerfacto [56] for its high-quality rendering of complicated large-scale scenes.Diffusion-Guided Text-to-3D Generation. The recent successful 2D diffusion-based generative models [3, 4, 23, 50, 51, 55] have inspired a series of innovative text-to-3D methods [10, 31, 43, 46, 59, 66, 75] to distill powerful 2D pretrained models for 3D content creation. DreamFusion [46] proposes the score distillation sampling (SDS) module to optimize scene representations, e.g., NeRF [41] or Gaussian Splatting [27] , of objects by denoising their rendered views. Building on top of DreamFusion, SJC [63] , Magic3D [31] , and ProlificDreamer [66] alleviate the oversaturation problem and improve the generation quality. Despite impressive results, these methods are restricted in generating small-scale objects without complex semantic composition. More recently, Text2Room [24] , SceneScape [17] , and Text2NeRF [75] propose to extend object generation to scene generation with off-the-shelf text-image inpainting models [39, 50] , where they iteratively inpaint unseen parts of the scene from novel camera perspectives. Another work [37] proposes progressively distilling a text-conditioned indoor panorama generation model [60] using different groups of camera views to optimize a scene representation. However, all of these methods lack semantic control over the generation output other than a simple text prompt. Hence, they cannot be used in the creation of 3D scene models where users want to specify the structure and layouts of the environment. In comparison, our work learns to generate 3D scenes that adhere to user-specified room layouts and textual descriptions, allowing precise control over the environment.Scene Generation with Semantic Guidance. A recent line of work has studied 2D generation with semantic guidance for better controllability [9, 15, 18, 30, 50, 71] . Attempting to extend image generation to 3D creation, prior work has studied single image to 3D object reconstruction [33, 34, 36, 38, 47, 57, 68] and single image to video reconstruction [5, 61] . These methods face great challenges in 3D consistency, due to the lack of large scene-level datasets for training and the use of the auto-regressive generation paradigm. Meanwhile, Set-the-scene [12] , CompoNeRF [32] , and Compo3D [45] learn to generate object compositions from semantic layouts with the SDS method [46] . Discoscene [70] aims to disentangle the scene and then perform object-level scene editing leveraging the layout priors. DiffuScene [58] and GraphDreamer [19] utilize scene graphs together with textual descriptions as conditions to generate compositional 3D scenes. However, these methods are restricted to generating small-scale scenes composed of only several objects. They also neglect representations of walls, doors, ceilings, and ground, which are essential in defining indoor scenes but difficult to control in generation. Close to our work are three concurrent methods, ControlRoom3D [53] , Ctrl-Room [16] , and UrbanArchitect [35] . The first two methods generate 3D room meshes from user-defined or estimated layouts with multi-view diffusion followed by a monocular depth estimation process. While achieving outstanding generation performance, they rely on panorama images [60] as their preliminary results, which not only simplifies the scene generation problem, but also limits the complexity of their room layouts and diversity of their camera viewpoints. Our method, by learning a 3D-consistent multi-view generator and a scene renderer without viewpoint constraints, is able to generate more complex and consistent scenes with diverse camera trajectories. UrbanArchitect [35] focuses on street-view scene generation with semantic-aware layout controls. However, it allows for greater geometric approximation due to simpler conditions: fewer object categories, sparser and non-overlapping object placement, and more predictable camera trajectories. In contrast, indoor scenes feature dense objects that overlap with more fine-grained categories, which are all effectively addressed in our method. 
scenecraft: methodology: 
 Our SceneCraft is a novel method for text-and layout-guided scene generation. As illustrated in Figure 2 , the input to SceneCraft consists of (1) a prompt as a coarse description of the target scene's style and content, (2) a "bounding-box scene" (BBS) serving as the layout guidance of the target scene, and (3) a camera trajectory defined in the space of BBS. SceneCraft renders the BBS in the camera trajectory to construct "bounding-box images" (BBI) as the layout condition for a pretrained 2D diffusion model "SceneCraft2D" to generate high-quality 2D images of the scene. With the high-quality images generated by SceneCraft2D, SceneCraft is able to use an SDS-equivalent paradigm [46] to aggregate them into a scene representation (e.g., NeRF [41] or 3D Gaussian splatting [27] ) of the generated 3D scene. Notably, our SceneCraft does not require a panoramic view. Instead, our camera view can move freely in the 3D space, enabling the generation of much more complicated indoor layouts consisting of multiple rooms, unlike prior work which only supports single-room scenes.To provide a user-friendly format for free-form indoor layouts, we design the bounding-box scene (BBS) representation. As shown in Figure 2 , BBS is similar to the "Proxy Room" of Control-Room3D [53] , but each object in the scene can be represented by a union of several intersecting bounding boxes in BBS with a category label, to indicate the coarse shape and category of an object. This provides users with the ability to indicate the shape of the object, e.g., an L-shaped or even an S-shaped desk, while still maintaining the freedom of using a single bounding box for generation.BBS can be regarded as a draft or a coarse version of the scene. In order to generate the actual room accurately conditioned on BBS, we use a distillation-guided framework. Each view of the generated scene corresponds to a 2D generation task, conditioned on the "bounding-box image (BBI)" of the same view in BBS, where each pixel of BBI contains both the semantic category and the depth of the pixel in BBS. By rendering BBS into BBI on the projected camera trajectory provided, we decompose the layout-guided 3D scene generation task into a set of layout-guided 2D image generation tasks, with BBI as conditions. To solve these tasks, we propose SceneCraft2D, a 2D diffusion model for high-quality layout-guided 2D image generation.Augmented SD for BBI Conditions. Our SceneCraft2D is augmented from Stable Diffusion [50] , with an additional BBI condition at the current viewpoint, which contains both the semantic category map and the BBS depth map. The semantic map (converted to one-hot vectors based on the category) and depth map are injected into the model, as conditions via two separate ControlNets [76] .Finetuning. We finetune the augmented Stable Diffusion with scenes in indoor datasets like ScanNet++ [72] and Hypersim [49] . Each scene is converted to a generation task by generating the prompt, converting its semantic point cloud into a BBS, and using the camera trajectory provided by the dataset. We split the generation task into several 2D generation tasks at each view in the dataset, and train the SD model with these tasks. During the finetuning process, instead of using existing caption tools such as BLIP [29] to generate prompts, we use a single base prompt for all training samples. During inference-time generation, our model supports more specific and customized scene-specific prompts to produce the results that users desire. Note that the base prompt does not need to contain any information describing the image content, it merely serves as a placeholder to avoid the model overfitting to any particular word or sentence. Specifically, we use "This is one view of a room." as the base prompt and user-desired target prompts like "This is one view of a bedroom in Van Gogh painting style." for generation. The results showed that this method effectively controls the style of the generated outputs via prompts while maintaining a good layout-conditioned generation.After finetuning, SceneCraft2D can generate high-quality images according to the given BBI and text prompt.Distillation Process with Annealing. To generate 3D scenes, we distill the generation ability of our pretrained SceneScraft2D model in a Score Distillation Sampling (SDS) [46, 63] -equivalent pipeline.Unlike the vanilla SDS [46] that works in the latent space and directly works with gradients, our pipeline applies an IN2N [21] -style, which is proven SDS-equivalent by HiFA [78] . In this pipeline, we maintain a multi-view dataset for continual scene representation training while simultaneously and iteratively replacing the multi-view dataset with newly generated images by SceneCraft2D. Through this process, the multi-view dataset will be gradually replaced with views of the generated scenes, which are used to fit the scene representation towards the generation.Within this pipeline, we also propose an annealing-based distillation strategy inspired by [39, 78] , for a more efficient and high-quality distillation. Leveraging the SDEdit method [39] to control the similarity of generated images with the currently modeled scene, we gradually decrease this similarity along with the entire distillation procedure. In other words, at an early stage of distillation, SceneCraft2D can freely generate the room to satisfy the BBS and the prompt; while at a later stage, by generating similar but higher-quality images, SceneCraft2D can also serve as a refiner of the scene representation to refine the rendering result and improve the scene representation. With this pipeline, our SceneCraft is able to generate high-quality scenes.Layout-Aware Depth Constraint. When generating a complex indoor scene based on free camera trajectories, learning a reasonable geometry of the scene from scratch is both crucial and challenging. However, we have prior knowledge of the BBS input, which allows the model to quickly capture the geometry of the scene through the layout-aware depth constraint. Specifically, at the initial stage of distillation, we add a normalized depth loss L depth , where the pseudo-supervision signal comes from our BBS input. We set a soft threshold δ that allows the pixel depths D render modeled by the scene representation to fluctuate within a reasonable range around the pseudo-ground truth depths D layout . This ensures that the model quickly converges to an initial coarse geometry. This loss is modeled in the following form:EQUATION)Later in the distillation process, we disable this loss term to allow the model to learn more fine-grained geometry.Floc Removal with Periodical Migration. The images generated at the initial steps of the distillation process have a lower consistency, which can result in blurry flocs close to the surface and in the air when "averaging" inconsistent multi-view images on the scene representation side. At a later stage, even when the diffusion's output is relatively 3D-consistent with annealing, the flocs, with condensed volume density, are still hard to remove and may result in Janus problems. Therefore, instead of "fixing" flocs issues in the original scene representation, we propose a method to migrate the current relatively coarse scene to another scene from scratch, to obtain a finer version. After the Our method is able to generate complex and free-form scenes from challenging room layouts.first several iterations as early-stage training, we begin to maintain two scene representations, S c and S f , to indicate the previous coarse representation and the mitigated fine representation, respectively. We freeze S c and generate new images to supervise S f by generating images similar to S c 's rendering results (by only applying t < T noise adding steps), to refine S c and store into S f with the diffusion model's generation. We also periodically update S f with S c (with a smaller interval of training iterations) to synchronize the latest information in both two scene representations. With the periodical migration method, we achieve more and more fine-grained and clear scenes during the training procedure.Texture Consolidation. The generation of high-quality images by our SceneCraft2D ensures that the scene representation can converge accurately to the intended scene geometry. This advancement negates the necessity for explicit mesh exportation from scene representation as commonly required in previous work. To assign the modeled scene with sharp and clear textures, we incorporate the use We show our generated color and depth renderings under two common layout conditions (a bedroom and a living room) alongside three other baselines. SceneCraft demonstrates higher credibility in following the layout conditions and is capable of handling more complex scenarios.of VGG [25] perceptual and stylization loss during the distillation process. This strategy allows the scene representation to produce rendered images that share semantic meaning and stylistic elements with SceneCraft2D-generated images, rather than striving for pixel-perfect replication, which often leads to blurred results. By employing this loss, our SceneCraft framework emerges as a unified model to generate scenes in a sharp and clear manner, thereby eliminating the need for labor-intensive processes of mesh exportation and optimization. 
experiments: 
 In this section, we focus on demonstrating the quality of SceneCraft generation under various layout conditions and prompts, and compare our performance with publicly available methods quantitatively and qualitatively. Then we present more challenging generation that is beyond the scope of the previous methods.Implementation and Datasets. For the development of our SceneCraft2D diffusion model, we finetune Stable Diffusion [50] with our produced layout data. We use multi-view images from ScanNet++ [72] and HyperSim [49] to construct BBI data. In the distillation process, we choose Nerfacto from NeRFStudio [56] as our backbone for scene representation. During distillation, we use a duo-GPU pipeline to parallelize diffusion generation and NeRF training. More details are provided in Appendix Sec. A.BBS Sources. For efficiency and effectiveness, we employ two distinct approaches to leverage bounding-box scenes (BBS), one of which utilizes original 3D bounding boxes (axis-aligned or oriented) by directly rendering them into 2D images. This straightforward method is already sufficient for the generation in our experiment, as we applied on Hypersim [49] data. Another approach enhances traditional bounding boxes by voxelizing them into a more detailed collection of smaller, fine-grained voxels. This method is particularly adept at capturing the nuances of more complex geometries and arrangements within a scene, such as L-shaped tables or S-shaped desks, which often pose challenges for more simplistic modeling techniques. We find that this significantly improves the model's ability to accurately represent and understand the spatial dynamics and intricate designs of various objects within a scene. We use this strategy for realistic and challenging scenes [72] .Baselines. Most existing work does not generate scenes conditioned on user-specified layouts [17, 24, 37] . The only two concurrent scene generation methods that support layout guidance have not released their codebases [16, 53] for comparison. Hence, we make our best effort to create a fair comparison with open-sourced scene generation methods and demonstrate the effectiveness of our method through ablation study (Sec. 4.2). Text2Room [24] uses a text-conditioned inpainting model to construct the scene frame by frame. Following their original instructions, we change the text prompt along the trajectory to reflect which objects are visible in the current frame. Similarly, for MVDiffusion [60] , we construct different prompts for each of the eight views that make up the panorama image. For Set-the-scene [12] , we follow their official guidelines, using 3D modeling software (e.g., Blender) to create the same layout input for training and set the same prompts as SceneCraft.Qualitative Results. In Figure 3 , we demonstrate qualitative generation results of SceneCraft on Hypersim [49] provided room layouts. These illustrations vividly demonstrate the model's proficiency in crafting detailed, complex, and free-form scenes, showcasing its application across both the realistic and synthetic datasets. Not only does it highlight the technical prowess of SceneCraft in navigating the intricacies of scene generation, but also its adaptability to the diverse requirements of real-world and artificially constructed environments. In Sec. 4.3, we demonstrate more challenging generation, which includes extremely challenging cases for panorama-based methods, but naturally supported by our framework. Quantitative Results. We present quantitative comparisons with baseline methods [12, 24, 60] using both 2D and 3D metrics in Tab. 1. For 2D metrics, we compute the CLIP Score (CS) [48] and the Inception Score (IS) [52] , which do not require ground truth scenes from the dataset, and therefore are agnostic to the dataset used in training. We also measure 3D quality by conducting a user study with 32 participants, who scored 3D consistency (3DC) and overall visual quality (VQ) of rooms generated by different methods on a scale of 1 to 5. Our experimental design follows previous work [51] . The quantitative results highlight that our method consistently outperforms prior approaches in terms of the CLIP Score, 3D consistency, and visual quality. Regarding the Inception Score, we anticipate that our diffusion model's finetuning with fixed categories slightly limits generation diversity. However, this is not a major concern for our task, as previous work has struggled to achieve both high consistency and visual quality while being controlled by layout prompts. Additionally, we did not provide other common metrics on generative tasks, e.g., Fréchet Inception Distance (FID) Score, since it is dependent on the ground truth dataset and would result in unfair and inaccurate comparison if applied to our experiments.Comparison with Existing Methods. In Figure 4 , we present our results compared with three baselines under two common layout conditions. SceneCraft significantly outperforms previous methods. For panorama-based methods (MVDiffusion), the biggest limitation lies in the inability to model rooms with complex shapes, such as L-or S-shaped structures. When using prompts to describe layout conditions, MVDiffusion fails to generate the desired results accurately. For inpainting-based methods (Text2Room), although they support free camera trajectories, their iterative Figure 5 : Generation results of SceneCraft in complex scenes. We demonstrate SceneCraft's ability to generate more complex indoor scenes leveraging arbitrary camera trajectories. Such non-regular shape of rooms cannot be naturally achieved by previous work.generation nature often results in repetitive or contradictory frames. In the example shown in Figure 4 , Text2Room generates four beds in that room simply because the prompt contains the word "bedroom," completely failing to adhere to the specified layout conditions. For NeRF-composition methods (Set-the-scene), the main drawback is the inability to generate objects with significant size differences. Set-the-scene trains and combines different objects within the unified NeRF space. In Figure 4 , Set-the-scene fails to generate objects hanging on walls, such as blinds or televisions. Our model, however, addresses all these issues: it can generate scenes of any scale and complexity following the given layout conditions and can also be adjusted via prompts.We conduct various ablation studies to validate our methods. Specifically, we test the effect of the base prompt used in finetuning, the layout-aware depth constraint, and the texture consolidation. The appendix section offers a comprehensive introduction to all of our evaluated models and additional experimental details, and includes further visualization and ablation experiments. We also elaborate on the limitations, failure cases, broader impacts, and future directions of our work. Please refer to Appendix Sec. B.1 for more details. Effect of Base Prompt. To verify the effectiveness of using our base prompt, we test different prompt settings: e.g., generating image captions from BLIP2 [29] with user-defined specific prompts.In Figure 6 , we show that our method successfully achieves the control of the generation style through prompts, while maintaining a good layout-following ability. Considering the failure of the BLIP2 prompt and the complexity of our layout conditions, we believe that the more complex the condition, the more general the prompt we should take. In our case, we use "This is one view of a room." to generally guide the model to fit the entire dataset of the indoor scene, rather than focusing on a particular class or object.Generation on Irregular Shape. In Figure 5 , we showcase the results of more complex scene generation with fully customized layout on the free-camera trajectory. In the first example (Scene A), we customize an indoor layouts input where a bedroom is connected to a living room, along with the corresponding arbitrary camera trajectory. Theoretically, we can generate indoor scenes of any scale, for example, complex indoor room systems composed of multiple interconnected small rooms (Scenes B-D of Figure 5 ). Such tasks are not well-supported by methods based on panorama generation [60] or NeRF composition [12] . Although some other work [24] supports arbitrary camera trajectories, it performs poorly in establishing reasonable scene geometry and controlling the scene content.Style Variants Generation with Fixed Layouts. In Figure 7 , we show three variants of generation with the same room layout and different appearance, simply achieved by using different prompts.The results demonstrate the various control abilities of SceneCraft, allowing us to accurately define the shape and appearance of generation. 
conclusion: 
 This work has introduced SceneCraft, an innovative method for generating complex and detailed indoor scenes from textual descriptions and spatial layouts. By leveraging a rendering-based operation, and a layout-conditioned diffusion model, our work effectively converts 3D semantic layouts into multi-view 2D images and learns a final scene representation that is not only consistent and realistic but also adheres closely to user specifications. Experimental results show the superiority of our model over existing state-of-the-art methods, highlighting its ability to generate diverse textures and maintain geometric consistency across complex indoor scenes. Effect of Layout-Aware Depth Constraint. We conduct experiments to validate the effectiveness of the layoutaware depth constraint. As depicted in Figure B , without this strategy, the model is completely unable to learn the correct geometry of the scene from 2D guidance. Although a reasonable appearance is achieved, it is mainly due to the fully flexible camera trajectories and complex layout conditions. Notably, using fixed camera poses as in panorama generation can partially alleviate this issue. With the layout-aware depth constraint, the scene geometry rapidly converges to the ground truth during the very initial training stage. Although some area may find incorrect positions (red box), they are corrected during subsequent training after intermediately disabling this strategy, eventually leading to final convergence (green box). Once the geometry converges well, the color and texture begin to emerge. 
experimental setting/details: 
 Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes]Justification: We introduce all the training and evaluation details necessary to understand the results.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [No] Justification: Due to the large amount of experiments required to be run in this paper, we do not have enough computational resource and time to generate error bars for all our experiments.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: We have discussed both the potential positive and negative societal impacts.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)? Answer: [NA] Justification: The paper does not involve crowdsourcing nor research with human subjects. Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: In Figure D, we take the complex room layouts from the ScanNet++ [72] dataset as an example. The first two rows represent the input layout condition, and the last row shows the SceneCraft2D output. Note that for rooms with irregular object geometries, we convert voxelized 3D bounding boxes into more fine-grained voxels. This method particularly captures layouts with irregular geometries, such as L-shaped tables or S-shaped desks, which pose challenges to more simplistic modeling techniques.The results show that SceneCraft2D performs well in such complex room layouts.Although we have achieved promising results in complex scene generation from user prompts, 3D generation remains a very challenging task with many unsolved problems, and our method is limited in some aspects. This section provides a detailed discussion of the limitations and outlines potential future directions.Discussion on Failure Cases. Despite our promising performance, some failure cases also exist:(1) Extremely Complicated Scenes. Our method may struggle to reason the layouts and generate rooms when layouts are excessively complex, containing many closely placed objects or highly overlapped bounding boxes (see Figure E ). In this case, our optimized method using voxelization does not provide clear and accurate representation of object layouts, which also reflects the difference between indoor and outdoor (street-view) scenes;(2) Mismatched Layout and Prompt Inputs. When the prompt does not align with the actual room layouts (e.g., a bedroom layout with a "kitchen" prompt), our method may fail to generate appropriate room contents or achieve good convergence (see Figure F ). Hence, users may need to adjust the corresponding prompts when generating a large complex scene with a long-term trajectory. All of these failure cases reflect the limitations of our method.Quality of Images Requires Further Improvement. The quality of our generated 3D scenes still requires further improvement. When dealing with more challenging objects that typically have irregular geometries, such as hollowed-out chairs, lamps, or blinds, our results tend to be somewhat blurry. Moreover, complex layout condition inputs still limit the control ability of the prompt. For instance, we are unable to generate objects as vivid and richly detailed as those produced by the original diffusion model.Extension to Generation of Outdoor Scenes. Most existing literature separates the task of generating indoor and outdoor scenes and focuses on one of the scenarios in their work. The generation of indoor and outdoor scenarios features different challenges, with indoor scenes having denser and more complex layouts, and outdoor scenes having more dynamic objects and a larger space to cover. Although it is a valid choice to start from indoor scene generation because of its relatively smaller scale and complex layouts, it is not comprehensive. The generation of outdoor scenes will likely lead to unique challenges and insights, and we consider this a direct future direction.Other Future Directions. There are also challenges in defining fair, accurate, and comprehensive metrics for evaluating 3D scene generation methods in all aspects. The development of such metrics remains an open challenge and serves as a potential direction for exploration in the generative AI community. Another promising direction is flexible and controllable scene editing using decomposed 3D representations, given that our layout input is freely defined and adjustable. For complex scene generation, creating layouts by hand could be time-consuming and laborious, hence integrating some methods of automatic scene layouts and camera trajectories creation will also be a future topic, such as LLM (large language model)-based [77] and transformer-based approaches [16] . Extending our framework to incorporate user feedback loops for iterative refinement of generated scenes offers another promising direction.We anticipate a potential positive social impact from our work. Being able to generate high-quality 3D digital content based on user queries, SceneCraft has the potential to revolutionize various industries, from VR/AR to architectural design and gaming. By significantly reducing the time and expertise required to create detailed 3D scenes, our work also leads to more accessible and fair content creation.Potential Negative Societal Impact. We do not see a direct negative societal impact of our work. Indirect potential negative impact involves misusing scene generation models for digital content creation. We believe that it is crucial for researchers to proactively consider these concerns and establish guidelines to ensure responsible usage of these models. In this section, we list the licenses of all the datasets we have used during our evaluation:• ScanNet++ [72] : The ScanNet++ data are released under the ScanNet++ Terms of Use3 .• HyperSim [49] : Creative Commons Attribution-ShareAlike 3.0 Unported License.In addition, we utilize a number of public foundation model checkpoints pretrained on various data sources in our paper. Please refer to their original papers [39, 50] for the license of datasets they have used in pretraining their models.• The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. , with an open-source dataset or instructions for how to construct the dataset). (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.Layout Scannet++: https://huggingface.co/datasets/gzzyyxy/layout_diffusion_scannetpp_voxel0.2Layout Hypersim: https://huggingface.co/datasets/gzzyyxy/layout_diffusion_hypersimhttps://kaldir.vc.in.tum.de/scannetpp/static/scannetpp-terms-of-use.pdf 


Decision: This paper introduces an interesting approach to generating scenes given the 3D layout. The reviewers questioned technical novelty, evaluation of the 3D consistency, more visualization, training details, and limitation analysis during the review phase. The authors provided solid feedback to relieve the concerns, and most of the reviewers mentioned that the rebuttal helps to understand the methods better, and the provided additional results are acceptable. After the rebuttal phase, reviewer DRMD raised the score to 'weak accept', reviewer Nerf keep the 'weak accept' score, reviewer Gzhh increased the score to weak accept, and reviewer DHDM keep the score 'borderline accept'. As a result, no reviewers opposed the paper's rejection. Although there is a concern about the similarity of the concurrent work published in ArXiv, AC confirmed that that issue could not affect paper judgment, given the NeurIPS paper submission policy. AC also confirms that the paper has several merits worth presenting in the field.

Reviews: review: paper_summary: paper_summary: This paper addresses the task of 3D-consistent indoor scene generation using 2D diffusion models. As input the user provides a 3D bounding-box layout of the scene and a text prompt coarsely describing the scene. The proposed method contains a two-stage training phase. In the first stage, a 2D diffusion model is fine-tuned to synthesize a rendered view of a scene based on its rendered semantic layout and a depth map from the same view. In the second stage, the fine-tuned diffusion model is distilled into a NeRF representation with an SDS-based approach. The authors qualitative evaluate their method and compare with existing baselines, showcasing the effectiveness of their method.
strength_weakness: strength_weakness: - The proposed method might not be very novel. The contribution of the paper seems mainly to be fine-tuning the diffusion model for semantic-guided image synthesis. The distillation stage is mostly similar to the existing methods.

- The authors only evaluate their method qualitatively. Although the provided results seem very promising in terms of the 3D consistency of the scenes, a quantitate evaluation could provide more certainty about the advantage of the proposed method.

- The authors could have provided video visualizations in the supplementary, as assessing the 3D consistency of the method from multi-view images is challenging
questions: questions: See the weaknesses
limitations: limitations: Adequately discussed.
paper_summary: paper_summary: DreamScene proposes a method for 3D indoor scene generation with text and layout as input. For this, they propose using a 3D bounding box scene representation as the means to provide layout guidance. This is then used as an input, along with text prompt, to a 2D diffusion model, dreamscene2D, which is capable of generating high-fidelity and high-quality views of rooms following the rendered “bounding-box image” (containing 2D semantic maps and depth) from the bounding box representation. Dreamscene2D is fine-tuned from stable diffusion using LoRA adapters. This is then used for training a NeRF using Score Distillation Sampling. Additional measures, such as a depth constraint, floc removal and texture consolidation with backbone models helps generate the desired scene. In comparison with presented baselines, the results show the ability to model complex shapes in rooms, greater consistency across views, as well as the ability to generate objects of varying sizes. Ablation experiments are provided for the effect of the base prompt, the effect of the depth constraint and the effect of the texture consolidation using backbone models.
strength_weakness: strength_weakness: - Lack of quantitative metrics: The paper claims superiority over baselines both qualitatively and quantitatively (L55). However, I am unable to find any quantitative metrics in the paper.
- Generation quality: While the proposed method seems to perform better than the chosen baselines, the quality of the generated scenes seems to be relatively lower overall. For example, in the last row of Figure 4, the scene lacks texture and shows a degeneracy in color generation.
- Lack of failure cases: In such works as scene generation, and especially since the scope of this work is limited to indoor scenes, I believe it is necessary to contextualize the kind of scenes where the proposed method is successful and where it fails.
questions: questions: - Runtime: I believe the time taken for 3D asset generation through such text-guided approaches is an essential metric for comparison. However I am unable to find this quantification in this work. 
- Quantitative metrics: I think it is important to quantify performance of the model (using maybe something like a CLIP score or FID), especially since this is mentioned in the introduction of the paper.
- It would be good to get an understanding of failure cases of the proposed method, or settings where the proposed method is unable to generate a favorable output.

For now, I would like to give a weak accept score and wait for the authors' responses to the mentioned questions and weaknesses.
limitations: limitations: The authors have addressed limitations and future directions in the appendix which is appreciated. I have some questions with regards to failure cases that I have addressed in previous sections.
paper_summary: paper_summary: This work tackles the problem of 3D scene generation from bounding box scene, text prompt and camera trajectory. The core contribution of the work stems from a 3D scene generation method where the method is able to produce a complex scene, which the previous works could not generate due to using a panoramic representation. The method generates 2D images from multiview images using a diffusion model and then generates 3D. The results show that method is able to generate realistic 3D scenes from the user input.
strength_weakness: strength_weakness: - The scenes look more synthetic compared to the baselines such as MVDiffusion and the text2room.
- More qualitative results are required to be convincing. What fascinates me about this work is that the method is able to produce a complex layout scene. However, there is only a single example in the appendix, which the panoramic scene methods cannot generate. The authors should provide more qualitative examples to show that the method can indeed generate complex scenes. Also, the authors claimed that the method can generate various scenes, but only few of the visual examples were provided. 
- In line 55, the authors mentioned that the method achieves SOTA performance quantitatively. Could the authors provide the quantitative results to back the claim? Also, I would like to see what the “qualitative” here refers to.
- In Line 66, the authors mentioned “our DreamScene achieves high-quality generation in various fine-grained and complicated indoor scenes, which are never supported by previous work”. To me this sounds a little bit confusing. Previous works such as ControlRoom3D were able to generate various high-quality scenes, yet they were not able to generate complicated indoor scenes. I think the authors should clearly mention that the completed indoor scenes is the key difference between the previous works.
questions: questions: - I’m not fully understanding why camera trajectory is required to generate a 3D scene. If the users no the bounding box, couldn’t the method just use random camera trajectory to make the method work?
- In line 134, the authors mentioned “.glet@tokeneonedot”, what does this mean?
- What is the approximate run time of the method? Since the method generates 2D images and generate 3D, it maybe slow.
limitations: limitations: The method is limited to generation of a small scene. It would be nice to provide results on a large outdoor scene, which does not anyway diminish the work's contribution.
paper_summary: paper_summary: This paper proposes a layout-guided method for room generation. The proposed method uses 3D scene layout as a prior and leverages semantic and depth maps as 2D conditions. Specifically, the authors start by training a 2D conditional diffusion model named DreamScene2D, which utilizes ControlNet to incorporate projected semantic and depth maps and conditions. Then the 3D scene can be distillated from the conditional model. The proposed method achieves good rendering quality in complex room scenes.
strength_weakness: strength_weakness: 1. The key insight of the proposed method is similar to recently released paper UrbanArchitect [1], which also uses layout as prior to generate urban scenes (more complex and larger scale). DreamScene uses a pipeline based on InstructNerf2Nerf and UrbanArchitect directly uses a layout-guided SDS. Although the two works can be seen as concurrent works, I hope the authors can include a discussion.
2. As a 3D generation model, it would be better to provide video results for better illustration. The authors can also provide 3D results for better visualization (e.g., generated 3D mesh like text2room).
3. Quantitative results are missing. The authors should at least perform User Study regarding the visual quality and the 3D consistency. Besides, since the ControlNet is trained on specific datasets, metrics like FID can also be used for better evaluation.
4. Some training details are missing, e.g., requirements for memory and time.
5. Since the layout prior is used, it would be desired to use decomposed 3D representations for further editing. 
6. It would be better to include a method for automatic scene layout creation. For example, Ctrl-Room utilizes a scene code diffusion mode to generate scene layouts [2]. LLM can also be used to generate scene layouts [3].

[1] Lu, Fan, et al. "Urban Architect: Steerable 3D Urban Scene Generation with Layout Prior." arXiv preprint arXiv:2404.06780 (2024).

[2] Fang, Chuan, et al. "Ctrl-room: Controllable text-to-3d room meshes generation with layout constraints." arXiv preprint arXiv:2310.03602 (2023).

[3] Hu, Ziniu, et al. "SceneCraft: An LLM Agent for Synthesizing 3D Scenes as Blender Code." Forty-first International Conference on Machine Learning. 2024.
questions: questions: 1. 3DGS has been widely used in 3D generation models due to its high speed and good quality. Have you tried to use 3DGS to replace NeRF?
2. Is there a limit to the complexity of the scenarios a method can handle?
3. Text2Room uses SD for missing parts inpainting. Have you tried to replace SD model in Text2room with your finetuned SD model for inpainting for controllability?
limitations: limitations: Please refer to weaknesses.

Comments: title: Reply to reviewer Gzhh’s feedback
Thank reviewer Nerf for your positive feedback!
Post-Rebuttal Comment
Thank reviewer DRMD for your positive feedback!
Post-Rebuttal Comment
Response to rebuttal
Reply to Reviewer Gzhh's feedback at General Response
Could the authors provide more qualititative results if possible?
Clarification required for the quantiative results
Video Link
comment: We appreciate the reviewer for checking our rebuttal. 

1. **For SDS-based pipeline**, we believe that you are referring to the comparison with recent work UrbanArchitect. 
   1. We would like to point out that UrbanArchitect is an arXiv manuscript released just before the NeurIPS submission deadline, which, as the reviewer also noted, should be considered as a concurrent work of ours. Not comparing with this method should not be regarded as a weakness of ours, especially since this method focuses on a different task than ours (outdoor scene v.s. indoor scene).  
   2. Due to the limited time in the rebuttal phase, and the substantial difference between the outdoor and indoor scene generation settings (as we have discussed in Q1 of the rebuttal), we are unable to finish training of UrbanArchitect with any indoor dataset to use it for indoor scene generation.  
   3. As mentioned in the previous rebuttal, we will cite and discuss UrbanArchitect in the revision.  
2. For **layout-enhanced Text2Room**:  
   1. We acknowledge that this can be seen as an interesting exploration. However, we believe this is weaker than the baselines used in our submission, as Text2Room relies on a depth-based naive 3D reconstruction approach.   
      1. This fundamental aspect of its architecture would likely still result in challenges related to inappropriate or incomplete geometry, regardless of the diffusion model used for inpainting.   
      2. Simply replacing the diffusion model of Text2Room to our DreamScene2D could only enable the control of the appearance of generated images, but cannot at all help with the geometry issues resulting from the naive reconstruction, as such reconstruction is still only based on a weak monocular depth estimator.  
      3. Hence, we anticipate that such a model is unlikely to achieve the same level of quality as our current method. Importantly, lacking this ablation study does not demolish the contribution of our work, which was already demonstrated by our state-of-the-art results.  
   2. For the reviewer’s reference, we would also like to point out that previous published work Ctrl-Room [16] and ControlRoom3D [48], while performing layout-guided scene generation, do not include this additional experiment.  
   3. Per the reviewer’s request, we have already implemented this baseline and the model is training. However, due to the limited time in the rebuttal phase, we are not able to get the final results before the end of the discussion phase. The current preliminary results indicate that this method is significantly worse than ours. We will add this experiment in the final revision.

We hope that our clarifications and additional results will address the reviewer’s remaining concerns. We look forward to a potentially more favorable evaluation of our work.
We appreciate the reviewer's feedback on the need for more complex scene examples. 
- During the limited rebuttal period, we have tried our best to generate two new complex scenes, as shown in this anonymous link: https://github.com/DreamScene3D/DreamScene/blob/main/rebuttal-visualization.jpg?raw=true. The last two rows showcase these new results. 
- We would also like to point out that “complex scenes” are not just limited to non-square shaped ones – square scenes can also be complex so that the baselines will fail. In our Fig.3, the BBS-A/B/C contains many objects that are not adjacent to the walls and corners, which will also create occlusions in panoramas and make the depth map non-continuous and hard to predict. Therefore, these scenes should also be regarded as “complex scenes” where our DreamScene is the only work that can work well. 

We hope these results could establish our DreamScene’s ability to generate rooms based on various complicated layouts, and address your concern about this. We commit to including these new examples, along with additional complex scene generations, in our future revision.
We appreciate the reviewer for the positive feedback. Your constructive comments and suggestions are indeed helpful for improving the paper.

We will continue to improve our work and release the code. If the reviewer has any follow-up questions, we are happy to discuss!
I thank the reviewers for their careful response to the questions that were raised. I appreciate the presented quantitative results, the discussion on limitations and training details. Given the scope and visual quality of the results, and taking into account the author responses, I would like to retain my score of weak accept.
We appreciate the reviewer for the positive feedback. Your constructive comments and suggestions are very helpful for improving the paper. Also, many thanks for raising the score.

We will continue to improve our work and release the code. If the reviewer has any follow-up questions, we are happy to discuss!
I thank the authors for their answers, in particular for providing the quantitative evaluation and video visualisations. I raise my score to Weak Accept
The authors' rebuttal has addressed some of my concerns (training details, quantitative results, video results, etc.). Several experimental results are still missing (comparison with SDS-based pipeline, layout-enhanced Text2Room). Considering the weaknesses discussed above, I keep my score at Borderline Accept.
We sincerely appreciate reviewer *Gzhh* for checking our rebuttal. We address this remaining concern as below.

***
1. **Different training datasets used for quantitative evaluation.**
    - Our DreamScene2D is designed to generate high-quality 3D scenes that match the input layout. To achieve this, we require a dataset with **high-quality 3D bounding box annotations** with accurate category labels for training.
        - We therefore chose to train our DreamScene2D using the Hypersim [45] dataset. This choice of the training dataset is also *consistent* with ControlRoom3D [48].
        - Other baseline methods use ScanNet as their training dataset. However, the bounding box annotations in ScanNet are of low quality (noisy and inaccurate), and cannot satisfy our requirements.
        - While we understand the reviewer’s concern that using different training datasets might influence specific quantitative metrics, re-training the baseline models using the Hypersim dataset we employ is challenging and time-consuming. Additionally, we think that dataset selection is a critical design choice in developing these methods and should be considered during evaluation. For example, as mentioned above, we chose Hypersim for DreamScene2D specifically because of its high-quality 3D bounding box annotations. Therefore, using the exact same training data for comparison in our context may not be practical or necessary.
        - In fact, comparing methods trained on different datasets is a common practice in generative model research. For example, ControlRoom3D [48] also compares with methods (MVDiffusion and Text2Room) trained on different datasets. Similarly, the well-known 2D diffusion model Instruct-Pix2Pix is compared with T2L and SDEdit, which are trained on other datasets.
    - Importantly, our quantitative experiments focus on the CS, IS, and user study metrics, which evaluate the final generation results using either pre-trained models (CLIP and Inception) or participants’ judgments. These metrics do *not* require ground-truth scenes from the dataset and, therefore, are *agnostic* to the dataset.
    - Therefore, even when using a different dataset for training, our quantitative comparison remains fair and reasonable with the current metrics and adheres to established practices.

2. **Not using FID metrics.**
    - First, both our method and [48] focus on the overall quality of generated scenes and layout-following capability, unlike MVDiffusion which focuses on the quality of the diffusion image generation. Therefore, our method and [48] choose to report current metrics rather than FID.
    - On the other hand, the value of the FID metric is dependent on the dataset. Since we use different datasets for diffusion finetuning, applying FID would result in unfair and inaccurate comparison.

***
If the reviewer has any follow-up questions, we are happy to discuss them.
Thank you for the fruitful response. I really appreciate the author's effort for the response.

I know this is a last minute response (and I am sorry for this), but could the authors provide more qualitative results on complex scenes? I ask this because the main argument of the proposed method is that the method can generate complex scenes, however, the only complex scenes in the manuscript and the rebuttal is the 2 scenes in the rebuttal (the scene in the appendix has the same L shaped room as in the rebuttal, otherwise the scenes look like a bbox to me). It's not very convincing to believe that the proposed method can generate complex scenes with just 2 (complex scene) visualizations. I reallize that the authors have only one more day for the rebuttal, but could the authors provide more examples with complex scenes?
I appreciate the authors for the dedicated response. I have some questions regarding the provided quantiative metrics. 
Is the quantiative metrics presented here during the rebuttal with same datasets? In the rebuttal to the comments that I have wrote, the authors mentioned that 

> Our diffusion model is primarily fine-tuned on the synthetic Hypersim [45] dataset, chosen for its high-quality layout data. This choice aligns with other recent works like ControlRoom3D [48].
    In contrast, MVDiffusion uses a version of Stable Diffusion fine-tuned on the real-world ScanNet dataset, while Text2Room uses the original Stable Diffusion model.

which means that models to obtain the quantiative metrics might have been trained with different datasets, as far as I know. 
Could the authors clarify this? Because, the dataset matters and might be critical to the obtained quantitative results. Also, what were the choices for not using FID as an evaluation metric?
Dear Reviewer,

This is a gentle reminder to please review the rebuttal provided by the authors. Your feedback is crucial to the decision-making process. Please consider updating your score after reading the rebuttal.

Thank you for your help with the NeurIPS!

Best, Your AC
Dear Reviewer,

This is a gentle reminder to please review the rebuttal provided by the authors. Your feedback is crucial to the decision-making process. Please consider updating your score after reading the rebuttal.

Thank you for your help with the NeurIPS!

Best, Your AC
Dear Reviewer,

This is a gentle reminder to please review the rebuttal provided by the authors. Your feedback is crucial to the decision-making process. Please consider updating your score after reading the rebuttal.

Thank you for your help with the NeurIPS!

Best, Your AC
Dear Reviewer,

This is a gentle reminder to please review the rebuttal provided by the authors. Your feedback is crucial to the decision-making process. Please consider updating your score after reading the rebuttal.

Thank you for your help with the NeurIPS!

Best, Your AC
Dear Reviewers, here is the **link to the anonymous video with our generated scenes**. https://github.com/DreamScene3D/DreamScene/raw/main/rebuttal_video.mp4