LiteReality: Graphics-Ready 3D Scene Reconstruction from RGB-D Scans
Authors: Zhening Huang
Xiaoyang Wu
Fangcheng Zhong
Hengshuang Zhao
Matthias Nießner
Joan Lasenby
Venue: NeurIPS 2025 poster ()
Keywords: 3D Reconstruction; Appearance Modeling; Graphics-Ready Reconstruction
Primary area: applications
TL;DR: LiteReality is an automatic pipeline that converts RGB-D scans of indoor environments into graphics-ready 3D scenes, featuring high-quality meshes with realistic PBR materials and articulated objects.

Abstract: We propose LiteReality, a novel pipeline that converts RGB-D scans of indoor environments into compact, realistic, and interactive 3D virtual replicas. LiteReality not only reconstructs scenes that visually resemble reality but also supports key features essential for graphics pipelines—such as object individuality, articulation, high-quality physically based rendering materials. At its core, LiteReality first
performs scene understanding and parses the results into a coherent 3D layout and objects, with the help of a structured scene graph. It then reconstructs the scene by retrieving the most visually similar 3D artist-crafted models from a curated asset database. Later, the Material Painting module enhances the realism of retrieved objects by recovering high-quality, spatially varying materials. Finally,
the reconstructed scene is integrated into a simulation engine with basic physical properties applied to enable interactive behavior. The resulting scenes are compact, editable, and fully compatible with standard graphics pipelines, making them suitable for applications in AR/VR, gaming, robotics, and digital twins. In addition, LiteReality introduces a training-free object retrieval module that achieves state-of-the-art similarity performance, as benchmarked on the Scan2CAD dataset, along with a robust Material Painting module capable of transferring appearances from images of any style to 3D assets—even in the presence of severe misalignment, occlusion, and poor lighting. We demonstrate the effectiveness of LiteReality on both real-life scans and public datasets.

LiteReality: Graphics-Ready 3D Scene Reconstruction from RGB-D Scans
Zhening Huang' Hengshuang Zhao?
Xiaoyang Wu² Matthias NieBner
Fangcheng Zhong'
Joan Lasenby'
'University of Cambridge
The University of Hong Kong https: Attps://1tterealicc github... 1o
Technical University of Munich
Imput: RGB-D Sean
LiteReality
Output: Graphics-Ready 3D Scene
Inter tebbl
Near Phae-PPrtoccaae caltoraalstie
Appetrance
Micrias for Relighning
Phasice- Bosed Interaation.
Figure 1: From RGB-D Scan to Craphics-Readh 3D Scene. LiteReality reconstructs compact, realistic 3D environments from real-world RGB-D scans, featuring near near-photoreealstic appearance, articulated geometry, and physically based rendering (PBR) materials providdnng assets that can be easily integrated into simulation or rendering pipeline.
Abstract
We propose LiteReality, novel pipeline that converts RGB-D scans of indoor environments into compact, realistic, and interactive 3D virtual replicas. LiteReality not only reconstructs scenes that visually resemble reality but also supports key features essential for graphics pipelines -ucch as object individuality, articulation, high-quality physically based rendering materials. At its core, LiteReality first performs scene understanding and parses the results into a coherent 3D layout and objects, with the help of structured scene graph. It then reconstructs the scene by retrieving the most visually similar 3D artist-crafted models from a curated asset database. Later, the Material Painting module enhances the realism retrieved objects by recovering high-quality, spatially varying materials. Finally, the reconstructed scene is integrated into simulation engine with basic physical properties applied to enable interactive behavior. The resulting scenes are compact, editable, and fully compatible with standard graphics pipelines, making them suitable for applications in AR/VR, gaming, robotics, and digital twins. In addition, LiteReality introduces training- ffree object retrieval module that achieves state-of- the-art similarity performance, as benchmarked on the Scan2CAD dataset, along with robust Material Painting module capable of transferring appearances from images of any style to 3D assets- even in the presence of severe misalignment, occlusion, and poor lighting. We demonstrate the effectiveness of LiteReality on both real-life scans and public datasets.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Introduction
Creating digital replicas of real-world environments remains a central challenge in machine perception, computer vision, and graphics. Although differentiable rendering techniques have achieved remark- able photorealism in 3D reconstruction [41, 22]. their outputs often resemble "3D photographs". they capture geometry and texture but lack true interactivity and structure. A truly effective digital replica must go beyond visual fidelity to offer semantically rich, simulation-ready. interactive envi- ronments. Such representations empower users to interact with, manipulate, and simulate complex scenes capabilitts essential for robotics, virtual and augmented reality, and embodied AL.
(SIM) [49, 56], Gaussian Splatting [22], and Neural Radiance Fields (NeRF) [41, 7]-we introduce the concept of graphics-ready reconstruction. This form of reconstruction goes beyond visual fidelity by incorporating additional structural and functional information to ensure compatibility with graphics pipelines and support downstream tasks such as interactive simulation. To achieve this, several key criteria must be satisfied. First, the scene should be object-centric, treating each object as an individual entity to enable physically plausible placement and manipulation. Second, object functionality must be modeled, allowing for interactive behaviors such as opening doors, drawers, or appliances. Third, the scene should use PBR materials to ensure photorealistic appearance under varying lighting conditions. Lastly, the integration of physical properties including mass, gravity, and collision dynamics-is essential to support realistic interactions grounded in real-world physics. By meeting these requirements, we move beyond passive, geometry-focure reconstructions toward truly graphics-ready environments that combine visual realism with functional structure.
Compared to traditional 3D reconstruction techniques- as
Several research efforts have addressed different aspects of this broader challenge. First, in the simulation domain, recent work study how to create interactive virtual environments from text descriptions [61], room layouts [11], large-scale procedural generation pipelines [12], and even from single images [9]. However, these approaches often prioritize diversity over realism, aiming to generate training data for embodied Al rather than faithfully reconstructing digital replicas of real-world scenes. Other methods pursue realism through compact, abstract representations- instance, using CAD retrieval and alignment to represent scenes [21, 17, 20, 26, 25] or estimating object textures and materials from images [63, 59, 36]. While promising. these approaches often overlook the importance of integrating components into a coherent, functioning system and tend to neglect the difficulties posed by real- world indoor scans scans-such as clutter, severe occlusions, and poor lighting. As a result, existing systems frequently struggle to produce faithful reconstructions from in-the-wild scans, limiting their applicability beyond curated or controlled environments.
To this end, we introduce LiteReality, a method that automatically transforms RGB-D scans into realistic, interactive indoor environments (see Figure 1). LiteReality is designed to handle the diverse challenges of real-life scans. At its core, LiteReality consists of four main stages: Scene Perception and Parsing: Objects and room layouts are extracted from RGB-D scans using off-the-shelff methods [53, 28] and represented a compact scene graph. This graph enforces spatial constraints and refines noisy input into a structured, physically plausible layout. 2. Object Reconstruction: Artist-designed 3D models that best match real-world objects are retrieved from a curated asset database. Though not perfectly aligned geometrically, these models provide high- figh-fidelity geometry and support interactive functionality, including articulated components. 3. Material Painting: A robust material estimation and optimization framework assigns high-quality PBR materials to the retrieved 3D models, based on multi-view RGB images. 4. Procedural Reconstruction: The final scene is assembled by constructing room structures, assembling doors and chairs, placing objects, and assigning basic physical properties and interaction reeiicing in a lightweight, graphics-reat environment. We evaluate LiteReality on the ScanNet dataset [8] and on real-world indoor scans captured with an iPhone. Across diverse environments, our method consistently converts noisy, incomplete scans into compact, and realistic digital replicas. Our main contributions are as follows:
L ReaeReality framework: To the best of our knowledge, LiteReality is the first system capable of reconstructing room-level RGB-D scans into compact and realistic CAD representations that feature high-quality PBR materials, providing a strong foundation for downstream applications such as rendering, simulation, and virtual interaction. The system generalizes well across a wide range of real-world scenes.
Training-free retrieval pipeline: LiteReality includes a training-free object retrieval system that achieves ofte-oofthheart (SOTA) similarity performance on the Scan2CAD benchmark.
2

---

Input
Ansticaley Daagttod
Pornel SD stadeeiig
Semaniic Filcring
Altedoottly Uprimization
Ozger Recosataciio
Imsge Fealuzs
Sinddatonn Babilig
Panal
Material Rebieval
Layour & 3D Bbox Detection
Poscskwanc
Knder Campan
Languagstescal Masurial
Oulput
Scone Graph Componnng
Scare Puninng
Gamatial Seletion
Ban Detation
Heiraved 3tl otgacia
Cupped ineges
Dutihese
Lioghrcelleat Soote
(3) Scone Farsing Stage (Sec 2.13
Ofject Rooentracion Stage (Sex 3.2) Matriall Painting Stage (Sex 3.3)
(di Phocadaral Reorsstreeten
Figure 2: Pipeline of LiteReality. Given imput RGB-D scans, the process begins with scene percep- tion and parsing, where room layouts and 3D object bounding boxes are detected and organized into a structured, physically plausible arrangement using a scene graph. In the object reconstruction stage, identical clustering first identifies repeated objects, followed by hierarchical retrieval procedure that matches 3D models from the LiteReality database. The material painting stage retrieves and optimizes PBR materials by referencing the observed images. Finally, the procedural reconstruction stage assembles all components into a graphics-reade environment featuring realistic appearance and seamless integration with standard graphics pipelines.
Robust material painting: We introduce novel material-painting approach that reliably transfers high-fidelity PBR materials onto 3D models-ewe in the presence of shape misalignment, challenging lighting, and diverse image styles. Quantitative evaluation shows SOTA performance.
2 Related Work
Graphics-Reade Environments. While industries such as gaming, AR/VR, film production, and physical simulation rely heavily on traditional computer graphics pipelines to create 3D environments, most state-of-the-art 3D reconstruction methods [41, 49, 22] are not readily compatible with these pipelines. This incompatibility poses a barrier to their practical adoption in real-world applications. In the research domain, simulation environments designed for embodied Al training are the closest counterparts to graphics-ready environments, as they emphasize both interactivity and realism. Sig- nificant efforts have been made develop such simulation platforms. A12-THOR [24], RoboTHOR [10]. OmniGibson [30], Habitat 3.0 [48], ThreeDWorld [15], and VirtualHome [47] are notable examples. Despite their capabilities, these platforms often rely on manually crafted scenes, which limits scalability. Several works have proposed scalable scene generation methods. ProcTHOR [12] extends AI2-THOR with procedurally generated environments: Holodeck [61] creates scenes from natural language inputs; Phone2Proc [1]]. generates room layouts by placing semantically similar objects; and Digital Cousin [9] produces diverse simulated environments from single images. More recent contributions in this include Scenethesis [35]. PAT3D [33] and VIGA [64]. While these approaches improve scalability and diversity, they primarily focus on synthesizing virtual scenes for robotic training rather than faithfully reconstructing real-world environments. This highlights key gap: building graphics-ready scenes that combine realistic appearance with physical interactivity. A concurrent work, Metascenes [65], is also working in similar direction by proposing a large-scale dataset that contains large number of simulatable scenes constructed from richly annotated ScanNet scans for embodied Al benchmarks. LiteReality, by contrast, is presented general-purpost tool for converting real-life scans to graphics-ready scenes.
Object-Centric Reconstructions. Joint 3D scene understanding and reconstruction [43, 19, 37, 32] essential for building object-centric scene representations. Recent advances in 3D scene under- standing parttcclaalllllyyy in oriented bounding box detection [28, 52] and room layout estimation [5, have made this direction increasingly promising. An emerging line of work focuses on reconstructing object-centric 3D scenes from single RGB image [62, 59, 9]. To model individual objects, some methods perform mesh reconstruction from multi-view RGB images [43, 37], while others leverage 3D generative models [62]. Despite rapid and ongoing progress in these domains,

---

current approaches often suffer from instability or produce low- low-quality results, limiting their applica- bility in high-fidelity, simulation- ready applications. An alternative to retrieve artist-created CAD models, enabling clean, lightweight, and visually coherent scene reconstruction- commonly used in simulation environments. Also, these models often include articulated variants, making them valuable for interactive and setups, though sometimes at the expense of realism. A key direction in this area is CAD model retrieval and alignment for scene reconstruction, as demonstrated by Scan2CAD [3] and many follow-up works [21, 17, 20, 26, 25, 31, 39, 54, 38, 27, 5, 42, 4, 18, 1J. Note that these methods primarily emphasize alignment accuracy post-retrieval, often assuming that retrieving any object from the correct category is sufficient. While some works aim to improve retrieval similarity [1. 2, 55], they typically require accurate instance segmentation challenging prerequisite in real-world scan data. In contrast, our retrieval method is explicitly designed to address the challenges posed by real-world scans.
Materiel Estimation. Physically-based rendering (PBR) materials are crucial for achieving photo- realism in advanced applications. Estimating PBR materials typically requires recovering spatially varying bidirectional reflectance distribution function (SVBRDF) parameters from RGB images-an an inherently ill ill-posed problem due to the complex interaction of lighting, geometry, and reflectance. A common approach involves retrieval followed by optimization, where materials are initially retrieved based on visual similarity and then refined via differentiable rendering [50, 29]. For instance, PSDR- Room [59] and PhotoScene [63] optimize procedural PBR parameters from image observations using differentiable rendering. Similarly, MAPA [66] carries out part-level segmentation and style transfer, while Material Palette [36] uses a learning- based framework to estimate plausible materials from segmented object views. Although effective in clearly visible and well-cropped scenarios, these methods struggle in cluttered, room-scale environments. Estimation quality often degrades due to occlusion, inaccurate segmentation, low-resolution inputs, and complex lighting conditions. Additionally, per-object differentiable rendering makes these pipelines computationally expensive, limiting their scalability to scenes with many objects. In LiteReality, we introduce self-containe and robust method tailored for room-leve material estimation in messy, real-world scans. Our method generalizes well under occlusion and partial visibility, and remains reliable across diverse lighting conditions.
3 Methodology
LiteReality is a framework that automatically converts indoor RGB-D scans into Graphics-ready scenes. begins by performing scene perception extract the key information of layout and objects. The scene is then procedurally constructed in a physics engine to enable interactivity and simulation capabilities. At its core, the pipeline consists of four key stages: scene perception and parsing, object reconstruction, material painting, and procedural reconstruction, as shown in Figure 2. These stages work together automatically assemble a complete, graphics-ready environment. We highlight the key features and design of each stage in the following sections.
3.1 Scene Perception and Parsing
Scene Understanding. LiteReality takes RGB-D images of room as input. In the scene perception and parsing stage, it extracts key spatial information necessary for producing graphics-ready recon- structions. Among various 3D scene understanding approaches, identified two key nass--room layout estimation and oriented bounding box detection-as as particularly effective due to their ro- bustness and scalability for generating structured 3D environments. Specifically, for real-life scans, we used Apple's RoomPlan [53] with an iPhone to capture layouts and oriented bounding boxes (O-Bbox) interactively. There are also many open-source alternatives for layout and O-Bbox detec- tion [40, 6, 52] that can be used. Raw detection results often exhibit noise, such as collisions and misalignments, which can result in issues like overlapping objects, floating items, and discontinuous walls when constructing 3D world. These problems break physical plausibility and break realism. To address this, we propose a scene graph representation that organizes the scene by explicitly representing spatial and appearance relationships between objects and their surroundings. This structured representation essential for guiding later constrained collision resolution and ensuring consistency in later stages of the pipeline.

---

Scene Graph Representation and Parsing. In the scene graph, nodes represent key elements in the scene, such as walls, windows, doors, and detected objects. Each node is associated with several attributes: 1) spatial properties, including center location C E R$ (where C = (x, y, =)), dimensions D E TR3 (where D = (w, hi I)), and orientation E RR3 (three rotation angles around the y. and axes); and 2) appearance properties, represented by the top-k most visible cropped images of the object, denoted as a set I = {41 Jz. . Jx These attributes are essential for material estimation, model selection, and layout optimization. Edges between nodes define key spatial relationships and, once constructed, can impose constraints during the spatial arrangement process, ensuring physically plausible scene. We define four key spatial relationships: 1) attached to walls, for objects physically connected to walls (e.g.. shelves, cabinets); 2) on top of, for objects placed top of others (e.g.. vase table); 3) table-chair pair, grouping tables and chairs in typical arrangements; and 4) connecting to, where objects are next to each other and touched. These relationships provide essential constraints for maintaining consistent object placement, preventing collisions, and improving overall scene realism. To compute the edges of the scene graph, the construction process begins by parsing walls into connected components, forming closed polygonal regions that delineate individual rooms. Noisy wall detections are aligned by snapping them to a grid, ensuring well-defined room boundaries. Spatial relationships between objects are inferred based on their center location and orientation, with predefined rules governing each type of relationship. Collision Resolution approach.
We resolve object collisions iteratively using a Constraint- For each intersecting pair, virtual forces are applied along collision vectors to separate them, while adhering to spatial constraints from the scene graph. For example, wall-attached objects are restricted to wall-aligned movement, and table-chair pairs retain their relative positions. This process continues until collisions are resolved or a maximum iteration count is reached, ensuring a realistic, physically plausible layout. More details can be found in the supplementary materials.
3.2 Object Reconstruction Stage
In the age of ever-expanding 3D asset datasets, it is crucial to develop a robust and scalable retrieval mechanism. Previous approaches [27, 4, 3] often rely on pair-wise pre-trained models for retrieval, but these are limited to specific datasets. We believe it is important to adopt a training-free approach, as it would be more adaptable and scalable when the dataset expands. We adopt a hierarchical retrieval approach that leverages multiple information modalities semantic labels, visual features, and camera pose- to progressively refine the search space and improve accuracy. The retrieval begins with semantic filtering stage, where objects are categorized by subcategories (e.g., two-seat sofas, bar chairs) to exclude irrelevant candidates. This followed by an image-based retrieval stage, where cropped 2D views of the input object are compared against pre-rendered images of candidate objects using a pretrained feature encoder (D1NOv2[45]). The top 10 visually similar matches are selected. In the subsequent pose-aware rendering and comparison step, these candidates are placed into the scene according to the detected pose and rendered from the same camera angles. The resulting views are then cropped and re-encoded for visual feature extraction, further narrowing the selection to the top four candidates. Finally, a contextual selection step employs a language model to assess high-level attributes such style, proportion, and visual coherence, yielding the best matched object. This structured, statt--stge pipeline ensures scalable and context-aware retrieval across large datasets. To group identical highly similar objects, we cluster items within each subcategory using DINOv2 features extracted from cropped images. These features, which capture shape and structure, are averaged per object and clustered via KMeans, with the number of clusters selected by maximizing the silhouette score. Since DINOv2 is color-invariant, we further subdivide clusters by their dominant color to form fine-grained, visually and stylistically consistent groups for joint retrieval.
3.3 Material Painting Stage
One of the primary challenges in the pipeline lies in effective and accurate material recovery. While prior approaches [59, 63, 36] have made notable progress using differentiable rendering or prediction- based techniques, they struggle to scale effectively to real-world, noisy environments. This limitation arises from three key factors. First, these methods rely on precise alignment between material segments and cropped image regions. In practice, due to geometric misalignment of objects, their approaches to obtaining reliable crop mappings often fail, resulting noisy or inconsistent references. Second, poor lighting conditions frequently lead to dark or visually ambiguous crops which, even if correctly mapped to material segments, still result degraded material initialization. Finally, proce- dural, graph- differentiable material rendering introduces significant computational overhead,
5

---

Layout Reconstruction
Winddwws Doors Assembl
3.Object Reconstruction
Enabling Interactivity
Figure Procedural Reconstruction Stage. This stage progressively rebuilds rooms by reconstruct-
ing layouts, assembling dooors and windows, placing objects, and enabling interactive attributes.
limiting the ability process large-scale scenes efficiently. In the Material Painting Stage, we address prior limitations with a compact pipeline that includes automatic crop mapping, multi-step material selection, and lightweight albedo-only optimization. For each retrieved 3D asset, predefined material groups are assigned materials inferred from captured images.
Auto-crop Mapping. The input to this stage includes multiple cropped image patches and corre- sponding 3D models. We begin by applying SAM [23]-guided segmentation extract meaningful patches from the images. Specifically. we compute the largest rectangular region within each mask and filter out invalid segments using Grounding DINO detection. Priority is given smooth, clean regions and larger areas, from which we select the top-k ranked patches. To associate 3D material seg- ments with these image patches, generate informative visual prompts and employ Multi-Modal Large Language Model (MLLM) [44] for semantic mapping.
Semantic and Visual Guided Material Search. To initialize high-quality materials, we combine language-guided and vision-based retrieval from material database. First, inspired by Make-It- Real [13], we use multi-step prompting to refine material category predictions and select the top 10 candidates. Then, we extract CLIP embeddings from cropped multi-view patches and compare them with embeddings of the selected materials. Finally, GPT-4 evaluates the visual compatibility of albedo maps with reference patches, ensuring semantic and visual alignment. This multi-stage process produces reliable PBR material initialization for each 3D segment.
Albedo-Only Optimization. With initial mate- rial assignments in place, further refinement is possible using procedural graph-based methods like MaTCH [50] PSDR-Room [59], but these incur high computational costs. Instead, we ap- ply lightweight, albedo-only adjustment in the CIE LAB color space. Rather than re-optimizing full PBR parameters, we shift the global color dis- tribution of the albedo to better match cropped image briches-preservinng high-frequency details while correcting hue and brightness. Formally, let S(p) R the LAB vector of the source albedo at pixel p. T E R the target LAB vector, and P the set of pixels in the albedo map. The adjusted color is:
Reionne saga
Input
(a) Material Painting Stage Input
Auto Crop Mapping
Fay wutorall yer muek
Alleed 73yt Vasudal Tueber
S'(p) = S(p) +
T
Σ S(g) OEP
Seleciio
[P]
Albata-Dely Calor Adielaas
Output
Pained TD rankes
This shift aligns the mean color of the albedo with the target while preserving texture details. To infer T. we query an MILM for part-specific RGB values across views, convert the consensus to LAB, and apply the shift. This efficient post-processing balances visual fidelity and scalability.
(6) Multi-Sup Material Sclection and Optimization
Figure 4: Materials Painting Stage Pipeline. Given a 3D model and reference images, this stage recovers realistic PBR materials that en- hance visual realism for the 3D object.

---

RGBD
LiteReahty
RGBD
LiteReality
RGBO
bitekcaling city
RGBD
Litecca
RGBD
LiteRdality
RGBD
liteRealiyy
RGBB
Litekeality
RGBD
LiteReality
RGP
LiteRiality
RGBD
LiteRealh
RGBD
LiteReality
Figure 5: Graphics-Reade Reconstruction by LiteReality. Noisy indoor scans are converted into compact, realistic scenes with full PBR materials for all objects, ready for downstream graphics tasks.
3.4 Procedural Reconstruction Stage in Blender
Finally, we use predefined procedural pipeline to reconstruct the scene. The process begins with constructing the walls, followed by assembling windows and doors, and finally placing objects within room. To support physics-based interaction, we assign rigid-body properties to each object and use their mesh geometry collision boundaries. Large structural elements, such as walls, are marked as passive rigid bodies, while movable items are set as active rigid bodies. Estimating object mass crucial for realistic simulation. To achieve this, crop images of each object and input them into a Multimodal Large Language Model (MLLM), which predicts their mass late-later used to simulate realistic rigid-body dynamics. The entire procedural reconstruction process is carried out within Blender.
4 Experiments and Results
We evaluate LiteReality both end-to-end and per stage. Since no prior work directly tackles this task, we propose three benchmarks: retrieval similarity, object-centric PBR material estimation, and full-scene graphics-ready reconstruction. These enable quantitative comparisons with prior work and provide foundation for future research. We also demonstrate advanced applications of our outputs in the supplementary material.
7

---

Reference Images
3D models
3D model
Reference Images
Figure 6: Material Painting Stage Demo: Sin- gle reference image applied multiple 3D mod- els. Close-to-reality PBR materials are recovered, even with sever geometries misalignments.
Figure 7: Material Painting Stage Demo: A 3D model is painted using diverse reference images. Realistic PBR materials are recovered, even under challenging conditions like cartoons or drawings.
LiteReality Database. LiteReality database is structured to align with Apple's RoomPlan, support- ing real-world scene reconstruction across 17 semantic classes. We organized our dataset accordingly, ensuring each category relevant subcategories- has corresponding retrieval database. To build this database, we curated assets primarily from 3ID-Future [14] and AI2-THOR [24]. 3D-Future offers diverse indoor furniture, while A12-THOR, though smaller, provides detailed articulation data for interactive scenes. For underrepresented categories, we supplemented assets from Sketchfab [51] under public-access licenses. As of May 2025, our retrieval database includes 5,283 assets; full statistics are available in the supplementary materials.
Retrieval Similarity Benchmarking, To evaluate the performance of proposed object retrieval method, we conduct experiments on the ScanNet dataset [8]. Since Scan2CAD[3] provides hand- picked ShapeNet[57] models as ground-truth annotations, we perform retrieval over the full ShapeNet database. Following [2, 16, 55], we measure retrieval accuracy using the L1 Chamfer Distance between the retrieved model and the ground-truti Unlike Scan2CAD, which searches within per-scene CAD pool of roughly
candidates, we retrieve from the entire ShapeNet repository, where each category contains 300-3 000 models. Although similar evaluations have been conducted before, direct comparison is hindered by sparse implementation details in prior work. provide a complete description of our evaluation protocol in the supplementary material to facilitate reproducibility and future comparisons. Experiments use the ScanNet validation scenes.
Object-Centris Material Recovering Benchmarking BRDF estimation of retrieved objects based on scanned images brings the appearance of real-world objects into the digital space. To evaluate the performance of our proposed method, we established benchmark using five indoor scenes captured with an iPhone 13 Pro Max running the 3D Scanner App in RoomPlan mode. The scenes include meeting room, a common area, a bedroom and two study rooms, contains total of 11 objects across eight categories. For each object, we selected the four most visible frames and tightly cropped each to isolate the object, highlighting real-world scanning challenges such as occlusion, poor lighting, and misalignment. We compare our material painting pipeline against four basclines: (1) PhotoShape[46], a material selection algorithm; (2) Make-It- It-Rel [33a[[1]), [3], which use language model to retrieve materials; (3) our Visual+Language Search method; and (4) Make-It-Real with our Albedo-Only Optimization. To enable benchmarking, we also created ground-truth CAD models for the scenes. PBR materials estimated by each method were applied to the models, which placed at their original poses, and rendered with global illumination using HDR environment maps. Evaluation was based on perceptual differences between the cropped scan images and the corresponding rendered views. More details are presented in the supplementary materials.
Graphics-Read, Reconstruction Benchmarking. To assess end-to-end reconstruction quality, introduce benchmark for graphics-ready reconstruction. Only scenes that are object-centric

---

Table I: Quantitative Evaluation of Retrieval Similarity. We compute the two-way Chamfer Distance between the normalized GT CAD models annotated in Scan2CAD and our retrieved results. Evaluation is performed the ScanNet validation set. Our retrieval method consistently outperforms prior approaches, achieving the most accurate shape matches across multiple object categories.
Method
avg/CAD2
avg/class
bath
blssl 1
eab
chr
sfa
[bl
bin
MSCD [55]
0.1 103
0,1188
0.1215
0.1114
0.0931
0.1019
0.1425
0.1071
0.1224
0.1119
Digital Cousin [9]
0.1411
0.1246
0.1439
0.1166
0.1105
0.1363
0.1762
0.1083
0,1832
0.1098
ScanNotate
0.1042
0.1110
0.1161
0.0870
0.0908
0.0995
0.1376
0.1046
0.1183
0.0921
LiteReality
0.0986
0.1067
0.1109
0.0845
0.0859
0.0943
0.1309
0.0951
0.1099
0.0901
Table 2: Object-Centric PBR Material Esti- mation Comparison. Evaluated on 110 objects across five real-world scanned scenes, LiteReal- ity method demonstrates strong capabilities.
Table Perceptual Quality Evaluation for
Full Scenes. Tested by comparing rendered im- ages to captured RGB, LiteReality significantly outperforms prior works across all metrics.
Methods
RMSE
SSIM
LPIPS
Methods
RMSE +
SSIM f
LPIPS $
MIR [13]
0.2377
0.3981
0.6111
Phone2Proc[1] [1]
0.3604
0,5512
0.7338
PhotoShape [46]
0.3225
0.2371
0.6558
Digital Cousin [9]
0.3653
0.5531
0.7364
MIR [13] + AO
0.2156
0.4203
0,5899
DC [9] Sem& Vis
0.3226
0,5425
0.6717
Sem& Vis
0.2835
0.3758
0.6362
DC [9] MIR [13]
0.3046
0,5492
0.6648
LiteReality
0.2163
0.4353
0.5854
LiteReality
0.2664
0,5818
0.6522
Crop
PhotoShupe
MIR
Sem&&Vis
MIR AOO
LiteRealtyy
RGDD Scan
Digital Cousia
Phone2Proe
LiteReality
Sem&
DC MIR
Figure 8: Object-Centric PBR Material Compari- Figure 9: Graphics-Reade Reconstruction Com- son. Note: object pose are unused in MP pipeline
parison. LiteReality showcases best realism.
and realistic (i.e., each object is an individual entity with PBR materials) qualify for graphics- ready reconstruction evaluation. We evaluate four methods in total. First, we reproduce two prior approaches: Phone2Proc [11] and Digital Cousin. Second, we extend the Digital Cousin framework by integrating material selection via (1) the original Make-It-Red Rt-Rel (MIR) pipeline and (2) our proposed remantic-and-visee guided scheme. All methods are tested on five diverse scenes captured with an iPhone, each containing hundreds of RGB frames. The results are shown in Table 3.
Results. shown Table 1, our training-free retrieval method achieves the best similarity matching among all baselines. Table 2 quantifies the effectiveness of our material painting approach, clearly outperforming prior methods. For overall scene reconstruction. LiteReality shows strong overall performance in Table 3, achieving the highest visual quality. Figures 5 and illustrate results across diverse real-world scenes and comparisons with baselines. Figure 8 showcases the superior visual results of our proposed methods in PBR comparisons. We further validate robustness in two challenging scenarios: painting multiple 3D models from a single image (Figure 6) and painting a single model using diverse images (Figure 7), including cartoons, drawing Al-generated content. In both cases, our method produces complete, visually plausible PBR materials that matched that in the image. Additional
-ellatioons-riigiiiing editing, physies, AR/VR, and robotics-ant are shown in the supplementary material.
Runtime Analysis On single NVIDIA RTX 3090 GPU with 24 GB of memory, the complete reconstruction of room-scale scene takes between and minutes, depending on scene complexity. For example, small study or bedroom scene containing around 10-15 objects typically requires about

---

20 minutes, whereas larger spaces such as meeting rooms or boardrooms with 40-50 objects take up to an hour. Among the individual stages, preprocessing and scene parsing complete within 1-3 minutes, object retrieval requires 2-5 minutes, while the material painting stage dominates the runtime, taking roughly 15-50 minutes depending on the number of objects. Procedural reconstruction and export in Blender add less than two minutes on average.
5 Conclusion
LiteReality demonstrates the ability to reconstruct realistic and interactive indoor scenes from RGB- D scans. Our results highlight the potential of the proposed pipeline in generating visually compelling environments with physics-basee interactions and dynamic lighting. We hope this work will inspire further exploration toward graphics-reat readyy 3D scene reconstruction.
The overall reconstruction quality remains closely tied to the accuracy of upstream perception. Errors in object detection and layout estimation can propagate later stages. In most real- world cases, our input scans are captured using Apple's RoomPlan app, which provides reasonably reliable geometry through interactive scanning. While propose comprehensive retrieval pipeline, object retrieval remains inherently challenging occasional mismatch can noticeably affect scene realism. Fortunately, due the object-centric nature of our representation, such errors are easy to correct through local replacement or refinement, Finally, our current constraint-basee collision solver may struggle densely packed or complex arrangements, sometimes leading to interpenetrations and preventing fully physics-valic simulations.
Limitations.
Future Work. Future work could focus on extending object detection and retrieval include smaller items, thereby enriching scene detail and overall realism. Incorporating learning- based light estimation together with full SVBRDF recovery would enable faithful relighting under novel illumination conditions. Scaling the pipeline from single-room to multi-room and building reconstructions would further unlock applications large-scale visualization, robotics simulation, and digital-twin creation. In addition, optimizing the entire system for real-time or near real-time performance would make it highly practical for interactive applications and on-device deployment.
Acknowledgment. This work was supported by the Girton College Graduate Research Award at the University Cambridge, the School Technology, and the EPSRC Centre for Doctoral Training. We gratefully acknowledge the insightful discussions and feedback from Prof. Kwang Moo Yi, Erqun Dong, Weiyang Liu, Prof. Ayush Tewari, and Shangzhan Zhang throughout the course of this work. We also appreciate the kind support of Prof. Angela Dai for providing the voiceover for the demo videos, and we thank Jicying Chen for her help in proofreading the paper.
10

---

References
[1] Stefan Ainetter, Sinisa Stekovic, Friedrich Fraundorfer, and Vincent Lepetit. Automatically annotating indoor images with cad models via rgb-d scans. In WACV, 2023. 4, 4,9, 25
[2] Stefan Ainetter, Sinisa Stekovic, Friedrich Fraundorfer, and Vincent Lepetit. HOC-Search: Efficient CAD model and pose retrieval from RGB-D scans. In 3DV, 2024, 4, 8, 25
[3] Armen Avetisyan, Manuel Dahnert, Angela Dai, Manolis Savva, Angel X. Chang, and Matthias Nießner. Scan2CAD: Learning CAD model alignment in RGB-D D seans. In CVPR, 2019. 4, 5,5,8, 24
[4] Armen Avetisyan, Angela Dai, and Matthias Nießner: End-to-end cad model retrieval and 9dof alignment in 3d scans, In ICCV, 2019. 5,5
[5] Armen Avetisyan, Tatiana Khanova, Christopher Choy, Denver Dash, Angela Dai, and Matthias Nießner. SceneCAD: Predicting object alignments and layouts in rgb-d scans, In ECCV, 2020. 3, 4
[6] Armen Avetisyan, Christopher Xie, Henry Howard-Jenkins, Tsun- Yi Yang. Samir Aroudj, Suvam Patra, Fuyang Zhang, Duncan Frost, Luke Holland, Camphell Orme, Jakob Engel, Edward Miller, Richard Newcombe, and Vasileios Balntas. Scenescript: Reconstructing scenes with an autoregressive structured language model. In ECCV, 2024. 4
[7] Jonathan T Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, and Pratul P Srinivasan, Mip-nerf: A multiscale representation for anti aliialliinng neural radiance fields. In ICCV, 2021. 2
[8] Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: annotael 3d reconstructions of indoor scenes. In CVPR, 2017. 2, 8, 25
[9] Tianyuan Dai, Josiah Wong. Yunfan Jiang. Chen Wang. Cem Gokmen, Ruohan Zhang. Jiajun Wu, and Fei-Fei Li, Automated creation of digital cousins for rohust policy learning. In CoRI, 2024. 2, 2,3, 9, 25, 27
[10] Mant Deitke, Winson Han. Alvaro Herrasti, Aniruddha Kembhavi, Eric Kolve, Roozbeh Mottaghi. Jordi Salvador, Dustin Schwenk, Eli VanderBilt, Matthew Wallingford, Luca Weihs, Mark Yatskar, and Ali Farhadi. RoboTHOR: An Open Simulation-to to-R Embodied AI Platform. In CVPR, 2020. 3
[11] Matt Deitke, Rose Hendrix, Ali Farhadi, Kiana Ehsani, and Aniruddha Kembhavi. Phone2proc: Bringing robust robots into our chaotic world. In CVPR, 2023. 2, 9, 27
[12] Matt Deitke, Eli VanderBilt, Alvaro Herrasti, Luca Weihs, Jordi Salvador, Kiana Ehsani, Winson Han, Fric Kolve, Ali Farhadi, Aniruddha Kembhavi, and Roozbeh Mottaghi. ProcTHOR: Large-Scale Embodied AI Using Procedural Generation. In NeurlPS, 2022. 2.3
[13] Ye Fang, Zeyi Sun, Tong Wu, Jiaqi Wang. Ziwei Liu, Gordon Wetzstein, and Dahua Lin. Make-it rtall Unleashing large multimodal model for painting 3d objects with realistic materials. In NeurlPs, 2024. 6, 8, 9, 26
[14] Huan Fu, Rongfei Jia, Lin Gao, Mingming Gong, Bingiang Zhao, Steve Maybank, and Dacheng Tao. 3d fl-future: 3d furniture shape with texture. NJV, 2021. 8, 23
[15] Chuang Gan, Jeremy Schwartz, Seth Alter, Damian Mrowca, Martin Schrimpf, James Traer, Julian De Fre- itas, Jonas Kubilius, Abhishek Bhandwaldar, Nick Haber, Megumi Sano, Kuno Kim, Elias Wang, Michael Lingelbach, Aidan Curtis, Kevin Feigelis. Daniel M. Bear, Dan Gutfreund, David Cox, Antonio Torralba, James J. DiCarlo, Joshua B. Tenenbaum, Josh H. McDermott, and Daniel L. K. Yamins, Threedworld: A platform for interactive multi-modal physical simulation. In NeurlPs, 2021.
[16] Daoyi Gao, Dávid Rozenberszki, Stefan Leutenegger, and Angela Dai. DifICAD: Weakly probabilistic cad model retricval and alignment from an rgb image. In ACM TOG (Transactions on Graphics), 2024. 8, 25
[17] Can Gümeli, Angela Dai, and Matthias Nießner. Roca: Robust cad model retrieval and alignment from single image. CVPR, 2022. 2,
[18] Shreyas Hampali, Sinisa Stekovic, Sayan Deb Sarkar, Chetan Srinivasa Kumar, Friedrich Fraundorfer, and Vincent Lepetit. Monte carlo scene scarch for 3d scene understanding. In CVPR, 2021.
[19] Siyuan Huang, Siyuan Qi. Yinxue Xiao, Yixin Zhu, Ying Nian Wu, and Song- -hhn Zhu. Cooperative holistic scene understanding: Unifying 3d object. layout, and camera pose estimation. In NewrlPs, 2018.
[20] Siyuan Huang, Siyuan Qi, Yixin Zhu, Yinxue Xiao, Yuanlu Xu, and Coug-Chun Zhu. Holistic 3d scene parsing and reconstruction from single rgb image. In ECCV, 2018. 2,
11

---

[21] Hamid Izadinia, Qi Shan, and Steven M. Seitz. Im2cad. In CVPR, 2017. 2,4
[22] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 2023. 2,
[23] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan- Yen Lo, Piotr Dollár, and Ross Girshick. Segment anything.
In ICCV, 2023. 6
[24] Eric Kolve, Roozbeh Mottaghi, Winson Han, Eli VanderBilt, Luca Weihs, Alvaro Herrasti, Daniel Gordon, Yuke Zhu, Abhinav Gupta, and Ali Farhadi. A12-THOR: An Interactive 3D Environment for Visual AL arXiv, 2017. 3, 23
[25] Weicheng Kuo, Anelia Angelova, Tsung- Yi Lin, and Angela Dai. Mask2cad: 3d shape prediction by learning to segment and retrieve. In ECCV, 2020. 2,4 4
[26] F. Langer, G. Bae, I. Budvytis, and R. Cipolla. Spare: Sparse render-and-compare for cad model alignment in single rgb image. In BMVC, 2022. 2,4
[27] Florian Maximilian Langer, Jihong Ju, Georgi Dikov, Gerhard Reitmayr, and Mohsen Ghafoorian. Fast- CAD: Real-time CAD retrieval and alignment from scans and videos. In ECCV, 2024. 4,5, 24, 25
[28] Justin Lazarow, David Griffiths, Gefen Kohavi, Francisco Crespo, and Afshin Dehghan. Cubify anything: Scaling indoor 3d object detection. In CVPR, 2025. 2,3
[29] Beichen Li, Liang Shi, and Wojciech Matusik. End-to-en end procedural material capture with proxy-free mixeeeintegeer optimization. ACM Transactions on Graphics, 2023. 4
[30] Chengshu Li. Ruohan Zhang, Josiah Wong, Cem Gokmen, Sanjana Srivastava, Roberto Martin-Martin, Chen Wang, Gabrael Levine. Michael Lingelbach, Jiankai Sun. Mona Anvari, Minjune Hwang. Manasi Sharma, Arman Aydin, Dhruva Bansal, Samuel Hunter, Kyu-Young Kim, Alan Lou, Caleb R Matthews, Ivan Villa Renteria, Jerry Huayang Tang, Claire Tang, Fci Xia, Silvio Savarese, Hyowon Gweon, Karen Liu, Jiajun Wu, and Li Fei-Fei. BEHAVIOR 1k: A benchmark for embodied Al with 1,000 everyday activities and realistic simulation. In CoRL, 2023. 3
[31] Kejie Li, Daniel Donen Steven Chen, Minh Vo, Ian D. Reid, Hamid Rezatofighi, Chris Sweeney, Julian Straub, and Richard Newcombe. Odam: Object detection, association, and mapping using posed rgb video. In ICCV, 2021.
[32] Zhenggin Li, Ting- Wei Yu, Shen Sang. Sarah Wang. Meng Song, Yuhan Liu, Yu-Yin Ying Ych, Rui Zhu, Nitesh Gundavarapu, Jia Shi, Sai Bi, Zexiang Xu, Hong-Xing Yu, Kalyan Sunkavalli, Miloš Hašan, Ravi Ramamoorthi, and Manmohan Chandraker. Openrooms: An open framework for photorcalistic indoor scene datasets. In CVPR, 2021.
[33] Guying Lin, Kemeng Huang, Michael Liu, Ruihan Gao, Hanke Chen, Lyuhao Chen, Beijia Lu, Taku Komura, Yuan Liu, Jun- Yan Zhu, and Minchen Li. Pat3d: Pryysics-augmentd text-to-3d scene generation. In ICLR, 2026. 3
[34] Hung-Jin Lin and Shang Hong Lai. Deeproom: 3d room layout and pose estimation from a single image. In Asian Conference on Pattern Recognition (ACPR), pages 211 225. Springer, 2020. 3
[35] Lu Ling, Chen- Hsuan Lin, Tsung- Yi Lin, Yifan Ding, Yu Zeng, Yichen Sheng. Yunhao Ge, Ming- Yu Liu, Aniket Bera. and Zhaoshuo Li. Scenethesis: Combining language and visual priors for 3d scene generation. In ICLR, 2026. 3
[36] Ivan Lopes, Fabio Pizzati, and Raoul de Charette. Material palette: Extraction of materials from a single image. In CVPR, 2024. 2,
[37] Xiaoyang Lyu, Chirui Chang, Peng Dai, Yang-Tian Sun, and Xiaojuan Qi. Total-decom: Decomposed 3d scene reconstruction with minimal interaction. In CVPR, 2024. 3
[38] Kevis-Kokits Maninis, Stefan Popov, Matthias NicBner, and Vittorio Ferrari. Cad-estate: Large-scale cad model annotation in rgb vidcos. In ICCV, 2023. 4
[39] Maninis, Stefan Popov, Matthias Nießner, and Vittorio Ferrari Vid2cad: Cad model
alignment using view constraints from videos. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45:1320- 10000113,,2000 2023. 4, 24
12

---

[40] Yongsen Mao, Junhao Zhong, Chuan Fang, Jia Zheng, Rui Tang, Hao Zhu, Ping Tan, and Zihan Zhou. Spatiallm: Training large language models for structured indoor modeling. In NewrlPS, 2025. 4
[41] Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields view synthesis. In ECCV, 2020. 2,
[42] Srivathsan Murali, Pablo Speciale, Martin R. Oswald, and Mare Pollefeys. Indoor scan2bim: Building information models house interiors. In IROS, 2017.
[43] Yinyu Nie, Xiaoguang Han, Shihui Guo, Yujian Zheng, Jian Chang, and Jian Jun Zhang. To- uuBdunderstanding: Joint layout, object pose and mesh reconstruction for indoor scenes from a single image. In CVPR, 2020. 3
[44] OpenAL. Gpt-4 technical report. arXiv:2303.08774 2023, 6
[45] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V. Vo, Mare Szafraniec, Vasil Khalidow, Pierre Fernandez, Daniel HAZIZA, Francisco Massa. Alaacidin El-Nouby, Mido Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Ya-Yao Huang, Shang- theng-Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Herve Jegou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Rojanowski. DINOv2: Learning robust visual features without supervision. Transactions on Machine Learning Research. 2024. Featured Certification.
[46] Keunhong Park, Konstantinos Rematas, Ali Farhadi, and Steven M. Seitz. PhotoShape: Photorealistic materials for scale shape collections. ACM Transactions on Graphics, 2018. 8, 9,
[47] Xavier Puig, Kevin Ra, Marko Boben, Jiaman Li, Tingwu Wang. Sanja Fidler, and Antonio Torralba. Virtualhome: Simulating household activities via programs. In CVPR, 2018. 3
[48] Xavier Puig, Eric Undersander, Andrew Szot, Mikael Dallaire Cote, Tsung-Yen Yang. Ruslan Partsey, Ruta Desai, Alexander William Clegg, Michal Hlavac, So Yeon Min, Vladimír Vondruš, Theophile Gervet, Vincent-Pierre Berges, John M. Turner, Oleksandr Maksymets, Zsolt Kira, Mrinal Kalakrishnan, Jitendra Malik, Devendra Singh Chaplot, Unnat Jain, Dhruv Batra, Akshara Rai, and Roozbeh Mottaghi. Habitat 3.0: A co-habitat for humans, avatars and robots. arXiv:2310.13724, 2023.
[49] Johannes Lutz Schönberger and Jan -in-Mich Frahm. Structure-from motooootion revisited. In CVPR, 2016. 2,
[50] Liang Shi, Beichen Li, Miloš Hašan, Kalyan Sunkavalli, Tamy Boubekeur, Radomir Mech, and Wojeiech Matusik. MATch: Differentiable material graphs for procedural material capture. ACM Transactions on Graphics, 2020.
[51] Sketchfab. models and assets. https://skettee
com/ 23
[52] Julian Straub, Daniel DeTone, Tianwei Shen, Nan Yang, Chris Sweeney, and Richard Newcombe, EFM3D:
A benchmark for measuring progress towards 3d egocentric foundation models. In ECCV, 2024. 3,
[53] Apple Inc. RoomPlan Team. 3d parametric room representation with roomplan. Apple Machine Learning Research, 2022. 2, 23
[54] Michal J. Tyszkiewicz, Kevis-Kokitsi Maninis, Stefan Popov, and Vittorio Ferrari. RayTran: 3d pose estimation and shape reconstruction of multiple objects from videos with ray-traced transformers. In ECCV, 2022. 4
[55] Jiaxin Wei, Lan Hu, Chenyu Wang, and Laurent Kneip. Accurate instance-level CAD model retrieval in large-scale database. In IROS, 2022. 4, 8, 9,
[56] Changchang VisualSFM: visual structure cenectur--foo----e from-motion system, 2011.
[57] Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Linguang Zhang, Xiaoou Tang, and Jianxiong Xiao.
3d shapenets: deep representation for volumetric shapes. CVPR, 2015.
[58] Qiangeng Xu, Weiyue Wang, Duygu Ceylan, Radomir Mech, UIrich Neumann. DISN: Deep implicit surface network for high-quality single-view 3d reconstruction. In NeurlPs, 2019. 25
[59] Kai Yan, Fujun Luan, Miloš Hašan, Thibault Groueix, Valentin Deschaintre, and Shuang Zhao. Psdr-room: Single photo scene using differentiable rendering. In SIGGRAPH ASIA, 2023. 2, 3, 5,
[60] Jianwei Yang, Hao Zhang, Feng Li, Xueyan Zou, Chunyuan Li, and Jianfeng Gao. Set-of-matt prompting unleashes extraordinary visual grounding gpt-4v, 2023. 24
13

---

[61] Yue Yang, Fan- Yun Sun, Luca Weihs, Eli VanderBilt. Alvaro Herrasti, Winson Han, Jiajun Wu, Nick Haber, Ranjay Krishna, Lingje Liu, Chris Callison-Burei Mark Yatskar, Aniruddha Kembhavi. and Christopher Clark. Holodeck: Language guided generation of embodied environments. In CVPR, 2024. 2,
[62] Kaixin Yao, Longwen Zhang, Xinhao Yan, Yan Zeng. Qixuan Zhang, Wei Yang, Lan Xu, Jiayuan Gu, and Jingyi Yu. CAST: Component-alignnn 3D scene reconstruction from an RGB image. ACM Transactions on Graphics, 2025. 3
[63] Yu- Ying Yeh, Zhengqin Li, Yannick Hold- ceold-eeffroy. Rui Zhu, Zexiang Xu, Miloš Hašan, Kalyan Sunkavalli, and Manmohan Chandraker. Photoscene: Photorealistic material and lighting transfer for indoor scenes. In CVPR, 2022. 2, 5.5
[64] Shaofeng Yin. Jiaxin Ge, Zora Zhiruo Wang, Xiuyu Li, Michael J. Black, Trevor Darrell, Angjoo Kanazawa, and Haiwen Feng. ssiion-s-inverse-gunne
agent via interleaved multimodal reasoning, 2026.
[65] Huangyue Yu, Baoxiong Jia, Yixin Chen, Yandan Yang. Puhao Li, Rongpeng Su, Jiaxin Li, Qing Li, Wei Liang. Song-Chun Zhu, Tengyu Liu, and Siyuan Huang, Metascenes: Towards automated replica creation for real-world 3d scans. In CVPR, 2025.
[66] Shangzhan Zhang, Sida Peng. Tao Xu, Yuanbo Yang. Tianrun Chen, Nan Xue, Yujun Shen, Hujun Bao, Ruizhen Hu, and Xiaowei Zhou. Mapa: Text-driven photorealistic material painting for 3d shapes. In SIGGRAPH, 2024.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Please refer the contribution sections for details.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We have dedicated session in the supplementary materials for this. Please refer to the supplementary materials for the limitations.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA].
15

---

Justification:
Not Applied
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer:| Yes]
Justification: We document the experimental results detail in the main paper and supple- mentary materials. The code will be released to reproduce the results.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
16

---

Answer: [Yes]
Justification: Yes, the dataset and code used will be released to reproduce the results and to test the proposed method on real-life scans.
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.c co/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to reproduce the results. See the NeurlPS code and data submission guidelines (https:
gwwg///ideee//e/ /CoeeSum piinnicyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We document all the details necessary to reproduce the results in both the main paper and the supplementary materials.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [NA]
Justification:
This is not applicable to our experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
17

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Yes, please refer to the supplementary materials for this.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https: https://neett httppp::///uuipp/// cc/public/Bthiicciiiiee
Answer: [Yes]
Justification: The research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Please refer this to the supplementary materials.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
18

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA].
Justification:
This is not applied to our paper.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All datasets used in this work are publicly available from prior publications and free from copyright restrictions.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
19

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: Yes, please refer Section 4 as well supplementary materials.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This is not applied to our paper.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification::he is not applied to our paper.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
20

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration is not required.
Answer: [Yes]
Justification: We have clearly documented how the LLM is helpful in proposed methods in the methodology section.
Guidelines:
The answer NA means that the method development in this research does involve LLMs as any important, original, or non-standard components.
cc/ /ccc//oooeeeeccce/22////1 Please refer to our LLM policy (https http://ips.cc/ooofeeeeeeeeeeeeee for what should or should be described.
21

---

Appendix
Scene0050 00
LiteReality
RGBD
LiteReality
RGDD
LiuReal
Scene0 163 000
ROBB
Lite
Sccne0294 02
RORD
LilRRa ht
RGBD
AttReality
RGBD
LiteR enlley
Scene0281 00
RGBD
LiteReality
RGBD
CieeRclity
RGP
Line
Figure 10: Graphics-Reas Reconstruction by LiteReality on ScanNet Dataset Noisy indoor scans are converted into compact, realistic scenes with full PBR materials for all objects, ready for downstream graphics tasks.
22

---

A Outline
In this supplementary document, we present five key sections that provide additional context and depth to our main paper:
1. Database Explanation. We explain the construction of our LiteReality database and discuss how it can be expanded in future work.
2. Benchmark Details. We describe the construction and presentation of three benchmarks, including the rationale behind their design and implementation details to ensure reproducibil- ity.
3. Methodology Details. We provide further description of each stage in our reconstruction pipeline.
4. Additional Results. We present further reconstruction results the ScanNet dataset, including qualitative examples and accompanying supplementary videos.
5. Application Demonstrations. We showcase several advanced applications enabled by our Graphics-Ready Reconstruction, demonstrating its versatility in the rendering pipeline, virtual reality, robotics, and scene editing.
Finally, we discuss broader limitations of our current system and outline directions for future work.
B LiteReality Database
The overarching goal in designing LiteReality is to create robust system capable of converting real-world seans into graphics-ready environments suitable for downstream applications such as VR, robotics, and interactive rendering. Many of our design decisions and technical innovations are driven by the challenge of working with in-the-wild scans. Currently, one of the most reliable solutions for high-quality bounding box and layout detection is Apple's RoomPlan system [53], which enables interactive indoor scanming with augmented reality support. Our database thus specifically tailored to be compatible with this capture system.
The RoomPlan framework recognizes 17 key room-defining furniture categories, including bathtub, bed, chair, fireplace, oven, refrigerator, sink, sofa. stairs, storage, stove, table, toiler, and washer- dryer, as well as architectural elements like windows and doors. Accordingly, LiteReality includes these 17 major categories. For broad object types-such as chairs, tables, storage units, and beds-we further divide them into finer subcategories to better reflect the diversity encountered in real-world settings.
Our object assets are sourced from a combination of public datasets, including 3D-FUTURE [14], the articulated dataset from A12-THOR [24], and several curated handheld scans from Sketchfab [51]. For most models, we provide material- segmentation to support the later Materiel Painting stage. Articulated assets primarily from A12-THOR are carefully processed to maintain sub-mest mesh structures that accurately represent jointed or movable parts. Note that the number of articulated furniture models remains limited, expanding this subset represents an important direction for future development.
Each 3D model LiteReality undergoes the following on-boarding steps:
1. Multi-view rendering: We generate object renders from multiple viewpoints and extract DINOv2 features from these images, which are stored for use in later retrieval stages.
2. Orientation normalization: Object orientations are aligned to be front-facing, ensuring consistent placement within axis axis-aligned bounding boxes.
3. Material segmentation rendering: We extract material segments and generate per-part segmentation renders from multiple viewpoints, then select the most visually representative image for each material segment.
4. Material segmentation visualization: We create informative visualizations of material segmentations by stitching together the representative images for each material part. These visualizations are used in later stages for material matching and retrieval, and have been
23

---

shown to outperform SOM-based [60] labeling methods when used with MLLM prompts (figure 11).
Figure 12 summarizes the database statistics for each subcategory, showcasing material segmentation availability and examples of articulated models with interactive components.
(a) Representative image stitching
(b) Sets of Mask
Figure Image stitching for materials image prompts MLLMs. We tested that image stitching provides more effective visual prompts for MILMs compared to the commonly used SOM approach.
Number models per Asset Type
ThreeBeaBora
Cheethar
585
DeirgChai
highntaad
100
TeaTablc
Two seat sola
KrgaizeBed
Wardrobw
404
ComputerCair
Latapeetione
2614
Bantee
Wardrobe
Shelf
Lounge Chair
EngleBed
1225
LucySulia
DraverChaal
DenngTisle
Desk
Dining Chair
Double
TromestBeia
Ricinggtai
Cameriddeleble
13
Coffeelasee
Doo
Single Bed
Three seat sofa
TV Stands
Windows
Ratigerato
Feepleax
Telona
Sent
T:ko1
Ovn
Weahherrryrr
8,m
(a) Example 3D models from the LiteReality Databuse. Each object includes material segmentation, as color-coded on the right (top). Some models also feature articulated components, enabling interactive settings (bottom).
(b) Models counts per asset categorics, constructed in the combination of 3D. Future, and A1-2Thor,
Figure 12: Literality Database: Statistics and Examples
Benchmarking Details
C.1 Retrieval Similarity Evaluation Protocol
Previous work on CAD model retrieval and alignment for 3D scenes-such as Scan2CAD [3]. Vid2CAD [39], and FastCAD [27]-ha primarily focused alignment accuracy. These methods

---

typically follow the Scan2CAD evaluation protocol, which considers retrieval successful if the retrieved CAD model belongs the correct object category, and then evaluates performance based on alignment precision.
In contrast, pipeline does not perform explieit alignment. Instead, rely off-the-shelf detector to predict 3D oriented bounding boxes for objects, making the Scan2CAD alignment-based protocol inapplicable our setting.
To evaluate retrieval similarity, we adopt more rigorous and fine-grained metrics proposed recent literature. Works such DifiCAD [16], HOC-Search [2]. ScanAnnotate [1], Accurate Instance-Level CAD Model Retrieval [55], and FastCAD [27] introduce similarity- -baed metrics that go beyond categorical correctness. Based on comparisons across these approaches, we adopt the L1 Chamfer Distance as our primary metric, measuring the geometric difference between the normalized retrieved CAD model and the normalized ground-truth model (as annotated in Scan2CAD). Our use of this metric consistent with prior works [2, 16, 55].
Unlike the Scan2CAD protocol, which limits retrieval to predefined set of CAD candidates, perform retrieval over the entire ShapeNet category corresponding to each ground-truth object. The number of CAD models per category used in our evaluation is listed in Table 4.
Table 4: Number of CAD model templates in the ShapeNet database for retrieval
Trash Can
Bathtub Bookshelf
Cabinet
Chair
Display
Sofa
Table
343
477
452
1571
6778
1093
3173
8436
Although some earlier works report similarity evaluations, direct comparison remains difficult due to missing details or unavailable implementations. To support reproducibility, we provide comprehensive description of evaluation methodology.
Representative Image Selection
Images of CAD models play key role in retrieval pipelines that involve 2D feature extraction. silhouette matching, or depth comparisons. To select representative views for each CAD object, we reproject all 3D points of scene instance into the image space and choose the top four images with the highest number of visible points i.e. the least occluded views. For ShapeNet CAD models, we use the pre-rendered milti-view dataset from DISN [58], which contains 32 rendered images per model from different angles.
Chamfer Distance Computation
To compute Chamfer Distance, each CAD model is centeeed at the origin and scaled to fit within unit cube (1x 1 1). We uniformly sample 10,000 surface points from both the retrieved and ground-truth CAD models. The L1 Chamfer Distance is calculated bidirectionally as follows:
(L,i(A,
Σ
min ||a
bll +
min | aed
a/l2
[A
BEB
|
aaA
The following three baselines are selected for comparison.
SCANnotate [1]: Utilizes an automatic CAD retrieval pipeline that leverages depth maps, silhouette comparison, and Chamfer Distance via differentiable rendering.
Digital Cousin [9]: Matches CAD models using 2D image features from multiple views to rank visually and semantically similar assets.
MSCD [55]: Employs a two-step approach: initial retrieval using 3D feature descriptors, followed by geometry-based re-ranking using a Modified Single-direction Chamfer Distance (MSCD), which improves robustness to noise and better captures shape similarity in large databases.
We conduct the evaluation all scenes from the ScanNet[8] validation set.
25

---

C.2 Object-Centric Material Recovery Benchmark
To evaluate object-centric material recovery in realistic conditions, we introduce a benchmark designed to reflect the challenges of real-world RGBD capture. Unlike global scene-level material estimation, our focus on object-level recovery aligns naturally with the object decomposition pipeline used in graphics, and enables per-object physically based rendering (PBR) material assignment- essential representation for high-fidelity simulation and interactive graphics applications.
(a)
(b)
(c)
(d)
(e)
Figure 13: Challenging real-world conditions, including occlusion, poor lighting, skewed angles, and numerous distractors.
We constructed this benchmark using five indoor scenes captured with an iPhone 13 Pro Max via the 3D Scanner App RoomPlan mode. These scenes cover range real-world environments, including meeting room, common area, bedroom, and two study rooms, and contain total of 111 manually curated objects spanning eight categories.
For each object, we identified and cropped the four most visually representative frames from the RGBD scan. These images often reflect real-world challenges such occlusion, suboptimal lighting, and partial visibilty-coondition under which we aim to test the robustness of various material recovery approaches. Some exampel crops are list in Figure 13. To benchmark performance, compare our material painting pipeline against four baselines:
1. PhotoShape [46] material selection method using appearance- appearance-based retrieval;
2. Make-It-Real [13] inguage-guided material retrieval pipeline;
3. Our Visual+Language Search combining visual similarity and textual prompts;
4. Make-It-Real + Albedo-Only Optimization variant incorporating our optimization stage.
Ground-truth CAD models are provided for all scenes. Estimated PBR materials from each method are applied to the CAD models and rendered at their original 3D poses using global illumination with HDR environment maps. For evaluation, we compare the rendered images of cropped objects against the corresponding cropped RGB images using perceptual metries, as shown in Figure 14. To ensure fair evaluation, manually selected retrieval results and object categories, avoiding retrieval-induce biases. Note that while we use known object poses for rendering and evaluation, used during material recovery preserve generalization to in-the-will scenarios. We will release the scanned scenes, selected objects, cropped image sets, and evaluation scripts for reproducibility and support future research in object-centric material recovery.
these poses
CartrrC Materal Paintig
Object Camera pose
Percoptual difference (RMSE, SSIM, LPIPS)
Reference images
Hondippd pick mode
Images
Prediction
Evaluation
Figure 14: Evaluation protocol used for assessing object-centric material painting performance.
26

---

C.3 Graphics-Ready Reconstruction Benchmark
use five indoor scenes captured using an iPhone 13 Pro Max with the RoomPlan app, each containing hundreds of RGB frames. These scenes were also used in the Object-Centric Material Re- covery benchmark. However, this benchmark evaluates a fully automated, end-to-end reconstruction pipeline-frowe raw imput scans to final graphics-reate scene. Lighting is not explicitly estimated; instead, we apply general environmental light to provide consistent global illumination.
To measure visual quality, we adopt a perceptual similarity metric. Rendered images from the reconstructed scene-viewed under the original camera poses are compared to real RGB captures.
We evaluate four reconstruction pipelines:
Phone2Proc [11]- - procedural reconstruction pipeline for embodied AI, operating at the room level with less emphasis on fidelity;
2. Digital Cousin [ integrated with our procedural reconstruction pipeline, but using the Digital Cousin flow for retrieval;
3. Digital Cousin Make-It-Real (MIR) enhanced with language-based material selection;
4. Digital Cousin Ours Semanii-Smmaticc- proposed material selection method using combined vision-language guidance.
D More on Methodologies
D.1 Scene Parsing Stage
In this section, we describe the scene-parsing stage, which utilizes a scene graph to structure spatial relationships. The input includes detected wall structures and oriented object bounding boxes.
This is rule-based approach, implemented five key steps:
Step 1: Wall Closure. We start by taking the raw 2D wall segments and snapping any nearly matching endpoints together using KD-Tree with small distance threshold to close tiny gaps. We then build a simple graph in which each snapped point is a node and each wall segment is an edge. We traverse this graph find closed loop; if no loop exists, we connect the two loose ends and try again. The final loop becomes our room-boundary polygon.
Step Wall-Object Alignment. Next, we examine each object's 2D corners and orientation to determine which wall segment it belongs to. We select the closest wall (within 0.2 m) whose orientation is within 10 of the object's facing direction. We then "snap" the object by moving its anchor point to the nearest point on that wall (leaving small clearance) and rotating it to align its front face with the wall.
Step 3: In-Room Adjustment. To ensure no object sits outside the room, we test every object corner against the boundary polygon. Any object that falls outside triggers local expansion of the offending wall segments and reconnection into single loop. We repeat this until all objects are safely inside.
Step 4: Scene Graph Construction. At this stage, we compute wall connectivity and wall-object relationships. We then identify object-object relationships: if two objects' projections onto the XY plane overlap and their Z. extents indicate a clear top-bottom arrangement, assign an "on-top" link; if two objects share the same orientation and lie within I m horizontally, we assign a "next-to" link.
Step 5: Collision Resolution. Finally, we resolve any remaining overlaps by treating each object footprint as a 2D polygon. Wall-attached objects are constrained slide along their wall: others move freely. We iterate to ten times, find every pair of overlapping objects, compute the minimal translation vector push them apart (projecting along the wall if constrained), and move them until no collisions remain, or reach the maximum iteration.
We demensarate the results of scene parsing quantitiviely in Figure 17
27

---

Input
will provide an lsage copposed of four sub- images arranged
tounding hightighting different vieepcints Carefully ly the all four lmages to urderstand onject's snape, rppearance, decermine whith category free satabase highlighted object west likely belongs to, Despet only catejory typets) explatations. following is the abj category structure in 3508 dictionary format:
Spatol Info
Object Node
Appeatiance
Top-4 Mart vizie Imagee
Asset Database
"Lounge_Cheir Chair_ wttrr_ffiie__hheee Cheir""
Step 1. Semantic Filtering
Assets
inder the class
CoS- Siityity
DINOv2
mestve the tmagges
Crop DINON2 ranking
lac Classesi
Step Image-Based Retrieval
Spatie Info
Crop & ZAOOIZ ranking
Posttioing Learned Feay/iclegheattion
Pose Aewere Rendering
Top-4
Top-2a Selected Classes
Step
RaI sections:
object with bounding boces, and the right contains
Step 4
tamplates (1-4),
repainted, Feces on the dbject's shape, structure, and style. Compare these trasts aligmment aen-syynttii objects. Select the best-ratching tomplate and 3,
Step 4, contextual selection
Output
Figure 15: Detailed Breakdown of Procedural Generation The input data includes object nodes with spatial, aesthetic, and semantic information, which are utilized different stages of the pipeline.
D.2 Retrieval Stage
detailed breakdown of each step in the proposed retrieval approach is illustrated in Figure 15.
28

---

E More Results
In this section, we present extended results of LiteReality, focusing particularly on the quantitative evaluation for some scenes in ScanNet, as shown in Figure 10.
For the individual stages of our pipeline, first showease the scene parsing results in Figure 17.
Regarding object retrieval, we provide detailed analysis that includes both the reference images and the corresponding retrieved objects, as illustrated in Figure 18. We also highlight the effectiveness of our contextual selection module, and include some failure cases to demonstrate limitations.
For the material painting component, we present further comparisons between our proposed method and several baseline models, with results visualized in Figure 16. Also the detailed per categories results tested on the proposed benchmark in Table 5.
Reference crops
PhotoShape
Make-It-Real
Sem&Vis
Ours
Figure 16: Object-centric material painting results. Our method compared against several baseline models, demonstrating improved texture alignment and realism.
29

---

Before
After
Before
After
Object Out-of-Room
Automatic Removal
Noisy Wall Segments
Avold Object Flooting
Automatic Repairing of incomplete Walls
Ensure enclosed wall lines
Figure 17: Scene Parsing Enhancements in LiteReality. Visualization of several improvements in- troduced by our scene parsing module. Left and right panels show "Before" and "After" comparisons.
30

---

(n) with bounding
resutsfrom feature
(d] GPT final selection
box
GPT final selection
Non-ideal Selection
Figure 18: Demonstration of retrieval results using the proposed multi-step object retrieval pipeline. The image on the left displays the top four visible images from dataset. The middle images show the top four ranked results from the visual selection process after Step Step 2. The right images present the results proposed by GPT-4o. GPT-4o leverages more contextual information, often making more reasonable selections compared to purely visual-based methods. However, failure cases can still occur, as shown in the bottom image, where unreasonable 3D model selection hinders the realism of LiteReality. With advancements in reasoning within MLLMs, anticipate further improvements in such pipelines in performance as MLLMS continue to evolve.
31

---

(a) Chair, Fireplace, Sofa, and Storage
Method
Chair
Fireplace
Sofa
Storage
RMSE
SSIM
LPIPS
RMSE
SSIM
LPIPS
RMSE
SSIM
LPIPS
RMSE
SSIM
LPIPS
PhotoShape
0.309
0.255
0.644
0.351
0.171
0.731
0.360
0.404
0.704
0.334
0.176
0.712
MIR+AO
0.227
0.403
0.582
0.285
0.403
0.677
0.270
0.489
0.639
0.218
0.389
0.609
MIR
0.242
0.384
0.604
0.291
0.394
0.668
0.263
0.488
0.645
0.238
0.365
0.629
Sim&
0.263
0.372
0.627
0.299
0.449
0.665
0.266
0.475
0.685
0.287
0.339
0.652
LiteReality
0.228
0.407
0.578
0.283
0.433
0.688
0.270
0.486
0.634
0.230
0.395
0.621
(b) Table, Television, Door, and Windows
Method
Table
Television
Door
Windows
RMSE
SSIM
LPIPS
RMSE
SSIM
LPIPS
RMSE
SSIM
LPIPS
RMSE
SSIM
LPIPS
PhotoShape
0.401
0.304
0.640
0.351
0.242
0.652
0.259
0.311
0.658
0.441
0.215
0.750
MIR+AO
0.250
0.467
0.576
0.219
0.639
0.537
0.214
0.507
0.590
0.289
0.323
0.692
MIR
0.256
0.448
0.600
0.218
0.632
0.543
0.207
0.503
0.603
0.332
0.285
0.689
Sem& Vis
0.344
0.391
0.626
0.261
0.613
0.618
0.234
0.519
0.619
0.482
0.193
0.747
LiteReality
0.248
0.477
0.573
0.220
0.636
0.531
0.212
0.546
0,581
0.286
0.361
0.689
Table 5: Per-category evaluation metrics (RMSE J. SSIM 1, LPIPS J) for different methods across
representative object types. Best results are in bold.
32

---

Application of LiteReality
In this section, we showcase how scenes reconstructed with LiteReatlyy can be used for wide range of application, benifits from its graphic realted featuers. The Demensration includes:
Reconstructed scenes with physically based rendering (PBR) materials support dynamic relighting under various conditions, as demonstrated in Figure 19. This adaptability enhances rendering fidelity and enables realistic, immersive experiences.
Relighting:
Indoor lighting
TV Lighting
Moringg lighting
Dawn lighting
RGB Scan
TV lighting
Indoor Lighting
Morning Lighting
This figure illustrates the versatility of procedural reconstructions enhanced with PBR materials under various lighting conditions. The top row showcases the reconstructed scene rendered under different lighting setups: standard indoor lighting, TV lighting where the television acts as the sole light source, morning lighting with natural daylight streaming through the window, and dawn lighting that mimics the soft glow of early sunrise. The bottom row provides a comparison to the original RGBD scans used as inputs. This comparison demonstrates the effectiveness of the pipeline in achieving realistic scene appearance. The ability to seamlessly re-light scenes post-reconstruction highlights the utility of this approach in diverse applications, such virtual reality, architectural visualization, and einematic production, where dynamic lighting enhances realism and usability. [Scene is reconstructed from ScanNet scene0166]
Figure 19: Lighting Effects on Procedural Reconstructions:
Physics- Maasic Interactions: The pipeline enables realistic rigid-body physics and collision detection, allowing natural interactions such objects falling colliding. Figure 20 illustrates these capabilities with examples like falling apple and dynamic indoor collisions.
Scene Editing: Modular reconstruction allows users to dynamically replace, modify, or add objects. Integrated physics ensures realistic placement and interaction, simplifying tasks such as adding messy, lifelike elements to scene (Figure 21).
Applications in Roboties, Gaming, and VR: The structured and interactive nature of reconstructed scenes facilitates seamless integration into robotics, gaming, and VR environments. Figure 21 showcases examples that demonstrate the pipeline's potential for creating realistic digital replicas of real-world spaces.
33

---

Free Osced Inlmuters
Rem-Reedy
Guanly Stadadk
Physics Interaction
Figure 20: Physics-Basee Interaction in Procedural Reconstructions: This figure demonstrates the application physics simulations to procedurally reconstructed scenes, enabling realistic interactions between objects. The left panel highlights two examples: (1) a gravity simulation where numerous apples are dropped into the scene, naturally landing on tables, scattering, and interacting with the environment; and (2) toy car driving at high speed, colliding with furniture, showcasing rigid body dynamics individual objects respond realistically impacts and forces. The right panel provides comparison of the original RGB images, reconstructed geometry, and final simulation-ready scenes, emphasizing how the reconstructed objects exhibit realistic physical behaviour. This integration of physics-based interactions enhances the realism and utility of reconstructed scenes, making them suitable for applications in simulation environments, gaming, and Srboticss[Scemt is reconstructed from ScanNet scene0291]
Editing with Physics
Robotic Simulation
Virtual Reality Roconstructed Scene Figure 21: Demonstration of applications enabled by graphics-ready reconstructed scenes, showcasing advanced features such as physics-based interactions, robotic training, gaming environments, and immersive virtual reality experiences. These examples highlight the versatility and interactivity of the reconstructed scenes for various use cases. (Scene is reconstructed from ScanNet Scene028/)
Gaming Reconstructed Scene
34


Reviews:
[{"review_id": "J8Gz5bR32A", "reviewer": "Reviewer_jpwk", "summary": "The paper tackles the task of converting RGBD indoor scans to 3D scenes for graphics rendering. It uses a multi-stage pipeline that first parses the scene into a structured scene graph, then performs 3D asset retrieval and material painting. The method achieves better quantitative performance on real-world scans than baselines.", "questions": "* What is the system's behavior when the retrieved asset departs far from the real object due to a retrieval error? How do the material painting and procedural reconstruction stages handle such discrepancies?", "limitations": "Yes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 1, "strengths_and_weaknesses": "Strengths\n* The paper presents a straightforward but well-motivated training-free pipeline to tackle an important task, and shows good empirical performance. \n\nWeaknesses\n* Errors may propagate across different stages in the pipeline. It would be informative for readers if in-depth failure analyses were presented. \n* If I'm not missing it, the computation cost is promised in the checklist but not included in the supplementary. \n* Some modules are added to the system to increase robustness, e.g., with VLM visual queries and renderings for analysis-by-synthesis for pose fitting. These modules should be thoroughly ablated and analyzed.", "was_revised": true, "final_justification": "The rebuttal addresses most of my concerns. I agree with reviewer meLw that the proposed pipeline is engineering-heavy and the technical contribution over prior works is not significant. I can see it being practically useful, though, if as the authors argued, there is no prior work that works on the exact same setting. I raised my score as the rebuttal experiments resolved my concerns in the original review, but I'm open to hearing other reviewers' opinions.", "rebuttal": "We thank the reviewer for the thoughtful feedback and for acknowledging the importance of our task. Indeed, LiteReality is, to the best of our knowledge, the **first** method that reliably reconstructs indoor scenes into **compact CAD representations** with **realistic PBR materials** and **parsed layouts**. We respond to the weaknesses and questions as follows.\n\n**Weakness1: \"Errors may propagate across different stages in the pipeline. It would be informative for readers if in-depth failure analyses were presented.\"**\n\nThe error propagation is a natural challenge in multi-stage methods. To mitigate this, LiteReality is explicitly designed to achieve strong performance at each individual stage—demonstrated by our state-of-the-art retrieval similarity (Table 1) and best performance in object-centric PBR material estimation (Table 2, Figure 6, 7). Overall robustness is evidenced by high-quality reconstructions on both the ScanNet dataset and diverse real-world scenes (Table 3, Main Figure 5, Supp. Figure 1).\n\nTo further analyze failure modes, we conducted a detailed user study based on four real-world scans captured using an iPhone—without any manual intervention. These reconstructions represent typical, challenging indoor environments. We evaluate each stage of the pipeline independently by asking users to assess the quality of outputs at each step.\n\nFor each stage, users rated the results using the following scale:\n- Good: A faithful and realistic representation of the real-world object or layout.\n- Sufficient: Not an exact match, but still plausible in context (e.g., retrieving a three-door wardrobe for a two-door one that maintains overall scene realism).\n- Bad: Visibly inaccurate or implausible, breaking scene realism and requiring modification for acceptable results.\n\n**1. Scene Understanding Results**\n\n| Scene | Total | TP | FP | FN |\n|-|-|-|-|-|\n| Meeting Room | 21    | 21 | 0  | 1  |\n| Boardroom    | 48    | 48 | 1  | 0  |\n| Common Room  | 20    | 20 | 0  | 0  |\n| Study Room   | 12    | 12 | 0  | 0  |\n\nFailure cases include the window-side embedded seating area in the Boardroom, which was falsely detected as a sofa, and a chair in the Meeting Room that was not detected.\n\n**2. Retrieval Quality**\n\n| **Scene**      | **Total Objects** | **Good (%)** | **Sufficient (%)** | **Bad (%)** |\n|----------------|-------------------|--------------|---------------------|-------------|\n| Meeting Room   | 21                | 12 (57%)     | 7 (33%)             | 2 (10%)     |\n| Boardroom      | 48                | 40 (83%)     | 6 (13%)             | 2 (4%)      |\n| Common Room    | 20                | 16 (80%)     | 2 (10%)             | 2 (10%)     |\n| Study Room     | 12                | 6 (50%)      | 5 (42%)             | 1 (8%)      |\n| **Total**      | **101**           | **74 (73%)** | **20 (20%)**        | **7 (7%)**  |\n\nTypical retrieval errors include cases such as a round table being retrieved as a square table, and a glass-door cabinet retrieved as a glass shelf. Additional failure examples are shown in Supplementary Figure 9. \n\n**3. Material Quality**\n\n| Scene        | Total | Good (%)   | Sufficient (%) | Bad (%) |\n|--------------|-------|------------|----------------|---------|\n| Meeting Room | 21    | 19 (90%)   | 2 (10%)        | 0 (0%)  |\n| Boardroom    | 48    | 43 (90%)   | 4 (8%)         | 1 (2%)  |\n| Common Room  | 20    | 18 (90%)   | 2 (10%)        | 0 (0%)  |\n| Study Room   | 12    | 9 (75%)    | 2 (17%)        | 1 (8%)  |\n| **Total**    | 101   | 89 (88%)   | 10 (10%)       | 2 (2%)  |\n\nOverall, the material painting is robust. Occasional failures occur due to misidentified albedo colors or mismatches between segmentation and crop regions.\n\n**4. Combined Quality**\n\n| Scene        | Total | Good (%)   | Sufficient (%) | Bad (%) |\n|--------------|-------|------------|----------------|---------|\n| Meeting Room | 21    | 8 (38%)    | 12 (57%)       | 1 (5%)  |\n| Boardroom    | 48    | 39 (81%)   | 6 (13%)        | 3 (6%)  |\n| Common Room  | 20    | 11 (55%)   | 7 (35%)        | 2 (10%) |\n| Study Room   | 12    | 7 (58%)    | 4 (33%)        | 1 (8%)  |\n| **Total**    | 101   | 65 (64%)   | 29 (29%)       | 7 (7%)  |\n\nThe combined quality reflects a few compounded errors, where imperfections in retrieval or material painting affect the final outcome. Nonetheless, the system remains robust across a diverse set of real-world scans. [see more in the video demo]\n\n\n**W2: the computation cost is promised in the checklist but not included in the supplementary**\n\nOur method runs on a single NVIDIA RTX 3090 GPU. Below is the runtime breakdown per scene:\n\n| Scene        | Preprocessing + Scene Parsing | Object Retrieval | Material Painting | Procedural Reconstruction |\n|--------------|-------------------------------|------------------|-------------------|----------------------------|\n| Boardroom    | 3m30s                          | 4m30s            | 51m               | 1m10s                      |\n| Common Room  | 1m20s                          | 3m39s            | 19m               | 48s                        |\n| Study Room   | 1m50s                          | 2m20s            | 14m               | 45s                        |\n| Meeting Room | 1m20s                          | 3m39s            | 19m               | 48s                        |\n\nThese runtimes are measured on RTX 3090 GPU. We can further speed up the run time by increasing parallelization, especially during the material prediction stage.\n\n**W3: These modules should be thoroughly ablated and analyzed.**\n\nBelow, we provide detailed ablation studies for both the **retrieval pipeline** and the **material Painting** module. \n\n**Retrieval Pipeline Ablation**\n\n| ID  | Semantic | Image Feat. | Pose R&C | MLLM | Avg. Chamfer (↓) |\n|-----|----------|-------------|----------|------|------------------|\n| I   | ✓        | ✓           |          |      | 0.142            |\n| II  | ✓        | ✓           | ✓        |      | 0.110            |\n| III | ✓        | ✓           |          | ✓    | 0.112            |\n| IV  | ✓        |             | ✓        |      | 0.104            |\n| V   | ✓        | ✓           | ✓        | ✓    | **0.097**        |\n\nSimilar to Table 1, these experiments are conducted on Scan2CAD datasets, meaning retrieval similiarty using average Chamfer Distance. Each added module contributes meaningfully to improving performance.\n\n**Material Recovery Ablation**\n\n| ID  | Language | Visual | Albedo-only | RMSE (↓) | SSIM (↑) | LPIPS (↓) |\n|-----|----------|--------|-------------|-----------|-----------|------------|\n| I   | ✓        |        |             | 0.241     | 0.397     | 0.613      |\n| II  |          | ✓      |             | 0.322     | 0.239     | 0.658      |\n| III | ✓        | ✓      |             | 0.286     | 0.378     | 0.637      |\n| IV  | ✓        | ✓      | ✓           | **0.218** | **0.436** | **0.584**  |\n\nPrevious methods often struggle to recover realistic materials under occlusion or imperfect lighting. In contrast, each component in our design contributes meaningfully to material quality, as evidenced by consistent improvements across RMSE, SSIM, and LPIPS metrics.\n\n\n**Q: What is the system's behavior when the retrieved asset departs far from the real object due to a retrieval error? How do material painting and reconstruction handle discrepancies?**\n\nWe rigorously benchmarked on the Scan2CAD dataset and achieved SOTA results on retriveal. This shows that the likelihood of a poor retrieval is lower in our approach compared to existing methods. In the rare case where a retrieval error does occur, the system remains robust through the following mechanisms:\n1. Robust Material Painting:\nThe material recovery stage is **specifically designed to handle geometric misalignments** (Figure 6). Even with significant discrepancies between the retrieved object’s shape and the scanned image, LiteReality is able to transfer realistic appearance with PBR materials. \n2. Physically Plausible Placement:\nThe reconstruction pipeline ensures that all objects are still placed in physically plausible locations and poses, maintaining global scene consistency despite local retrieval errors.\n\nIn cases of extreme mismatch, where the output may visibly diverge from reality, scene produced by our method are easily editable due to the decomposition of objects geometry and materails.\n\nThank you again for your thoughtful review and do let us know if you have further questions."}, {"review_id": "iTP3909icc", "reviewer": "Reviewer_meLw", "summary": "This paper presents a framework for reconstructing indoor scenes from RGB-D scans into articulated geometry with realistic materials, enabling both high-quality rendering and physical interactions. The pipeline comprises four stages: scene parsing, object reconstruction, material painting, and procedural reconstruction. Notably, it introduces a training-free object retrieval method and a robust material painting module, both achieving state-of-the-art performance.", "questions": "Question:\n1.\tIs it a fair comparison in Table 3 given that the proposed method uses RGB-D input but the methods like Phone2Proc and Digital Cousin use RGB input?\n2.\tWhat is the computation time of each stage, and compare to other baselines?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1.\tThe paper is well-written, with a clear presentation of the pipeline and clear visual demonstrations.\n2.\tThe proposed “graphics-ready” 3D reconstruction framework has broad application potential in AR/VR, gaming, and robotics.\n3.\tExperimental results show consistent improvements over prior works in retrieval accuracy, material recovery, and full-scene realism.\n\nWeaknesses:\n1.\tAlthough the authors claim to be the first to convert room-level scans into graphics-ready environments, prior works such as Digital Cousin [1] and Phone2Proc [2] address similar goals and concepts.\n2.\tMany components of the pipeline are built upon existing methods; the novelty lies primarily in system integration rather than in technical innovation. As such, the contribution is more engineering-driven than algorithmically novel.\n3.\tThe quantitative gains in Tables 2 and 3 are relatively small, and the absolute performance metrics for all methods remain low, raising concerns about the practical impact of the improvements. \n\n\n[1] Automated creation of digital cousins for robust policy learning. CoRL 2024\n[2] Phone2proc: Bringing robust robots into our chaotic world. CVPR 2023", "was_revised": true, "final_justification": "Most of my concerns have been addressed in the rebuttal. While I still share some reservations about the technical contribution, as noted by other reviewers, the paper shows a degree of originality in relation to prior work and potential applications. Therefore, I have decided to raise my score.", "rebuttal": "We thank the reviewer for the thoughtful and constructive feedback. We appreciate the reviewer highlighting the clarity of our paper, the strong visual and quantitative performance, and the broad applicability of the proposed method. Below, we address each point in detail.\n\n**Weaknesses 1: Although the authors claim to be the first to convert room-level scans into graphics-ready environments, prior works such as Digital Cousin [1] and Phone2Proc [2] address similar goals and concepts.**\n\nwe would like to clarify that neither method is designed to produce **realistic CAD representation with PBR materails from room-level scans**:\n\n- **Digital Cousin** works on a **single image** and aims to create simulatable scenes with objects semantically matched to reality. However, it does not support PBR material recovery from images and cannot robustly handle room-level reconstruction from video inputs. [Note that the Table 3 Digital Cousin baseline is built by replacing the retrieval step in our pipeline with theirs, while the rest of the pipeline uses LiteReality.]\n- **Phone2Proc** reconstructs room-level scenes from videos but lacks support for realistic PBR materials, as it is designed primarily for generating diverse simulation environments for robotic training.\n\n- In contrast, our goal is to reliably reconstruct indoor scans into graphics-ready 3D scenes (realistic CAD with PBR materials and parsed layout) at the room level—a task that neither Digital Cousin nor Phone2Proc is able to accomplish (detailed in the table below).\n\n| Method            | Input Type   | Semantic Matching for Retrieval | PBR Materials for Realism | Room-Level Reconstruction |\n|-------------------|--------------|---------------------------------|---------------------------|---------------------------|\n| **Digital Cousin**| single Image    | ✅ Yes                          | ❌ No                     | ❌ No                      |\n| **Phone2Proc**    | RGB-D Video  | ✅ Yes                          | ❌ No                     | ✅ Yes                    |\n| **PSDR-Room**     | single Image    | ✅ Yes                          | ✅ Yes                    | ❌ No                     |\n| **LiteReality**   | RGB-D Video  | ✅ Yes                          | ✅ Yes                    | ✅ Yes                    |\n\nA similar work worth mentioning is PSDR-Room, which shares a similar goal of producing realistic scenes. However, it only works on a single image and cannot be easily scaled to room-level reconstruction.\n\n**Weaknesses 2. Many components of the pipeline are built upon existing methods; the novelty lies primarily in system integration rather than in technical innovation. As such, the contribution is more engineering-driven than algorithmically novel.**\n\nThe goal of this work is to enable compact reconstruction from real-world scans at room level with with high quality geomtry mesh and relastic PBR materails—a practically important but previously unsolved problem. This motivation drives the design of LiteReality, which necessarily involves integration of multiple components.\n\nIt is important to highlight that simply combining existing modules does not lead to usable results, let alone robust ones. Prior to LiteReality, no method could take raw RGB-D scans and output realistic CAD representations of room-scale 3D scenes with PBR materials. The strength of our pipeline lies in how we redesign and invent components to handle real-world noise, occlusions, geometric misalignments, and lighting variations.\n\nAlso, for indivisual compoenent: \n- We introduce a scalable, robust PBR material recovery module that performs reliablely at scale—crucial for realistic room-scale reconstruction—and achieves state-of-the-art fidelity compared to existing methods with large margin [Table2M, Table3M].\n- We also achieve state-of-the-art retrieval performance using a training-free module that integrates semantic filtering and pose reasoning [Table1] with a substantial improvement (around 5%) over previous SOTA, and best performance on nealry very object categores.\n\nWe value the delivery of a complete, reliable pipeline more than isolated novelty in individual components, as it enables a new class of applications in AR/VR, data creation, and simulation—which we believe is ultimately highly impactful. \n\n**Weaknesses3. The quantitative gains in Tables 2 and 3 are relatively small, and the absolute performance metrics for all methods remain low, raising concerns about the practical impact of the improvements.**\n\nWe would like to highlight that numbers in perceptual quality metric for reconstruction using retrieved geometry and PBR material recovery typically falls substantially short of novel-view synthesis methods like NeRF and Gaussian Splatting. For example, **PSDR-room reports LPIPS ≈ 0.5–0.8 and SSIM ≈ 0.4–0.6—on par with our results—whereas NeRF-based approaches typically achieve LPIPS ≈ 0.1 and SSIM > 0.9**. Nevertheless, this remains an informative metric for demonstrating improvement, as commonly used in prior works. The qualitative results (Figures 5, 6, 7, Supp Figure 1, demo video) demonstrate the clear visual improvements. \n\nMeanwhile, during the rebuttal period, we conducted a user study asking participants to vote for the perferred PBR recovery method. Our proposed method showed a clear advantage in this study.\n\n| Method           | Boardroom       | Common Room     | MeetingRoom      | StudyRoom | Overall      |\n|------------------|----------------:|----------------:|----------------:|-----------------:|-------------:|\n| MIR + AO         | 62.4 %          | **61.8 %**      | 33.9 %          | 15.9 %           | 43.5 %       |\n| Sem & Vis        | 2.7 %           | 15.3 %          | 1.9 %           | 3.0 %            | 5.7 %        |\n| MIR              | 5.3 %           | 1.8 %           | 3.6 %           | 20.4 %           | 7.8 %        |\n| PhotoShape       | 7.2 %           | 3.4 %           | 4.2 %           | 5.1 %            | 5.0 %        |\n| **LiteReality**  | **81.4 %**      | 45.3 %          | **57.8 %**      | **57.2 %**       | **60.4 %**   |\n\n**Q1: 1. Is it a fair comparison in Table 3 given that the proposed method uses RGB-D input but the methods like Phone2Proc and Digital Cousin use RGB input?**\n\nPhone2Proc uses RGB-D videos as input, whereas Digital Cousin only supports RGB images and cannot perform room-level reconstruction. For more details, please see our response to Weaknesses 1.\n\n**Q2. What is the computation time of each stage, and compare to other baselines?**\n\nOur methods runs on a single NVIDIA RTX 3090 GPU. Below is the runtime breakdown per scene:\n\n| Scene        | Preprocessing + Scene Parsing | Object Retrieval | Material Painting | Procedural Reconstruction |\n|--------------|-------------------------------|------------------|-------------------|----------------------------|\n| Boardroom    | 3m30s                          | 4m30s            | 51m               | 1m10s                      |\n| Common Room  | 1m20s                          | 3m39s            | 19m               | 48s                        |\n| Study Room   | 1m50s                          | 2m20s            | 14m               | 45s                        |\n| Meeting Room | 1m20s                          | 3m39s            | 19m               | 48s                        |\n\nBaselines that skip material painting would take less time, but they lack PBR material recovery, a essential part for realistic reconstruction and downstream applications.\n\nThank you again for the thoughtful review."}, {"review_id": "h9ty3KNeen", "reviewer": "Reviewer_y37B", "summary": "This paper presents a pipeline for transforming RGB-D scans of indoor environments into detailed 3D virtual replicas. The approach begins with scene understanding, involving object detection and 3D bounding box prediction, followed by the construction of a scene graph to capture spatial relationships between scene elements. For each detected object, a corresponding artist-crafted 3D model is retrieved from a 3D asset database using a Multi-modal Large Language Model (MLLM) that leverages both visual and textual cues. The method then estimates materials and textures for each object using MLLM-based techniques, enabling realistic material painting. Finally, the entire scene is reconstructed in Blender for photorealistic rendering. The pipeline is evaluated across three tasks: object retrieval accuracy, material estimation quality, and procedural generation of the complete scene. This method outperforms the state-of-the-art methods in all three tasks.", "questions": "1. Dependency on Camera Pose Estimation:\nIt is unclear which camera pose estimation model is employed in the pipeline. Since accurate object placement relies heavily on correct pose information, it would be helpful to discuss how object retrieval behaves when the pose estimation is inaccurate. An ablation or sensitivity analysis demonstrating the effect of pose estimation errors on object retrieval accuracy would strengthen the paper.\n\n2. Scene Completeness:\nThe final reconstructed scenes appear to be missing several objects, which affects the perceived completeness and realism. A quantitative evaluation of scene completeness—such as the percentage of detected objects successfully reconstructed—would provide valuable insight into the robustness and practical usability of the method.\n\n3. Evaluation Metrics and User Study:\nWhile Chamfer Distance and image-based metrics like LPIPS and PSNR are informative, they may not fully capture perceptual quality or semantic accuracy. Including a user preference study would provide complementary qualitative insights, especially for evaluating object retrieval and material estimation results.", "limitations": "Discussed in the supplementary material.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1. The paper addresses an important research problem—generating virtual 3D replicas of indoor scenes—which has broad applications in areas such as AR/VR, robotics, and simulation.\n\n2. The paper is clearly written, and the detailed explanation of the proposed pipeline enhances reproducibility and facilitates future research.\n\nWeaknesses:\n\n 1. Limited Novelty: The proposed method primarily integrates existing components—object retrieval and object-centric material estimation—into a unified pipeline. While the combination is useful, the individual components are largely based on prior work, and the paper offers limited methodological innovation.\n\n 2. Insufficient Evaluation:\na) The paper lacks a comprehensive evaluation of the reconstructed scenes. For instance, in the object retrieval task, it does not analyze discrepancies in object scale or orientation compared to ground-truth assets (e.g., in Figure 5, second row, last example, the retrieved table differs noticeably in size).\nb) It is unclear whether the objects are aligned with ground-truth counterparts before computing Chamfer Distance. If alignment is performed, the metric may fail to reflect orientation mismatches, which are important for realistic reconstruction.", "was_revised": true, "final_justification": "Although the paper’s pipeline offers limited technical novelty, the problem setup—producing realistic CAD representations with PBR materials from raw RGB-D scans at the room level—is itself an important and non-trivial research direction. Conceiving such a setup from individual existing components is not straightforward, and therefore, I am inclined to recommend acceptance of the paper.", "rebuttal": "We thank the reviewer for the positive and constructive feedback. We are glad that the reviewer found our work technically solid and impactful for applications in AR/VR, robotics, and simulation. Below, we respond to the specific concerns raised:\n\n**W1: Limited methodological innovation—primarily an integration of existing modules**\n\nThe primary goal of this work is to **enable compact, graphics-ready 3D reconstruction from real-world scans at the room level—featuring high-quality geometry and realistic PBR materials**.  It is important to highlight that simply combining existing modules does not lead to usable results, let alone robust ones. The strength of our pipeline lies in how we redesign and invent new components to handle real-world noise, occlusions, geometric misalignments, and lighting variations. Specifically, \n- We introduce a scalable, robust, and self-contained PBR material recovery module that performs reliably even under poor lighting conditions, severe geometric misalignments, and heavy occlusions—challenges commonly encountered in real-world scans. Our method achieves state-of-the-art fidelity with a significant margin over baseline methods (Table 2 and Table 3).\n- We also achieve state-of-the-art retrieval similarity performance, rigorously benchmarked on the Scan2CAD dataset (Table 1), demonstrating substantial improvement in average Chamfer Distance per CAD over previous methods and achieving the best performance across nearly all object categories.\n\n\n\n**W2 a) Lack of object scale/orientation analysis in retrieval**\n\nIn the pipeline, the retrieval stage was conducted with off-the-shelf methods (here with Apple RoomPlan) to detect oriented bounding boxes. According to their report, RoomPlan offers 91% average precision / 90% recall at 3D IoU 30% for object detection. The error was introduced there. Since it is not part of our contribution, evaluation metrics are not proposed.That said, we agree that analyzing alignment and scale discrepancies (e.g., as done in Scan2CAD) would enhance the evaluation, and we plan to incorporate such analyses in future work.\n\n**W2 b) Alignment in Chamfer Distance evaluation**\nWe do not perform any alignment before computing Chamfer Distance. All 3D assets in our database are calibrated to have a canonical orientation (facing forward) and are positioned in the scene using the orientation provided by the 3D oriented Bbox. \n\n**Q1: Dependency on Camera Pose Estimation**\n\nCamera pose is obtained from off-the-shelf models used in the scene understanding stage. Pose information is primarily used at two points during the retrieval\n1.To project 3D bounding boxes into 2D images for object image cropping.\n2. In the Pose R&C step, to position objects within the scene and generate renderings for more effective candidate filtering.\n\n- For point 1, our implementation includes an optimization step that uses GroundingDINO to refine the back-projected 2D bounding boxes by aligning them with overlapping detection results from GroundingDINO. This yields a highly robust cropping scheme. As a result, the effect of camera pose errors for this is significantly mitigated.\n- For point 2, we showcase ablation results below comparing retrieval similarity with and without the Pose R&C stage. A general drop in retrieval similarity is observed, but the system still performs well. This ablation is conduct with similar manner as Table 1. \n\n| ID  | Semantic | Image Feat. | Pose R&C | MLLM | Avg. Chamfer (↓) |\n|-----|----------|-------------|----------|------|------------------|\n| I   | ✓        | ✓           |          |      | 0.142            |\n| II  | ✓        | ✓           | ✓        |      | 0.110            |\n| III | ✓        | ✓           |          | ✓    | 0.112            |\n| IV  | ✓        |             | ✓        |      | 0.104            |\n| V   | ✓        | ✓           | ✓        | ✓    | **0.097**        |\n\n**Q2: Scene Completeness Evaluation**\n\nThe completeness of the reconstructed scene is limited by the capabilities of the scene understanding model. Our system uses Apple RoomPlan, which currently supports 17 object categories (i.e. Bed, Sofa, Chair, Table, Desk, TV, TV Stand, Sink, Bathtub, Toilet, Refrigerator, Stove, Kitchen Counter, Cabinet, Dresser, Nightstand, Bookshelf, Wardrobe, and Fireplace). However, it does not support small objects or wall-mounted items such as lamps or picture frames, which introduces incompleteness in the final scene. \n\nFor the supported categories, below is a Detection Quality Table evaluated across four real-life scans. Most objects are accurately detected, with one notable false positive—where a seating area near a window was misclassified as a sofa—and one false negative involving a missed chair in a meeting room. \n\nIn the future, we plan to extend object coverage to include smaller and wall-mounted items. We also recognize the value of a completeness metric for evaluation, which can be supported by providing manual annotations of the objects in real world scans\n\n| Scene | Total | TP | FP | FN |\n|-|-|-|-|-|\n| Meeting Room | 21    | 21 | 0  | 1  |\n| Boardroom    | 48    | 48 | 1  | 0  |\n| Common Room  | 20    | 20 | 0  | 0  |\n| Study Room   | 12    | 12 | 0  | 0  |\n\n**Q 3 : Evaluation Metrics and User Study**\n\nTo evaluate both retrieval and material estimation, we conducted a small-scale user study on four real-world scenes reconstructed entirely without human intervention. Due to time constraints, the current user study is limited in scale. We plan to conduct a more comprehensive user study after the rebuttal phase.\n\nStudy Setup:\nSix Participants (graphics researchers, engineers, architects, and 3D vision researchers) were shown:\n- A reference image of a real-world object.\n- Outputs from different methods (randomized order), including:\n- Retrieval: LiteReality, Phone2Proc, and Digital Cousin.\n- Material: All methods in Table 2.\nParticipants selected up to two preferred results per object. The results show that LiteReality was favored in the user study for both the retrieval and material painting stages,\n\nFor Retrieval\n | Scene                   | LiteReality        | DC                 | Phone2Proc          |\n|-------------------------|-------------------:|-------------------:|--------------------:|\n| **Boardroom**           | 31 (46.3 %)        | 30 (44.8 %)        | 6 (9.0 %)           | \n| **Common Room**  | 10 (47.6 %)        | 7 (33.3 %)         | 4 (19.0 %)          |\n| **Study Room**            | 7 (46.7 %)         | 7 (46.7 %)         | 1 (6.7 %)           |\n| **Meeting Room**                 | 13 (61.9 %)        | 6 (28.6 %)         | 2 (9.5 %)           | \n| **Total**               | **61 (49.2 %)**    | 50 (40.3 %)    | 13 (10.5 %)     |\n\nFor Materail Painting\n| Scene                   | LiteReality | MIR + AO | Sem&Vis | MIR   | PhotoShape |\n|-------------------------|----------------------:|----------:|----------------:|------:|-----------:|\n| **Boardroom**           | **81.4 %**                | 62.4 %    | 2.7 %           | 5.3 % | 7.2 %      |\n| **Common Room**  | 45.3 %                | **61.8 %**    | 15.3 %          | 1.8 % | 3.4 %      |\n| **Meeting Room**          | **57.8 %**                | 33.9 %    | 1.9 %           | 3.6 % | 4.2 %      |\n| **Study Room**    | **57.2 %**                | 15.9 %    | 3.0 %           | 20.4 %| 5.1 %      |\n\nThank you again for the thoughtful review and supporting our work."}, {"review_id": "MFYqKtbRmy", "reviewer": "Reviewer_5sYV", "summary": "This paper proposes a framework that reconstructs digital scenes from RGB-D scans at the “graphics-ready” level. The “graphics-ready” scene provides control of each object, such as furniture and building components. To this end, it proposes a reconstruction pipeline that constructs digital components from the outside in, utilizing prebuilt modules. The fidelity of each object has been enhanced by estimating its material parameters from real photos. The final result is evaluated for three criteria: retrieval similarity, material estimation, and full scene reconstruction. The image metric and geometric difference metrics are used for these criteria.", "questions": "I have the following questions:\n\n- What types of metrics should be used to evaluate the proposed method's performance adequately? Does the proposed benchmark thoroughly measure the perceptual (meaningful) performance of methods? \n\n- Are the proposed criteria commonly agreed upon by designers or related industry workers? \n\n- Is each step evaluated individually, and does this paper conduct an ablation study for processing steps?   \n\n- Is measuring photorealism for a method that doesn't estimate lighting adequate?", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 1, "strengths_and_weaknesses": "This paper has the strength that it explicitly introduces and addresses the ultimate goal of the digital twin. The proposed method looks solid and produces a plausible digitization result. \n\nHowever, there are the following concerns that make me hesitate to support this paper;\n\nFirst, the topic addressed in this paper is too broad to be covered within the confines of a conference paper. Each step in the pipeline could become the topic of a single paper. This paper should be submitted to graphics journals or HCI journals for completion of this study.\n\nAlthough the introduction excites me, the method part yields a weak technical novelty, as a multi-modal large language model processes most steps.\n\nThe construction of this paper evokes the following question: “Does this paper provide the scientific perspective for the pursued direction?” In this perspective, the proposed evaluation could be significantly improved and theoretically grounded. The current method naively measures the image difference or geometric difference for final evaluation, but these metrics don’t tell us how users perceive the usability of the proposed method. Additionally, the current image metric implicitly evaluates the geometric alignment, texture similarity, and geometric similarity simultaneously, making it challenging to identify which part is problematic. \n\nAlthough the light is not estimated, the proposed method measures the difference between the rendered image and the ground truth. This means light estimation error distorts albedo estimation. This should be clarified in the paper.", "was_revised": true, "final_justification": "After discussing with the authors and reading other reviewers’ discussions, most of my concerns have been resolved, so I am leaning toward accepting this paper. I appreciate the authors’ thorough and logical replies. This work will demonstrate potential for other downstream tasks.", "rebuttal": "We thank the reviewer for the detailed and thoughtful review. We appreciate that the reviewer finds the proposed method solid, acknowledges the quality of the reconstruction results, and recognizes our approach as targeting the ambitious goal of building graphics-ready digital twins. We address the concerns and questions below.\n\n\n**Concern 1: First, the topic addressed in this paper is too broad to be covered within the confines of a conference paper. Each step in the pipeline could become the topic of a single paper. This paper should be submitted to graphics journals or HCI journals for completion of this study.**\n\nOur method specifically targets the problem of **reconstructing indoor 3D scenes into clean, compact, and realistic CAD-like representations with high-quality mesh models and PBR materials, while remaining scalable**. This task is central to spatial AI and has broad applications in data generation, robotics, AR/VR, and simulation. Prior works such as Scan2CAD and SceneCAD have tackled similar end-to-end goals, demonstrating the relevance and suitability of this topic for the conference format. Additionally, recent papers with related system-level designs—such as ProcTHOR (NeurIPS 2022 Outstanding Paper) and Architect (NeurIPS 2024)—have also been published at NeurIPS, further supporting the appropriateness of our submission for this venue.\n\n**Concern 2: Although the introduction excites me, the method part yields a weak technical novelty, as a multi-modal large language model processes most steps.**\n\nWe appreciate the reviewer’s interest in the introduction and would like to clarify that LiteReality is not simply a composition of LLM-based modules. In fact, several key components of our system introduce non-trivial technical contributions beyond the use of multi-modal LLMs:\n- Scene Parsing introduces a scene graph–guided layout optimization method to automatically generate physically plausible and well-structured room layouts. This step is critical for realism and usability, yet has been largely overlooked in prior work (see Supplementary Figure 8).\n- The hierarchical object retrieval module, which incorporates pose-aware rendering and comparison, is designed to robustly handle real-world noise and variation. It achieves state-of-the-art results on the Scan2CAD retrieval similarity benchmark (Table 1).\n- The albedo-only optimization scheme offers a novel and practical alternative to full lighting estimation for PBR material recovery. This approach improves scalability while remaining robust under varying illumination (see Figures 6 and 7).\n\nTo our knowledge, LiteReality is the first system that produces realistic CAD representations with PBR materials from raw RGB-D scans, addressing several challenges such as occlusion and lighting variation in an integrated and scalable way\nThe strength of our pipeline lies in how we redesign and invent components to handle real-world noise, occlusions, geometric misalignments, and lighting variations.\n\n**Concern 3: Does this paper provide a scientific perspective for the pursued direction?**\n\n- LiteReality is, to the best of our knowledge, the first method that reliably reconstructs indoor scenes into compact CAD representations with realistic PBR materials and parsed layouts.\n- We also build three rigorous benchmarks to evaluate retrieval similarity, object-centric material recovery, and overall realism, supporting the development of this research direction. [Supp Section C]\n- We provide a curated database containing manually cleaned and polished material segmentation for thousands of indoor furniture items, supporting retrieval and material-painting–based object reconstruction. [Supp Section B]\n\n\n**Concern 4: The proposed evaluation could be improved and theoretically grounded. Current metrics naively measure image and geometric differences and don’t capture user perceptions of usability.**\n\nOur evaluation metrics follow a well-established line of work in CAD retrieval and appearance modeling—we simply adopt the established metrics used in prior work. Methods such as Scan2CAD, DiffCAD, and MSCD all employ same retriveal silmirity metrics. For appearance modeling, perceptual accuracy metrics are widely used in PSDR-Room, PhotoScene, and other works that aim to recover appearance from images. While we acknowledge that no metric is perfect, these have shown strong correlation with human perception in prior literature and are commonly adopted in related works.\n\n\n**Concerns 5: Additionally, the current image metric implicitly evaluates the geometric alignment, texture similarity, and geometric similarity simultaneously, making it challenging to identify which part is problematic.**\n\nTable 1 and Table 2 in the paper evaluate the performance of two core modules independently in a controlled setting. Table 1 focuses solely on geometry similarity. In Table 2, we use the same retrieved geometry, with the only difference being the material painting methods, thereby isolating the performance of material painting. Table 3, on the other hand, is intentionally designed to jointly evaluate multiple aspects—geometry, texture, and alignment—to reflect the holistic performance of the entire system. \n\n**Concern 6: Although the light is not estimated, the proposed method measures the difference between the rendered image and the ground truth. This means light estimation error distorts albedo estimation. This should be clarified in the paper.**\n\n[Line 300 in main paper], we add lighting through global illumination using an HDR environment. Our PBR material estimation is performed without explicit lighting estimation, as experiments with lighting components were found to be less scalable and introduced significant computational overhead. Theoretically, this omission may introduce some distortion in albedo estimation. Our material selection module leverages both visual cues and language descriptions, and demonstrates encouraging results even without explicit lighting estimation (see Figures 6 and 7). We will clarify this point in the revision.\n\n**Question 1: What types of metrics should be used to evaluate the proposed method's performance adequately? Does the proposed benchmark thoroughly measure the perceptual (meaningful) performance of methods? Are the proposed criteria commonly agreed upon by designers or related industry workers?**\n\n- These metrics used in the paper are widely adopted and recognized in the community for evaluating both retrieval quality (DiffCAD, ScanNotate,  HOC-Search) and appearance modeling (PSDR-Room, Photo-Scene). \n- Additionally, we conducted a user study to evaluate both the retrieval and material-recovery components of our system. six  participants—graphics researcher, graphics engineer, architect, and 3D-vision specialists—each viewed four real-world scenes and were allowed to select up to two favorite results from all methods under comparison. Tables below show the percentage of favorite votes for each method in both tasks. Our proposed approach achieved the highest vote share in both retrieval and material painting, in line with the evaluation matrix presented earlier.\n\n\n | Scene                   | LiteReality        | DC                 | Phone2Proc          |\n|-------------------------|-------------------:|-------------------:|--------------------:|\n| **Boardroom**           | 31 (46.3 %)        | 30 (44.8 %)        | 6 (9.0 %)           | \n| **Common Room**  | 10 (47.6 %)        | 7 (33.3 %)         | 4 (19.0 %)          |\n| **Study Room**            | 7 (46.7 %)         | 7 (46.7 %)         | 1 (6.7 %)           |\n| **Meeting Room**                 | 13 (61.9 %)        | 6 (28.6 %)         | 2 (9.5 %)           | \n| **Total**               | 61 (49.2 %)    | 50 (40.3 %)    | **13 (10.5 %)**     |\n\n\n| Scene                   | LiteReality | MIR + AO | Sem&Vis | MIR   | PhotoShape |\n|-------------------------|----------------------:|----------:|----------------:|------:|-----------:|\n| **Boardroom**           | **81.4 %**                | 62.4 %    | 2.7 %           | 5.3 % | 7.2 %      |\n| **Common Room**  | 45.3 %                | **61.8 %**    | 15.3 %          | 1.8 % | 3.4 %      |\n| **Meeting Room**          | **57.8 %**                | 33.9 %    | 1.9 %           | 3.6 % | 4.2 %      |\n| **Study Room**    | **57.2 %**                | 15.9 %    | 3.0 %           | 20.4 %| 5.1 %      |\n\n**Question 2: Is each step evaluated individually, and does this paper conduct an ablation study for processing steps?**\n \nTable 1 and Table 2 evaluate the two core modules individually, while Table 3 reflects the overall system performance.\n\n**Question 3: Is measuring photorealism for a method that doesn't estimate lighting adequate?**\n\nOmitting lighting estimation is currently a practical and effective solution for PBR material estimation, as it allows the system to remain robust and scalable. However, we fully agree with the reviewer that incorporating lighting estimation would be a meaningful and valuable extension of this work, and would enhance the realism of the estimated PBR materials. For example, one could train a lightweight neural network to estimate HDR environment maps or point light positions directly from RGB-D inputs, which can then be used to refine material appearance. We will include a thorough discussion of this direction in the paper.\n\nThank you again for your time and thoughtful review of our work."}]

Author rebuttal:

Meta-review: This paper was reviewed by four experts in the field, and all of them are inclined to recommend acceptance of the paper after the rebuttal phase. The reviewers appreciate that the novel and important problem setting: producing realistic CAD representations from raw RGB-D scans at room scale, with reasonable technical improvements and relatively comprehensive experiments. Thus, this paper achieves the NeurIPS acceptance requirement.
Decision: Accept (poster)