ComGS: Efficient 3D Object-Scene Composition via Surface Octahedral Probes
Authors: Jian Gao
Mengqi Yuan
Yifei Zeng
Chang Zeng
Zhihao Li
Zhenyu Chen
Weichao Qiu
Xiao-Xiao Long
Hao Zhu
Xun Cao
Yao Yao
Venue: ICLR 2026 Poster (2026)
Keywords: Object-Scene Composition
Gaussian Splatting
Surface Octahedral Probes
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We introduce ComGS, a framework for realistic 3D object–scene composition, achieving real-time rendering (~28 FPS) with harmonious appearance and realistic shadows.

Abstract: Gaussian Splatting (GS) enables immersive rendering, but realistic 3D object–scene composition remains challenging. Baked appearance and shadow information in GS radiance fields cause inconsistencies when combining objects and scenes. Addressing this requires relightable object reconstruction and scene lighting estimation. For relightable object reconstruction, existing Gaussian-based inverse rendering methods often rely on ray tracing, leading to low efficiency. We introduce Surface Octahedral Probes (SOPs), which store lighting and occlusion information and allow efficient 3D querying via interpolation, avoiding expensive ray tracing. SOPs provide at least a 2x speedup in reconstruction and enable real-time shadow computation in Gaussian scenes. For lighting estimation, existing Gaussian-based inverse rendering methods struggle to model intricate light transport and often fail in complex scenes, while learning-based methods predict lighting from a single image and are viewpoint-sensitive. We observe that 3D object–scene composition primarily concerns the object’s appearance and nearby shadows. Thus, we simplify the challenging task of full scene lighting estimation by focusing on the environment lighting at the object’s placement. Specifically, we capture a 360° reconstructed radiance field of the scene at the location and fine-tune a diffusion model to complete the lighting. Building on these advances, we propose ComGS, a novel 3D object–scene composition framework. Our method achieves high-quality, real-time rendering at around 26 FPS, produces visually harmonious results with vivid shadows, and requires only 36 seconds for editing. The code and dataset are available at https://nju-3dv.github.io/projects/ComGS/.

Published as a conference paper at ICLR 2026
COMGS: EFFICIENT 3D OBJECT-SCENE COMPOSI- TION VIA SURFACE OCTAHEDRAL PROBES
Jian Gao Mengqi Yuan
Yifei Zeng*
Chang Zeng*
Zhihao Li
Zhenyu Chen?
Weichao Qiu
Xiao-Xiao Long'
Hao Zhu²
Xun Cao
Yao Yao 1
*Nanjing University
Huawei Noah's Ark Lab
Multi Multi-View Inputs
Object Images
Empty
Zanok PESTR 20.34
GS-IR FSNt 12:11
Diffusion lcht 10,02
Empty
Images
Reconstruction
Editing
Composition Ground Truth
DiffHammoyy
PSNC: 22 44 08.01 FFS)
PENi: 22
PSNR: 26.25
Ours
Rendering
(0.03 TTS)
22.14 F78
Composition
(a) Task Description
(b) Comparison with Existing Approaches
Real-Worke Composition
Figure (a) Task: From multi-view images, we aim for realistic 3D object- scene composition via reconstruction, editing, and rendering. (b) Comparison: Compared to existing methods, our approach achieves superior harmonious appearance and plausible shadows (red arrows), while run- ning at around 26 FPS. (c) Application: Our method also performs well on real-world captures.
ABSTRACT
Gaussian Splatting (GS) enables immersive rendering, but realistic 3D ob- ject-scene composition remains challenging. Baked appearance and shadow in- formation in GS radiance fields cause inconsistencies when combining objects and scenes. Addressing this requires relightable object reconstruction and scene lighting estimation. For relightable object reconstruction, existing Gaussian-based inverse rendering methods often rely on ray tracing, leading to low efficiency. We introduce Surface Octahedral Probes (SOPs), which store lighting and occlusion information and allow efficient 3D querying via interpolation, avoiding expensive ray tracing. SOPs provide at least 2x speedup in reconstruction and enable real-time shadow computation in Gaussian scenes. For lighting estimation, ex- isting Gaussian-based inverse rendering methods struggle model intricate light transport and often fail in complex scenes, while pearning-based methods predict lighting from single image and are niewpoint-sensitvve object-scene composition primarily concerns the object's appearance and nearby shadows. Thus, we simplify the challenging task of full scene lighting estimation by focusing on the environment lighting at the object's placement. Specifically, we capture a 360 reconstructed radiance field of the scene at the location and fine- tune diffusion model to complete the lighting. Building these advances, we propose ComGS, a novel 3D object-scene composition framework. Our method achieves high-quality, real-time rendering around 26 FPS, produces visually harmonious results with vivid shadows, and requires only 36 seconds for editing. The code and dataset available at https://nju-ddv wwt.tg.gtthhhhreeeeee http://wwww..guuboooooe////ooooo io/projects/CCooSSS.
We observe that 3D
'Equally Contributed,

---

Published as a conference paper at ICLR 2026
INTRODUCTION
Gaussian Splatting (GS) (Kerbl et al., 2023) has emerged as a powerful point-based differentiable rendering technique, enabling bigh-fidelity 3D reconstruction and rendering from multi-view im- ages. Despite its success, realistic 3D object-scene composition is still challenging, the GS ra- diance field inherently bakes appearance and shadows. Realistic object-scene composition requires integrating objects into scenes while ensuring visual harmony and physically plausible shadows. Two critical obstacles must be addressed to achieve this goal: (1) relightable object reconstruction to account for appearance variations, and (2) scene lighting estimation enable object relighting and realistic shadows.
For relightable object reconstruction, existing Gaussian-basee inverse rendering approaches (Gu et al., 2025; Sun et al., 2025) often rely on costly ray tracing for occlusion and indirect lighting, leading to low efficiency. This constitutes significant bottleneck for real-time object-scene compo- sition. Meanwhile, lighting estimation in complex scenes remains open problem. Gaussian- inverse rendering (Liang al., 2024b) methods struggle due to intricate light transport in complex scenes, while learning- based methods (Zhan et al., 2021: Phongthawee et al., 2024), which typi- cally predict lighting from single image, fail to ensure multi-view consistency. Consequently, both categories of methods fall short in providing reliable lighting for realistic object-scene composition.
To address these challenges, we propose two key innovations. First, we introduce Surface Octa- hedral Probes (SOPs) for efficient relightable Gaussian object reconstruction. SOPs store indirect lighting and occlusion, allowing shading points to access this information via interpolation rather than costly ray tracing. Thanks to the efficiency of SOPs, method achieves at least 2x speedup in reconstruction compared to state-of-the-art approaches, while maintaining comparable accuracy.
Second, we observe that achieving visually harmonious and plausible shadows in object-scene com- position does not require perfectly decoupling scene lighting, which presents significant technical challenges. In practice, it is sufficient to estimate the lighting around the object. Accordingly, reformulate scene lighting estimation as the task of inferring an environment map from partially reconstructed Gaussian radiance fields. We first perform 360-degree trace of the Gaussian scene to extract partial radiance and then input it into a fine-tuned Diffusion Model (Rombach et al., 2022) to generate complete lighting estimation. Our approach leverages the existing Gaussian radiance field, Icading to more accurate and realistic lighting results.
Building on these advances, we present ComGS, a novel object-scene composition framework lever- aging SOPs. Our solution operates in three stages: (1) Reconstruction, for relightable object and scene reconstruction; (2) Editing, including lighting estimation and SOPs-based occlusion caching; and (3) Rendering, covering object relighting and shadow casting. By combining reconstructed re- lightable objects with estimated lighting, we achieve visual coherence and vivid shadows. SOPs cache occlusion during the editing stage, enabling efficient calculation of plausible shadows during rendering. For static object placements, our method achieves approximately 26 FPS rendering, with an editing time of 36 seconds. We summarize our key contributions follows:
A complete 3D object-scenc composition framework that ensures visual harmony and plau- sible shadows, achieving 26 FPS rendering with an editing time of only 36s.
A novel inverse rendering pipeline using Surface Octahedral Probes (SOPs) for efficient relightable object reconstruction, providing over 2x speedup compared SOTA methods.
Extensive evaluations on SynCom, public datasets, and phone captures that show +1.4 dB PSNR, 21 % higher 3D consistency, and 56% greater harmony over existing methods, high- lighting our framework's potential for immersive 3D applications.
2 RELATED WORK
Object-Scene Composition Most methods aim to combine foreground object images with back- ground images. Some approaches (Niu et al., 2023; Zhou et al., 2024) focus specifically on achieving visual harmony between foreground and background. Diffusion-bars based methods (Chen et al., 2024; Zeng et al., 2024; Liang et al., 2024a) leverage generative models for flexible object placement and harmonious appearance. Intrinsics- and physics-based methods (Careaga et al., 2023; Zhang al.,
2

---

Published as a conference paper at ICLR 2026
Gaussian Seçue
Splatting
Scune (RGGRD)
Lah Eaianaiion
Ozasi hilghinn
Cerpoctiios
Lighting Ratimation 15
Depth
Imagis
HDR
Compositing
Ocolusion Cuching
Shadow
Casting
Relightable
Causslan
Object Relighting
Objject Images
Object
-5
Object (RGBD)
Composition
ObjectsSorne
Reconstruction (Sec. 3.1)
Editing (Sec. 3.2)
Rendering (Sec. 3.3)
Figure 2: Realistic 3D Object-Scene Composition Pipeline. Our approach consists of 3 stages: reconstruction (Sec. 3.1), where we reconstruct the Gaussian scene and relightable Gaussian object from multi-view images; editing (Sec. 3.2), where estimate scene lighting and cache occlusion using Surface Octahedral Probes; and rendering (Sec. 3.3), where we perform splatting, object re- lighting, shadow casting, and depth compositing. The pipeline achieves visually harmonious results with realistic shadows and near-real-time performance.
2025) decompose scene images into physical properties to enable photorealistic integration. Some works extend to 3D object insertion (Tarko et al., 2019; Ye et al., 2024b; Jin et al., 2025), using mesh-based representations or NeRF (Mildenhall et al., 2020), but they often rely simplified assumptions or focus mainly on appearance, leaving effects such as shadows and complex light- ing largely unhandled. MV-CoLight (Ren al., 2025) accomplishes object composition with feed-forward architecture, while its performance suffers from the domain gap of the training data.
Inverse Rendering Inverse rendering approaches built on mesh (Hasselgren et al., 2022; Dai et al., 2025) and NeRF (Zhang al., 2021a;b; 2022; Yao al., 2022; Jin al., 2023) have proven suc- cessful. Recently, Gaussian Splatting (GS) (Kerbl et al., 2023) has enabled more efficient inverse rendering (Jiang al., 2024: Gao et al., 2024; Gu al., 2025; Huang et al., 2025) thanks its fast rendering. However, inverse rendering still struggles with complex scenes due to incomplete scene captures and challenging light transport. While some GS- GSS-aased methods (Liang et al., 2024b; Chen et al., 2025) attempt scene-level reconstruction, they often rely on simplified lighting assumptions, resulting inaccurate lighting decomposition and reduced realism object-scene composition.
Lighting Estimation Early methods (Gardner et al., 2017: Wang et al., 2022a; Zhan et al., 2021; Somanath & Kurz, 2021) estimate high dynamic range (HDR) environment maps from single low dynamic range (LDR) photo to relight virtual objects for scene blending. For spatially-varying lighting. some methods predict per-pixel illumination (Li et al., 2020) use 3D octree- based rep- resentations (Wang et al., 2024). Recently. DiffusionLight (Phongthawee et al., 2024) leverages pretrained diffusion models for improved generalization. However, relying single image often leads to multi-view inconsistencies. While Lyu et al. (2023) integrate inverse rendering into diffu- sion denoising for 360 illumination, their method relies on an explicit Mesh-based representation.
3 METHOD
We propose realistic 3D object-scene composition pipeline, as shown in Figure 2. The pipeline consists of three stages: reconstruction, editing, and rendering. Each stage is detailed as follows.
3.1 RECONSTRUCTION
Our reconstruction process is divided into two steps. For objects, we apply both steps to reconstruct relightable Gaussian object. For scenes, perform only the first step for radiance field.
Multi-Targer Rendering To achieve better geometry than 3DGS (Kerbl et al., 2023), 2D Gaussian Splatting (2DGS) (Huang et al., 2024) employs surfels in 3D space as rendering primitives. Each surfel is defined by center p;, quaternion q: for orientation, and scaling vector s; for deformation.
3

---

Published as a conference paper at ICLR 2026
Surface Octahedral Probes
Environment Map
GT
Supervision
unfild
Ray Tracing
Radiamoe (4,)
Dcclusion (C))
Lighting (2g))
Probes Placement
Geometric Buffers
Interpolation
Render
Relightable
Incomtng Rays
Loss
Ote)
Low Discrepancy Sampling
Foetton (N)
Normal (e)
Material Buffers
The Rendering Equation
NAnde) + doallA balille aide
0
my
R. BRDF
Suppo Eepeorene Masii
Alleedo
Bougls Metallic (m)
Render
Figure 3: Inverse Rendering with Surface Octahedral Probes (SOPs), We utilize trained re- lightable 2D Gaussians to generate GBuffers via splatting, followed by deferred physically based rendering for a render image. Illumination split into direct lighting from environment map, indi- rect lighting and occlusion captured by textures in SOPs. Both the environment map and textures are stored as octahedral textures. Low-discrepancy ray sampling with random rotation is used to compute illumination shading point, with indirect light and occlusion derived via KNN interpo- lation from nearby probes. SOPs are initialized with ray tracing and optimized under guidance, avoiding intensive ray tracing per optimization iteration and boosting inverse rendering efficiency.
To enable relightable 2D Gaussians, we further equip them with material parameters {a,, rs, my } for albedo, roughness, and metallic. Through 2DGS, we perform differentiable multi-target rendering generate G-buffers B in a single pass via alpha blending:
N
B Σ 2=1
Toabb
T. = (1-)aa)) az),
(1)
where b, = (c,, 1, d, n1, T4, m.), with C₁ as color, d, as ray-surfel intersection depth and n, surfel normal. These G-buffers includes RGB buffer C, geometry buffers (W for weight, D for depth, / for normal) and material buffers (AA for albedo, R for roughness, and M for metallic). As in prior works (Gao et al., 2024; Liang et al., 2024b), we use the unbiased depth = D/W.
3.1.1
STEP RADIANCE AND GROMETRY RECONSTRUCTION
Loss We follow the rendering loss from (Kerbl er al., 2023):
Crge (C,C,,) + 0.2 (1 SSIM(C,C,)),
(2)
and apply depth-normal consistent regularization (Huang et al., 2024; Gao et al., 2024):
Lazn -Na).
(3)
where N2 is devided from depth. For objects, we apply mask constraint (Gao et al., 2024) as: Lmask -K W (1 - K.) log (1 W),
where IC is provided mask, The total loss for the first step is:
L Ergb Adam + AmaskLaaak
(5)
3.1.2 STEP 2: MATERIAL AND LIGHTING DECOMPOSITION
Deferred Physically Based Rendering From G-Buffers, we produce PBR image in a deferred shading manner. We first project the unbiased depth map D into world space as shading points (x), and then evaluate the rendering equation these shading points:
/ f(wo, wi, x) [illss))r ..n..nnneee n)dwi,
Lo Lowwo: x) =
(6)
4

---

Published as a conference paper at ICLR 2026
where is the surface normal, f denotes the BRDF, L; and Lo the incoming and outgoing radiance from direction Wi and Wo. and ft represents the hemisphere above the surface. We adopt a simplified Disney BRDF model (Burley & Studios, 2012). We evaluate this integral via Monte Carlo sampling by drawing S, rays {w;) from low-discrepancy Hammersley point set, computed
2x S.
Cpbr (x)
Σ S, [Wo, wi, Lx n).
Illumination Modeling We model illumination as direct lighting Lair and indirect lighting Lins combined through occlusion O:
L,(w;) (1 Owr, + Lin (wig).
(8)
We set direct lighting as a learnable environment map, and the indirect lighting as the inter-reflection. A straight-forwate method to obtain occlusion and indirect lighting is to perform ray tracing on Gaussian point cloud, similar to IRGS (Gu et al., 2025). However, ray tracing is computationally intensive, resulting in low efficiency.
To address this, we propose Surface Octahedral Probes (SOPs) for efficient querying of indi- rect lighting and occlusion. These probes are positioned near the surface, with their radiance and occlusion textures { O} initialized via ray tracing. In implementation, we employ octahedral textures (Praun& & Hoppe, 2003) for their low memory footprint and minimal distortion.
Automatic Placement of SOPs Careless placement can mix SOPs with Gaussian points, causing light-leaking artifacts. To mitigate this issue, we propose a placement strategy that positions SOPs near the surface. We first render geometry buffers for all viewpoints, followed by Multi-View Depth Fusion (Galliani al., 2015) to generate a dense surface point cloud with normals. We then apply Farthest Point Sampling (FPS) to obtain a uniform subsample of points. Finally, these points are slightly offset along their normals to define the placement locations of SOPs.
Efficient Querying through SOPs At each shading point x, we efficiently query indirect lighting and occlusion from SOPs via K-Nearest Neighbors (KNN) interpolation. As an example, the indirect lighting at x is queried by first identifying neighboring SOPs k E N(x) using Fixed Radius Nearest Neighbors (FRNN) search, followed by interpolation:
Lin (x) = Σx w₂ (,(kk) Lin (k)
(9)
Σx W.s (k)w)(k)
where w. and wa is the spatial and back-face weights, respectively. Given p& the location of k-th neighbor SOP, we define a direction vector as d₁ P x. Then, the spatial weight is defined as:
1
(10)
which assign greater influence for SOPs closer to the shading point. And, the back-face weight (Ma- jercik et al., 2019; McGuire et al., 2017) is defined as:
d2 n,) + 0.01,
wh = 0.5 (1 +
(1I)
||
indicating greater contribution when the direction vector da is more aligned with the normal ny.
Loss We use rendering loss similar to that in Eq. for the PBR image Cpbr, denoted as Lpbr- For material regularization, we adopt the Lambertian assumption (Yao et al., 2022) with a cost favoring high roughness and low metallic values under non non-view-depended lighting:
Liam = L1 (R, 1) + (M,,).
(12)
We also supervise SOPs' textures {Lin: 0} using the traced radiance Lyr and occlusion Ozr:
Loops = L1 (Lins Lir) + (0,)
(13)
The final loss for the second step is:
L Lybr Atamblam + Asops Csops + Dd22nhitnn + Amask Smask,
(14)
5

---

Published as a conference paper at ICLR 2026
3.2 EDITING
From reconstructed scene and relightable Gaussian object, the editing stage estimates scene lighting and caches occlusion, bridging relighting and real-time shadow computation during rendering.
3.2.1 LIGHTING ESTIMATION
We assume that the object is small relative to the scene and that its placement affects only its own appearance and nearby regions. Under this assumption, the challenging task of estimating lighting in complex scenes can be reformulated
more tractable environment map inpainting problem.
From reconstructed Gaussian scene, we generate HDR environment map for the target placement loca- tion (Figure 4). We first perform 360 sweep around the location to capture an incomplete RGB panorama, partial normal map, and alpha mask of the recon- structed regions. These inputs are processed by fine- tuned Stable Diffusion 2.1 (Rombach al., 2022) model to generate complete environment map. For HDR, we use an exposure value (EV)-conditioned prompting scheme (Phongthawee et al., 2024): base model is trained EV = and fine-tuned with interpolated em- beddings for different exposure levels. During inference, we generate maps at EV {-5, 22.5, 0} and fuse them into an HDR environment map, which is then converted to an octahedral texture for relighting and shadow com- putation. More details are included in Appendix D.
EV [5, 2.5.
HDR
Alpha
Fine-tuned Stable Diffasion
Merging
Nommal
Figure 4: given location, we create partial panoramic view via 360 sweep of the Gaussian scene, yielding an incom- plete RGB image, normal map, and al- pha mask of reconstructed areas. Then, use fine-tuned Stable Diffusion to infer HDR environment map.
Lighting Estimation. At
3.2.2 OCCLUSION CACHING AND SHADOW CASTING
The shadow cast by a newly placed object in the scene is determined by the occlusion O' intro- duces. Computing O' directly via ray tracing for every rendering is straightforward but computa- tionally expensive. Instead, we cache the object-induced occlusion O' uusin our proposed SOPs.
Under the Lambertian scene assumption, the rendering equation (Eq. 6) simplifies to:
Lo = fa ʃ L,(w))(w: n)dw,
(15)
When an object placed, the scene rendering becomes:
L' = fa L,(w))(1- (www))) n)dw,,
(16)
where O' is the object-induced occlusion cached by SOPs newly placed in the scene, distinct from the self-occlusion O of the object in Eq. 8. The cast shadow is then derived as:
S= L'
(17)
L.
To implement this, potential shadow region is defined around the object placement location, with size proportional to N times the object dimensions. Within this region, SOPs are distributed follow- ing the Automatic Placement strategy (Sec. 3.1.2). Their occlusion textures are then precomputed via ray tracing, enabling efficient shadow calculation during rendering.
3.3 RENDERING
We start by performing multi target rendering (Sec, 3.1) to obtain the RGBD of the Gaussian scene. Next, relight the object using the estimated environment map (Sec. 3.2.1) and compute shadows with cached occlusion (Sec. 3.2.2).
Both object relighting and shadow computation involve evaluating integrals, which accelerate via Monte Carlo importance sampling. The octahedral texture's low distortion and roughly uniform texel areas simplify defining the Probability Density Function (PDF) as:

---

Published as a conference paper at ICLR 2026
Composition Performance on SynCom Dataset. Objective metrics (PSNR, SSIM), subjective metries (3D consistency, Con.; harmony, Harm.), and efficiency metrics (editing time, FPS) are reported.
Table 1:
PSNR1
SSIM
Con.1
Harm.
FPS-
T(Edio)
Empty
Composition (Copper)
Difflarmeny
22.436
0.825
3.125
2.929
0.01
ZeroComp
20.344
0,780
1.642
1.504
0.40
MV-CoLight
21.045
0.855
2.800
2.633
1.01
GS-IR
22.418
0.824
3,283
2.125
2,11
GI-GS
22,877
0.849
3.746
2.908
0.29
IRGS
22,417
0.799
3,496
2,883
0.03
Editing (Silver)
Editing 2 (Red Matic)
Figure 5: Material Editing in real-world composition, from copper to silver and red malle. Please ZOOM IN for details.
Diffusionl.ight
21.842
0.841
1.913
2.171
0.02
Ours (Trace)
24.567
0.870
4,746
4.600
4.02
14.59
Ours (SOPS)
24.282
0.868
4.563
4.588
26.14
36.12
PDF(w.) = H.WW 4T
(18)
where He, Wt, and denote the height, width, and intensity of the texture, respectively.
Our relighting process begins by sampling set of rays S, from the probability density function
(PDF) in Eq. 18, followed by Monte Carlo integration of the rendering equation (Eq. 6). To account for self-occlusion, we modulate the direct lighting (w;) by the visibility factor (1 ((ws)) derived from the object's SOPs, yielding T,((w:) = (1 ( (w;)). This adjusted term then used in the integration (Eq 7). As for indirect lighting Lin, we intentionally omit it due to its prohibitive computational cost. Then, cast shadows are subsequently computed via Eq. 17, where the occlusion O' is interpolated from the SOPs placed in the scene using Efficient Querying (Sec. 3.1.2).
Finally, depth compositing is applied to produce a realistic 3D object-scene composition.
4 EXPERIMENTS
4.1 IMPLEMENTATION DETAILS
Reconstruction Details We set the octahedral environment map resolution to 256 256, and SOPs' texture resolutions to 16 x 16, using 128 sampling rays. SOPs number is set to 5k, with an offset distance empirically set to 1% of the object's size. At step 1, loss weights are Ad20 = Amask = 0.05, with other settings following 2DGS, training over 30k iterations. At step 2, learning rates are 0.01 for environment map, albedo, roughness, and metallic, and 0.001 for SOPs textures, initialized via 2D ray tracing; loss weights are {Aiem }sope) {0.001, 1}. over 2k iterations.
Editing and Rendering Details The estimated environment map resolution 1024 x 512, con- verted to an octahedral texture at 512 x 512. We define the scene within 6 times the object's 's size as the potential shadow region, placing 10k SOPs there. We sample 256 rays for rendering.
4.2 COMPOSITION PERFORMANCE
SynCom Dataset We create synthetic dataset to evaluate 3D object-scene composition. Multi- view images of 4 objects (bottle, horse, kettle, and toy), scenes (artwall, altic,forest. room), and their 16 compositions are rendered using the Blender Cycles Engine. The precise control of object placement and camera settings in Blender ensures a highly reliable reference for evaluation. Further details of our SynCom dataset are provided in Appendix A.
For the composition performance evaluation, we compare three distinct categories of approaches:
(1) Image Composition:
Scene and object are first reconstructed using 2DGS (Huang et al., 2024), synthesized for novel views, and then composed with a image composition algorithm, specifically DiffHarmony (Zhou al., 2024) and ZeroComp (Zhang et al.. 2025). Although MV-CoLight (Ren et al., 2025) is a two-stage method that first performs image composition and then addresses 3D composition, we categorize here for simplicity.
7

---

Published as a conference paper at ICLR 2026
Table 2: Reconstruction performance on TensoIR.
Our method achieves accuracy comparable to SOTA ap- proaches with least 2x efficiency improvement.
GS-IR
IR-GS
NVS
Albedo PSNR
Relighting PSNR
Training
Method
PSNR
Time.
NeRFactor
24.679
25. 15.
23,383
>100
InvRender
27.367
27.341
23.973
15
Diffusionlight 1 Diffusionl Light 2
Ours
TensoIR
35.088
29.275
28.580
5
Figure 6: Environment Maps Compar- ison. GS-IR and IRGS fail in com- plex scenes, DiffusionLight is viewpoint- inconsistent, while our method yields su- perior and consistent results.
GS-IR
35.333
30.286
24.374
16.40 min
R3DG
38.423
31.926
29.766
47.80 min
IRGS 35.751
31.658
30.250
21.45 min
Ours
35.822
31.683
30.474
7.93 min
(2) Gaussian-based Inverse Rendering: Scene and object are reconstructed with Gaussian- inverse rendering methods, composited, and then rendered with PBR. Specifically, GS-IR (Liang et al., 2024b), GI-GS (Chen et al., 2025) and IRGS (Gu et al., 2025) are selected for comparison. Note that R3DG (Gao et al., 2024) is not included due to excessive memory usage on scenes, result- ing reconstruction failure.
(3) Variants of Our Pipeline: (a) DiffussionLight, which estimates illumination from single RGB image via the DiffusionLight (Phongthawee et al., 2024); (b) Our (Trace), which employs computa- tionally intensive ray tracing on Gaussians to generate shadows; (c) Our (SOPs), which utilizes our proposed SOPs to cache occlusion and achieve efficient shadow rendering.
We report both objective and subjective metrics in Table 1. For objective evaluation, Peak Signal-to- Noise Ratio (PSNR) and Structural Similarity Index (SSIM) are computed against the ground truth. Subjective evaluation is based a Mean Opinion Score (MOS) survey, where 40 participants rate 3D consistency (Con.) and harmony (Harm.) of the compositions from (poor) to 5 (excellent). Editing time and rendering frame rate are also reported.
We show the composition results of all methods in Figure 7. Across both objective and subjective metrics, as well overall rendering performance, our methods Our (Trace) and Our (SOPs) con- sistently outperform competing approaches, delivering harmonious compositions and, importantly, realistic shadows. In particular, Our (Trace) yields smoother shadows but a lower rendering speed, whereas Our (SOPs) achieves comparable quality while sustaining significantly higher frame rate.
Real-World Datasets Beyond synthetic data, we further validate our pipeline on real-world ob- ject-scene composition. For public datasets, we select four objects from BlendedMVS (Yao al., 2020) and four scenes from Tanks and Temples (Knapitsch et al., 2017). with the results shown in Fig. 11. We also evaluate on smartphone-captured sequences, including two objects and two scenes reconstructed using COLMAP (Schonberger & Frahm, 2016), as illustrated in Fig. 14. Over- all, these results demonstrate that our approach generalizes well to real-world data, producing har- monious compositions with realistic shadows.
4.3 RECONSTRUCTION PERFORMANCE
Although our main focus is realistic 3D object-scene composition, we also assess reconstruction accuracy on TensoIR (Jin et al., 2023) and our SynCom-Object dataset. As shown in Table 2, our approach achieves accuracy comparable state-of-the-art methods while delivering the fastest re- construction, thanks to the proposed SOPs. R3DG attains the highest novel view synthesis accuracy due to its lack of indirect lighting supervision, but this causes incorrect indirect lighting and bright spot artifacts in relighting, as shown in Figure 19, Compared with IRGS, our method avoids expen- sive per-point ray tracing through efficient KNN interpolation and trains on the full image instead of random pixel sampling. Quantitative results accuracy and efficiency of our method. Additional results are provided Appendix
Cbejeet (Table 4) further validate the
4.4 LIGHTING ESTIMATION PERFORMANCE
We present environment maps estimated by different methods in Figure 6. Inverse rendering ap- proaches, such GS-IR and IRGS, struggle to capture the intricate lighting transports in com-
8

---

Published as a conference paper at ICLR 2026
horse in artwall
in attic
kettle in forest
bottle in room
Empty Scene
DiftHamonn
ZeroComp
GS-IR
IRGS
Ours (Trace) DiffusionI..o ightt
Oars (SOPs)
Ground Truth
Figure 7: Object-scene composition results on SynCom dataset. Our methods, Ours (Trace) and Ours (SOPs), outperform others in realistic object-scene composition, producing harmonious visu- als and realistic shadows. Ours (Trace) offers slightly better quality but at a lower rendering frame rate. DiffHarmony and ZeroComp. based on image-level harmonization with scene-object occlusion. Inverse rendering methods like GS-IR and IRGS deviate from real lighting in complex scenes: GS-IR's coarse model lacks harmony, while IRGS, though more de- tailed, fails to render realistic shadows. DiffusionLigh generates shadows in some cases but suffers from instability due to inconsistent single-image estimations across different viewpoints.
composition, struggles

---

Published as a conference paper at ICLR 2026
plex scenes. This difficulty is further exacerbated by the incomplete coverage of captured views. Learning-bast based methods such as DiffusionLight, rely on a single input image and often produce inconsistent lighting estimations across different viewpoints. By contrast, method leverages the reconstructed scene radiance field, enabling both higher-quality lighting estimation and multi-view consistent object-scene composition. Further results are presented in Appendix D.
DISCUSSION
Assumptions and Failure Cases 3D object-scene composition is highly challenging problem, and we rely on several assumptions (Ye et al., 2024b; Wang et al., 2022b) to make it tractable.
The inserted object is relatively small and affects only the local area of the scene.
The scene is predominantly Lambertian, allowing reasonable approximation shadows.
These assumptions naturally limit the applicability of our approach. As illustrated in Figure two representative failure cases highlight these limitations.
In the first case, the object fails to cast remote shadow, which is due to the first assumption. In the second case, we modify the table in the room scene from our SynCom dataset to be highly specular, which prevents our method from accurately modeling mirror-like reflections. This failure stems from both the second assumption and limitations of the 2DGS-basel reconstruction stage (Sec- tion 3.1.1), which struggles with reflective scenes.
Remote shadow Figure 8: Failure cases. We failed to cast remote shadow and model mirror- like reflections.
(b) Mirror reflection
SOPs Placement In our experiments, we place SOPs using a heuristic approach to reduce light leaking, by slightly offsetting each SOP along the surface normal. In all experiments, this offset is set to 1% of the object size. This is validated to some extent in Figure 20 and Table 5, and we did observe any noticeable failures severe light leaks in all our experiments. We also find that the shadow quality improve as the number of SOPs and the resolution of their textures increase in Figure 15. Building on these insights, future work could explore adaptive SOP placement strategies to achieve better fidelity with fewer SOPs, and reduce memory usage.
Incremental SOP Updates Our pipeline naturally handles camera motion, since SOPs cache oc- clusion in scene space and can be reused across views. When the inserted object moves, changes in visibility and lighting require recomputing SOPs. Developing efficient incremental update strategies for moving objects remains challenging and is an interesting direction for future work.
Single- View Scenes Our method relies on multi-view 3D reconstruction, but extending it to single- image inputs is interesting direction for future work.
CONCLUSIONS
In this study, tackle the challenge of realistic 3D object-scene composition, seamlessly inte- grating objects into scenes with visual harmony and physically plausible shadowing. We propose ComGS, organized into three stages: reconstruction, edit, and rendering. In the reconstruction stage, we reconstruct relightable Gaussian objects and the scene's Gaussian radiance field. Surface Octahedral Probes (SOPs) are introduced accelerate object reconstruction without compromising accuracy. In the edit stage, within the context of object-scene composition, we simplify complex scene lighting estimation via fine-tuned diffusion model. SOPs are also used to cache occlusion caused by the inserted objects. In the rendering stage, we perform object relighting and shadow computation, combining results through depth compositing to produce the final object-scene composition. Our framework enables near-real-ting rendering with vivid and physically plausible shadows.
local lighting completion problem the object placement site, solved
Acknowledgments This work was supported by National Key R&D Program of China 20033FB3309900), the National Natural Science Foundation of China (62472213), and was par- tially supported by Jiangsu Broadcasting Corporation.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. In CVPR, 2022.
Brent Burley and Walt Disney Animation Studios. Physically-base shading disney. In Siggraph, 2012.
Chris Careaga, S. Mahdi H. Miangoleh, and Yağız Aksoy, Intrinsic harmonization for illumination- aware compositing. In SIGGRAPHI Asia, 2023.
Hongze Chen, Zehong Lin, and Jun Zhang. Gi-gs: Global illumination decomposition on gaussian splatting for inverse rendering. In ICLR, 2025.
Xi Chen, Lianghua Huang, Yu Liu, Yujun Shen, Deli Zhao, and Hengshuang Zhao. Anydoor: Zero- shot object-level image customization. In CVPR, 2024.
Zhaoxi Chen, Guangcong Wang, and Ziwei Liu. Text2light: Zero-shot text-driven hdr panorama generation. TOG, 2022.
Yuxin Dai, Qi Wang, Jingsen Zhu, Dianbing Xi, Yuchi Huo, Chen Qian, and Ying He. Inverse rendering using multi-bounce path tracing and reservoir sampling. In ICLR, 2025.
Silvano Galliani, Katrin Lasinger, and Konrad Schindler. Massively parallel multiview stereopsis by surface normal diffusion. ICCV, 2015.
Jian Gao, Chun Gu, Youtian Lin, Zhihao Li, Hao Zhu, Xun Cao, Li Zhang, and Yao Yao. Relightable 3d gaussians: Realistic point cloud relighting with brdf decomposition and ray tracing. In ECCV, 2024.
Marc-André Gardner, Kalyan Sunkavalli, Ersin Yumer, Xiaohui Shen, Emiliano Gambaretto, Chris- tian Gagné, and Jean- François Lalonde. Learning to predict indoor illumination from a single image. TOG, 2017.
Marc-André Gardner, Yannick Hold-Geoffroy, Kalyan Sunkavalli, Christian Gagné, and Jean- François Lalonde. Deep parametric indoor lighting estimation. In ICCV, 2019.
Chun Gu, Xiaofei Wei, Zixuan Zeng, Yuxuan Yao, and Li Zhang. Irgs: Inter-reflective gaussian splatting with 2d gaussian ray tracing. In CVPR, 2025.
Jon Hasselgren, Nikolai Hofmann, and Jacob Munkberg. Shape, light, and material decomposition from images using monte carlo rendering and denoising. In NeurIPS, 2022.
Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao. 2d gaussian splatting for geometrically accurate radiance fields. In SIGGRAPH, 2024.
Letian Huang, Dongwei Ye, Jialin Dan, Chengzhi Tao, Huiwen Liu, Kun Zhou, Bo Ren, Yuanqi Li, Yanwen Guo, and Jie Guo. Transparentgs: Fast inverse rendering of transparent objects with gaussians. TOG, 2025.
Yingwenqi Jiang, Jiadong Tu, Yuan Liu, Xifeng Gao, Xiaoxiao Long, Wenping Wang, and Yuexin Ma. Gaussianshader: 3d gaussian splatting with shading functions for reflective surfaces. In CVPR, 2024.
Haian Jin, Isabella Liu, Peijia Xu, Xiaoshuai Zhang, Songfang Han, Sai Bi, Xiaowei Zhou, Zexiang Xu, and Hao Su. Tensoir: Tensorial inverse rendering. In CVPR, 2023.
Yanlin Jin, Ashok Veeraraghavan, Akshat Dave, Guha Balakrishnan, et al. Transplat: Lighting- consistent cross-scene object transfer with 3d gaussian splatting. ArXiv, 2025.
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splat- ting for real-time radiance field rendering. TOG, 2023.
Arno Knapitsch, Jacsik Park, Qian Yi Zhou, and Vladlen Koltun. Tanks and temples: Benchmarking large-scale scene reconstruction. ToG, 2017.

---

Published as a conference paper at ICLR 2026
Zhengqin Li, Mohammad Shafiei, Ravi Ramamoorthi, Kalyan Sunkavalli, and Manmohan Chan- draker. Inverse rendering for complex indoor scenes: Shape, spatially-varying lighting and svbrdf from single image. In CVPR, 2020.
Ruofan Liang Zan Gojeic, Merlin Nimier-Davio David Acuna, Nandita Vijaykumar, Sanja Fidler, and Zian Wang. Photorealistic object insertion with liffusion-guidee inverse rendering. In ECCV, 2024a.
Zhihao Liang, Qi Zhang, Ying Feng, Ying Shan, and Kui Jia. Gs-ir: 3d gaussian splatting for inverse rendering. In CVPR, 2024b.
Linjie Lyu, Ayush Tewari, Mare Habermann, Shunsuke Saito, Michael Zollhöfer, Thomas Leimkühler, and Christian Theobalt. Diffusion posterior illumination for ambiguity-aware verse rendering. TOG, 2023.
Zander Majercik, Jean- Guertin, Derek Nowrouzezahraii, and Morgan McGuire. Dynamic diffuse global illumination with ray-traced irradiance fields. JCGT, 2019.
Morgan McGuire, Mike Mara, Derek Nowrouzezahraai, and David Luebke. Real-time global illumi- nation using precomputed light field probes. In SIGGRAPH I3D, 2017.
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. In ECCV, 2020.
Li Niu, Junyan Cao, Wenyan Cong, and Liqing Zhang. Deep image harmonization with learnable augmentation. ICCV, 2023.
Pakkapon Phongthawee, Worameth Chinchuthakun, Nontaphat Sinsunthithet, Varun Jampani, Amit Raj. Pramook Khungurn, and Supasorn Suwajanakorn. Diffusionlight: Light probes for free by painting chrome ball. In CVPR, 2024.
Emil Praun and Hugues Hoppe. Spherical parametrization and remeshing. TOG, 2003.
Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, al. Sam 2: Segment anything in images and videos. ArXiv, 2024.
Kerui Ren, Jiayang Bai, Linning Xu, Lihan Jiang, Jiangmiao Pang, Mulin Yu, and Bo Dai. Mv- colight: Efficient object compositing with consistent lighting and shadow generation. arXiv preprint arXiv:2505. 20..25555....8.... 2025.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In CVPR, 2022.
Johannes L Schonberger and Jan-Michael Frahm. cineturr-from-motie revisited. In CVPR, 2016.
Gowri Somanath and Daniel Kurz. Hdr environment map estimation for real-time augmented reality. In CVPR, 2021.
Hanxiao Sun, YuPeng Gao, Jin Xie, Jian Yang, and Beibei Wang. Svg-ir: Spatially-varying gaussian splatting for inverse rendering. CVPR, 2025.
Joanna Tarko, James Tompkin, and Christian Richardt. Real-time virtual object insertion for moving 360 videos. In SIGGRAPH VRCAI, 2019.
Guangcong Wang. Yinuo Yang, Chen Change Loy. and Ziwei Liu. Stylelight: Hdr panorama gener- ation for lighting estimation and editing. In ECCV, 2022a.
Xuecan Wang, Shibang Xiao, and Xiaohui Liang. Lightoctree: Lightweight 3d spatially-coherent indoor lighting estimation. In CVPR, 2024.
Zian Wang, Wenzheng Chen, David Acuna, Jan Kautz, and Sanja Fidler. Neural light field estimation for street scenes with differentiable virtual object insertion. In ECCV, 2022h.
12

---

Published as a conference paper at ICLR 2026
Yao Yao, Zixin Luo, Shiwei Li, Jingyang Zhang. Yufan Ren, Lei Zhou, Tian Fang, and Long Quan. Blendedmvs: A large-scale dataset for generalized multi-view stereo networks. In CVPR, 2020.
Yao Yao, Jingyang Zhang, Jingbo Liu, Yihang Qu, Tian Fang, David McKinnon, Yanghai Tsin, and Long Quan. Neilf: Neural incident light field for physically- material estimation. In ECCV, 2022.
Chongjie Ye, Lingteng Qiu, Xiaodong Gu, Qi Zuo, Yushuang Wu, Zilong Dong, Liefeng Bo, Yuliang Xiu, and Xiaoguang Han. Stablenormal: Reducing diffusion variance for stable and sharp normal. TOG, 2024a.
Keyang Ye, Hongzhi Wu, Xin Tong, and Kun Zhou. A real-time method for inserting virtual objects into neural radiance fields. IEEE TVCG, 2024b.
Zheng Zeng, Valentin Deschaintre, lliyan Georgiev, Yannick Hold- hood-Geoffroy, Yiwei Hu, Fujun Luan, Ling-Qi Yan, and Miloš Hašan. Rgb++x: Image decomposition and synthesis using material- and lighting-awe -eaare diffusion models. In SIGGRAPH, 2024.
Fangneng Zhan, Changgong Zhang, Yingchen Yu, Yuan Chang, Shijian Lu, Feiying Ma, and Xu- ansong Xie. Emlight: Lighting estimation via spherical distribution approximation. In AAAI, 2021.
Kai Zhang, Fujum Luan, Qianqian Wang, Kavita Bala, and Noah Snavely. Physg: with spherical gaussians for physies-based material editing and relighting. In CVPR, 2021a.
Inverse rendering
Xiuming Zhang, Pratul P Srinivasan, Boyang Deng, Paul Debevec, William Freeman, and Jonathan T Barron. Nerfactor: Neural factorization of shape and reflectance under an unknown illumination. TOG, 2021b.
Yuanqing Zhang, Jiaming Sun, Xingyi He, Huan Fu, Rongfei Jia, and Xiaowei Zhou. Modeling indirect illumination for inverse rendering. In CVPR, 2022.
Zitian Zhang, Frédéric Foouiar- Mathieu Garon, Anand Bhattad, and Jean-François Lalonde. Zerocomp: Zero-shot object compositing from image intrinsics via diffusion. In WACV, 2025.
Pengfei Zhou, Fangxiang Feng, and Xiaojie Wang. Diftharmony: Latent diffusion model meets image harmonization. In ICMR, 2024.
13

---

Published as a conference paper at ICLR 2026
A DETAILS OF SYNCOM DATASET
A.1 OVERVIEW
The task of realistic 3D object-scene composition aims to reconstruct objects and scenes separately from two sets of multi-view images, and then integrate the reconstructed 3D objects into the 3D scenes in a visually harmonious manner. Realism here means that the composed objects match the scene in appearance and produce physically plausible shadows.
To enable quantitative evaluation, we introduce SynCom (Synthetic Composition), a synthetic dataset designed for this task. Compared to real-world data, synthetic data offers precise control over object placement and camera setting, ensures accurate ground for the composed scenes, and provides high reproducibility, all of which facilitate reliable quantitative assessment, Below, provide detailed description of the SynCom dataset.
A.2
DATA SOURCE
attic
forest
artwall
room
kettle
toy
horse
bottle
Figure 9: Assets collected from BlenderKit, serving as the data source for our SynCom dataset, comprising four objects (bottle, horse, kettle, toy) and four scenes (artwall, attic, forest, room).
We collect 4 object assets and 4 assets from BlenderKit all under permissive licenses. The assets are shown in Figure 9. The object assets are designated as bottle, horse, kettle, and toy, while the scenes are labeled artwall, attic, forest, and room. To facilitate camera placement and approximate realistic environments, we perform careful manual editing of the scenes, adjusting layouts and enriching content.
A.3 SYNCOM DATASET
Our SynCom dataset is organized into three distinct collections: object, scene and composition. We render the dataset using the Cycles engine in Blender² to achieve high physical realism, particularly accurate global illumination and shadow effects.
Object Collection This collection created by rendering each of the four objects individually. The objects are illuminated using freely available IK-reollution HDRIs from PolyHaven' envi- ronment maps. The collection divided into a training set and a testing set. The training set focuses on the upper hemisphere of each object, with 200 camera positions randomly sampled around this region. The testing set comprises 100 images captured from three distinct latitude circles around
https://wwww.beedeeriiii
httpss//wwwwblendere hhtps://poyyhaven.com
14

---

Published as a conference paper at ICLR 2026
the object. For all images, cameras point toward the object center, and the resolution is fixed at 800 x 800 pixels.
Scene Collection This collection focuses rendering each of the four scenes individually. To create high-fidelity and complex scenes, we start with 4 scenes sourced from BlenderKit and modify them by adding, removing, or altering elements. This process results in 4 distinct environments with real-world-lig complexity: indoor settings (artwall, attic, room) and 1 outdoor setting (forest).
The process of rendering scenes is inherently more complex than rendering individual objects. Cap- turing a scene requires moving the camera inside it, which can lead to occlusion by the scene's ' structures or even cause the camera intersect with them, resulting in invalid views. To prevent this, We employ virtual auxiliary ellipsoid, with scales and center carefully adjusted along each axis, so that its upper surface remains clear of the scene's structures. Cameras are then placed on the ellipsoid's surface to capture the scene. Compared to spherical boundary, an ellipsoid provides greater flexibility accommodating the diverse shapes and layouts of different environments.
For the training set, 144 camera positions are randomly sampled from cells of uniform grid on the ellipsoid's ' surface, with all cameras pointing toward its center. For the test set, 72 camera positions sampled along spiral path slightly smaller, concentric ellipsoid. All images captured at a resolution of 1280 x 720 pixels.
Composition Collection This collection combines the four objects with the four scenes, yielding 16 distinct object-scene pairs. Each object is manually positioned with specified 3D location, ori- entation, and scale, and these placement parameters are recorded for later use. The ground truth for each composition is obtained by rendering the scene after object placement. To ensure controlled evaluation, all composite scenes are rendered from the same 72 test viewpoints as their correspond- ing empty scenes.
Leveraging the controllability synthetic data, these renderings provide a reliable benchmark for assessing realistic object-scene composition and related tasks such as object insertion, novel-view synthesis, and inverse rendering.
Object Collection
Scene Collection
Composition Collections
toy
room
toy in room
horse
forest
horse in forest
Figure 10: Samples selected from the SynCom dataset. The three columns from left to right correspond to the Object, Scene, and Composition collections. The precise control over cameras and the highly controllable scene illumination in synthetic data enable alignment between empty and composed scenes, facilitating quantitative evaluation of realistic 3D object-scene composition.
15

---

Published as a conference paper at ICLR 2026
View 1
View
View
View 4
Empty
Composition
(a) Bull in Ignatius
Empty
Composition
(b) Sculpture in Caterpillar
Empty
Composition
(c) Bell in Courtroom
Empty
Composition
(d) Egg in Playground
Figure 11: 3D object-scene composition on public datasets. We select four scenes (Ignatius, Caterpillar, Courtroom and Playground) from the Tanks and Temples dataset and four objects (Bull, Sculpture, Bell and Egg) from the BlendedMVS dataset. Starting from the multi-view images pro- vided by these datasets, we perform 3D object-scene composition using our pipeline, achieving harmonious composition of objects into the scenes along with physically plausible cast shadows.
16

---

Published as a conference paper at ICLR 2026
B MORE RESULTS ON COMPOSITION
B.I PUBLIC DATASETS
In addition
synthetic SynCom dataset, also evaluate method on public datasets. Specif- ically, we select several scenes from the Tanks and Temples (Knapitsch et al., 2017) and several objects from the BlendedMVS (Yao et al., 2020), with object masks obtained using SAM2 (Ravi et al., 2024). Our method is then applied to perform 3D object-scene composition on these data, as shown in Figure 11. For more intuitive visualizations, please see the supplementary video.
Material Editing We further demonstrate multi-view consistent material editing of objects in Fig- ure 12. For clearer visualization of these results, we refer readers to our supplementary video.
View
View 2
View 3
View 4
Original (Copper)
Edit.
(Silver)
Edit. 2
(Stone)
Edit.
(Red Matte)
Edit. (Red Glossy)
Figure 12: Material Editing. Our reconstructed objects are represented as relightable Gaussian point clouds, enabling flexible material editing. By adjusting the albedo, metallic, and roughness properties, we demonstrate material modifications of object from original copper silver, stone, red matte, and red glossy. The edited results remain consistent across multiple viewpoints.
(e) Figurine
(f) Box
(g)
Courtyard
(h) Hall
Figure 13: Shartphone-Capturee Objects and Scenes. Top row: sample images of our captured objects and scenes. Bottom row: corresponding camera pose visualizations.
17

---

Published as a conference paper at ICLR 2026
B.2 SERTPHONE-CAPTUREE DATA
We capture separate videos of scenes (i.e., outdoor courtyard and indoor hall) and objects (i.e., a figurine and a box) using an iPhone 16 Pro. Objects were placed under soft lighting and captured following a systematic spiral camera trajectory, which ensures sufficient viewpoint coverage for reconstruction. Figure 13 shows sample images along with visualizations of the captured camera poses, providing an overview of our real-world data collection setup.
Multi-view images of each are independently extracted from their respective videos via frame sam- pling, while object masks are obtained with SAM2 (Ravi al., 2024). Our method then applied to perform 3D object-scene composition. As shown in Figure 14, even on challenging smartphone- captured data, our approach produces harmonious appearances and realistic shadows. For a clearer and more vivid demonstration, please refer to the supplementary video.
View 1
View
View
View 2
Empty
Composition
(a) Figurine in Courtyard
(b) Box in Hall
Figure 14: Qualitative results on smartphone-captured data. Our approach enables 3D ob- ject-scene composition with harmonious appearances and realistic shadows, even on challenging smartphone-capuured data.
B.3 MEMORY CONSUMPTION SOPS
We evaluate the GPU memory consumption of our SOPs on the Bottom in Room scene from the SynCom dataset. The results for different numbers of SOPs are summarized in Table 3.
The memory usage with 20K SOPs is comparable to the ray-tracing baseline, which requires maintaining a BVH structure. This indicates a moderate overhead introduced by our method. The mild growth in mem- ory consumption as the number of SOPs increases
Table 3: GPU memory usage comparison
under different numbers of SOPs.
Setting
Memory (GB)
Ray Tracing (Baseline)
4.4
SOPs Num. -1,000
4.3
suggests that the memory is dominated by the Gaus- sian point cloud, which consists of a large number of Gaussians with high-dimensional attributes.
SOPs Num. -2,,00
4.4
SOPs Num. =40,000
4.5
B.4 ABLATION ON NUMBERS SOPS AND TEXTURE RESOLUTION
We analyze the influence of the number of SOPs and the probe texture resolution within our com- position pipeline. Our baseline configuration uses 10k SOPs at resolution of 16, and is compared against setups with fewer (5k) and more (20k) SOPs, as well as lower (8) and higher (32) texture resolutions.
As illustrated Figure 15, reducing the number of SOPs to 5k or the resolution to 8 introduces visible shadow aliasing and loss of fine details. Specifically, at resolution of 8, the angular sampling over the full 360 degrees becomes too coarse, resulting in inadequate shadow representation and significant directional errors. A resolution of 16. by contrast, yields visually acceptable results. Increasing either the number of SOPs or the texture resolution beyond the baseline values reduces aliasing and improves the overall shadow quality.
18

---

Published as a conference paper at ICLR 2026
Num,=10k, Res.=16
Num.=5k, Res.=16
Num.=20k Res.=16
Ray Tracing
Num.=10k, Res.=8
Num.=10k, Res.=32
Figure 15: Influence of the Number of SOPs and Texture Resolution. A resolution of 16 provides visually acceptable results, while resolution of 8 causes significant directional errors due to coarse angular sampling. Increasing the number of SOPs beyond 10k further reduces aliasing.
B.5 SELF-OCCLUSION
Figure 16 presents two composition results for illustrating the effect of modeling self-ocelusion. One result is generated without incorporating self-occlusion, and the other produced by our full method, which models self-occlusion by modulating direct illumination using the visibility factor (ws computed from the object's SOPs (see Section 3.3). Incorporating self-occlusion provides additional shading cues that contribute to more realistic 3D object-scene composition.
(a) Without self-occlusion
(b) With self-occlusion
(c) Ground Truth
Figure 16: Effect Self-Ocelusion. Comparison compositions without and with self-occlusion modeling. Incorporating self-occlusion further enhances the composition's 's harmony and realism.
B.6 MULTI-OBIECT COMPOSITION
(a) Without inter-object occlusion
(b) With inter-object occlusion
Figure 17: Multi-object Composition Scene. Our approach enables multi-object composition
through sequential placement.
19

---

Published as a conference paper at ICLR 2026
Our approach supports multi-object composition through sequential procedure. Although simul- tancous placement of multiple objects is not supported, we address this by inserting objects one after another. After placing the first object, the modified scene is rendered from multiple views and reconstructed using Stage of our method. This reconstructed scene then serves as the base scene for inserting the next object. Figure 17 demonstrates that this sequential strategy allows method to successfully compose multiple objects and model partial inter-object occlusion between them.
B.7 SHADOW FROM MULTIPLE LIGHT SOURCES
(a) With a light source
(b) With two light sources
Figure 18: Shadows from Multiple Lights. Our method correctly composites shadows in overlap- ping regions, producing darker, additive shadows when multiple light sources are present.
Our method naturally handles scenes with multiple light sources, producing shadows that cor- rectly composite in overlapping regions. To validate this, we conducted an experiment the Ket- fle_in_Room scene by manually adding a secondary point light to the predicted environment map.
After introducing the additional light, the shadows cast by the original light become less pronounced due to the increase overall illumination. In areas where shadows from both light sources overlap, darker, composite shadow emerges, demonstrating the expected additive effect of multiple light sources. The results are illustrated Figure 18.
C MORE RESULTS ON RECONSTRUCTION
C.I RECONSTRUCTION PERFORMANCE ON SYNCOM-OBIECT
We also conducted experiments for NVS and albedo, and PSNR, SSIM, and LPIPS for relighting, along with training time. For NVS and albedo, only PSNR is presented due to the negligible differences in SSIM and LPIPS across most methods. Results are presented in Tables 4. Our approach achieves accuracy comparable to SOTA methods, and some cases slightly surpasses them, while enabling 2x faster reconstruction.
SynCom object dataset. We report MAE for normals, PSNR
Table 4: Quantitative results the SynCom- dataset. Our method achieves accuracy comparable to the state of the art while delivering over 2x higher efficiency.
Normal MAET
NVS
Albedo PSNR
Relighting SSIM
Training
Method
PSNR
PSNR1
LPIPS.
Time(min)]
GS-IR (Liang et al., 2024b)
1.801
40,534
27. 27.409
22.150
0.951
0.041
15.51
R3DG (Gao al., 2024)
1,510
41.364
27.992
28.646
0.967
0.036
40.62
IRGS (Gu al., 2025)
0.990
39,250
28.596
29.752
0.970
0.039
21.99
Ours
1.044
39.700
28.840
29.601
0.975
0.027
7.42
C.2 ABLATION ON SOPS INITIALIZATION
To assess the robustness of the heuristic offset used for SOPs initialization, we perform an ablation study by applying different placement offsets (0%, 1%, and 2% of the object size) the Toy scene
20

---

Published as a conference paper at ICLR 2026
GS-IR
R3DG
IRGS
Ours
GT
Render
Albedo
N/A
Roughness
N/A
N/A
N/A
Metallic
Env.map
N/A
AO
N/A
Indirect
Relit. I
Relit.2
Figure 19:
Qualitative Results on TensoIR Dataset. Our method achieves effective relightable object reconstruction, enabling satisfactory relighting. In contrast, GS-IR's 's oversimplified lighting assumptions lead to inaccurate decomposition. R3DG lacks supervision on indirect lighting, leading to unnatural indirect lighting and bright spot artifacts in relighting.
21

---

Published as a conference paper at ICLR 2026
from the SynCom dataset. Quantitative results summarized in Table 5. It can be observed that the 1% offset consistently yields the best performance across all metrics, including material albedo estimation, rendering fidelity, and relighting quality.
In addition, we include visual comparisons of relighting and ambient occlusion under different off- sets in Figure 20. The visual results demonstrate that the 0% offset leads to noticeable light leakage in both occlusion and relighting, whereas the 1% setting produces clean and stable outcomes. Based on these findings, adopt the 1% offset strategy uniformly in all experiments reported in this work, and we did not observe any notable failures under this configuration.
Relighting (0%)
AO (0%)
Relighting (1%)
AO (1%) Relighting (2%)
AO (2%)
Figure 20: Comparison of relighting and ambient occlusion (AO) under different SOPs initial- ization offsets. The 0% offset introduces noticeable light leakage, while the 1% setting produces clean and stable results.
Table 5: Quantitative ablation study on the SOPs initialization offset.
Offset
Albedo PSNR
Rendering PSNR
Relight PSNR
0%
33.222
37.204
30.449
1%
33.304
37.265
31.324
2%
33.130
37.202
31.281
D DETAILS OF LIGHTING ESTIMATION
D.1 QUESTION STATEMENT
Since the scene's geometry and radiance field have already been reconstructed, and our task only re- quires local illumination around the inserted object for relighting and shadow casting, the inherently complex problem of lighting estimation in the full scene can be significantly simplified. Specifi- cally, reformulate it environment lighting estimation at designated location, conditioned the reconstructed 3D Gaussian radiance field. Our solution proceeds in three steps:
Panoramic Projection. The reconstructed radiance field is projected onto a panoramic sphere centered at the object placement location. This is achieved through 360 ray tracing from the location, producing partial local environment map comprising an RGB image, normal map, and an alpha mask that distinguishes reconstructed from missing regions.
Panorama Completion. To complete the partial panoramas, we fine-tune Stable Diffusion 2.1 (Rombach al., 2022). In addition to the RGB images and mask channels, the corre- sponding normal maps are also fed into the network to provide geometric guidance during inference, which yields slightly improved performance, shown in Table
HDR Expansion. Finally, the completed panoramas are extended to high dynamic range. We adopt an EV-conditioned prompting strategy (Phongthawce et al., 2024), fine-tuning the model to generate LDR panoramas exposure values of -5, -2.5, and 0. These outputs are subsequently fused into a single, comprehensive HDR environment map.
D.2 TRAINING DATASET
As discussed in Section D.I, 1, the main challenge of our task lies in Panorama Completion. Compared with conventional natural image completion, our task has two distinctive characteristics:
Panoramic input: the input images panoramas rather than ordinary perspective images.
22

---

Published as a conference paper at ICLR 2026
Tex12L.ight (Indoor)
Poly Haven (Indoor)
Laval (Indoor)
Tex(2Ligh(Qurdooff
Poly Haven(Qurdoor)
Our Mask
Figure 21: HDR environment maps and a Gaussian mask from our training dataset. We lever- age publicly available environment maps from PolyHaven and synthesize a large, diverse set of HDR environment maps using the text-to-light model, Text2Light (Chen et al., 2022). Gaussian masks are generated from publicly reconstructed Gaussian scenes and exhibit artifacts typical of Gaussian radi- ance fields, such as needle artifacts, which are similar to those encountered in practical applications. This carefully curated dataset well suited for training our panorama completion model.
Gaussian-derived masks: the masks traced from the Gaussian and exhibit unique Gaussian characteristics, such as needle-like artifacts.
To better adapt the network to this setting, our fine-tuning step relies on two key resources: (I) sufficiently large and diverse collection of panoramic RGB images, and (2) a substantial set of masks generated from Gaussian scenes. Training samples are constructed by pairing each RGB image with mask. Figure 21 illustrates representative panoramas and masks from dataset. In the following, we provide a detailed account of how these data are obtained.
HDR enviromment maps Following DiffusionLight (Phongthawee et al., 2024), we used Text2Light (Chen et al., 2022) to generate numerous HDR panoramas from text descriptions. We use Large Language Models (LL.Ms) to generate textual descriptions. To ensure diversity and minimize prompt similarity, we adopt multi-IILM strategy. Specifically, we employ three distinet L.LMs: ChatGPT Grok and Gemini Each model produces 500 descriptions for indoor scenes and 500 for outdoor scenes. In addition, incorporate HDR environment maps from public repositories such as Polyhaven and the Laval Indoor HDR (Gardner et al., 2017).
After curation and removing corrupted files, final collection totals 5991 HDR environment maps, with 3938 indoor and 2053 outdoor scenes. We then randomly partitioned this collection into train- ing and valid sets following 8:2 ratio, resulting in 4792 and 1199 maps, respectively. For each map, we further obtain corresponding normal maps by using an off-the-shelf StableNormal (Ye et al., 2024a) model infer these cues.
Mask from Gaussian Splatting Scenes We process various publicly available scenes (Barron et al., 2022) to obtain their 2D Gaussian representations and randomly select multiple points within them. From each point, we perform 360 panoramic trace of the scene's 's radiance field. We then retain only the resulting alpha maps as masks for our dataset construction.
Assuming the scenes Manhattan-alin reconstructed Gaussian scenes often exhibit missing data in the upper hemisphere. To make our training data more representative of this scenario and emphasize Gaussian artifacts, we apply random-region dropout strategy during tracing. Specifically, before tracing, we exclude Gaussian primitives within randomly oriented solid angle from the trace point. This generates sharper and more challenging mask boundaries, better reflecting real-world conditions.
with the Z-axis pointing upwards, panoramic views of
'nttps: ://chat. openai................... iinaii.e com/ pottts::/x.alt "https://dens idd goog1 e/model gem /i/
23

---

Published as a conference paper at ICLR 2026
D.3 TRAINING DETAILS
Our Lighting Estimation model is fine-tuned from the pre-trained Stable Diffusion 2.1 (Rombach et al., 2022). We employ an EV-conditioned prompting strategy (Phongthawee et al., 2024) gen- erate environment map completions corresponding to specified Exposure Values (EVs). Finally, the outputs generated under different EVs (c.g.. -2.5. 0) are fused into a single, comprehensive HDR -2.5,
environment map. Our training process is divided into two main steps. The first step aims to adapt the model for LDR panorama completion. Then the second step further fine-tunes the model to be conditioned on EV prompts, enabling direct control over the exposure of the generated results.
Step I: Fine-tuning on LDR Panorama Completion. In the first step, the model input is formed by concatenating the partial RGB image, its corresponding normal map and alpha mask, a resolu- tion of 512 x 1024 pixels. To guide the completion process, we use fixed prompt: "A realistic 360 degree panoramic [indoor/outdoor] scene, overexposed, bright". We manually specify the indoor or outdoor token based on the scene type to help the model better distinguish and adapt to the sig- nificant content disparities between indoor (e.g., ceilings and lights) and outdoor (e.g., sky and sun) scenes, particularly in the upper regions of the panorama.
To enhance the model's generalization and robustness, we apply two data augmentation techniques to the input HDR data: random rotation and random EV adjustment. Note that the EV adjustment here serves only as data augmentation. The model is trained to produce an output with the same EV as the augmented input, with no explicit EV conditioning, We train this step by tasking the model with a v-prediction objective, minimizing an L1 loss between the prediction and the ground truth.
Step 2: EV-Conditioned Fine-tuning The core of the second step is the introduction of EV- conditioned prompting strategy. We dynamically generate prompts for arbitrary EV by linearly interpolating the text embeddings of two base prompts. Specifically, we define a base prompt em- bedding & for EV=0 (the prompt used in step 1) and a dark prompt embedding & for EV = -5 (by replacing "overexposed, bright" with "underexposed, dark"). The target prompt embedding for given EV is then computed as E.w- & + (ev/ EV (Ex - E.).
To create the training pairs, we first take an original HDR environment map Tay and apply a random EV shift to it, resulting in an augmented map Tuy with EVaug- Then, we apply a second, randomly sampled conditional exposure value, denoted as FVg., and generate the corresponding ground truth image ler for supervision. The model is trained to adjust the exposure from the input EV to the target VV All other training settings remain the same as in step
Implementation Details We train our model using the AdamW optimizer with 81 0.9, 82 0.999, and a weight decay of 0.01. The learning rate follows a constant schedule with a linear warmup of 500 steps. We train step for 65 epochs with a learning rate of 1 x 10 and fine-tune step 2 for 20 epochs with a reduced learning rate of 5 x 10-6, The batch size is for both steps. All models are implemented in PyTorch and run NVIDIA A6000 (48GB) GPUs. step training uses 4 GPUs and takes approximately 6.5 hours, while step 2 runs on 2 GPUs for about 3 hours.
D.4 ABLATIONS
We conduct ablation studics on our lighting estimation method. First, we investigate the impact of scene completeness, measured by the effective area of the mask, on lighting estimation. Sec- ond, we examine the effect of including surface normal information as an additional input channel. We adopt the widely used evaluation protocol with an array of spheres from prior lighting estima- tion works (Gardner et al., 2017; 2019). The evaluation metrics include scale-invariant Root Mean Square Error (si-RMSE), Angular Error, and normalized RMSE.
Ablations on Scene Completeness. Following the method described in Section D.2, we generate 5,000 masks and group them according to the proportion of their valid coverage area. Specifically, the masks are divided into three groups: 40-60%, 60-80% and 80-100%, and the statisties of each group are analyzed. Masks with less than 40% coverage are excluded, as such cases typically correspond to insufficient scene images suboptimal object placement, making them unlikely to reflect practical usage. We then evaluate the model's performance across these three groups using the protocols described above. The results in Table show that the accuracy of lighting estimation
24

---

Published as a conference paper at ICLR 2026
Table 6: Ablations on Lighting Estimation. Performance improves with higher mask coverage, and
incorporating the normal map enhances lighting estimation by providing explicit geometric cues.
Scale-invariant RMSE 4
Angular Error
Normalized RMSE &
coverage 40-60%
0.066
6.037
0.103
coverage 60-80%
0.060
5.599
0.089
coverage 80-100%
0.052
4.967
0.074
w/o Normals with Normals
0.067
0.061
0.100
0.065
0.058
0.098
consistently increases with higher effective coverage. This trend is expected, since larger coverage provides more informative scene data for the model infer illumination. These findings suggest that, in practical usage, capturing the scene completely as possible is beneficial for achieving more reliable and accurate lighting estimation.
Ablations Geometric Information. We compare two model settings: our full model, where the UNet input is concatenation of the RGB image, normal map, and alpha mask; and an ablated version (w/o Normal Map), where the normal map is omitted from the input. To ensure fair com- parison, both models were trained from scratch using identical training data and hyperparameters. Their performance is then evaluated using the same protocols and metries. As shown in Table 4, incorporating the normal map leads to more accurate lighting estimation, which can be attributed to the additional structural information provided by the normals.
E LLM USAGE STATEMENT
We use ChatGPT, large language model developed by OpenAI, polish the language of our manuscript. The model is not involved in research ideation, experimental design, data analysis, or the drawing of conclusions. The authors take full responsibility for the content of the paper.
25


Reviews:
[{"review_id": "tgKx6jeE0M", "reviewer": "Reviewer_7Qbv", "summary": "This work presents a novel pipeline for realistic 3D object-scene compositions, which is formulated as composing given scene and object multi-view images into a harmonious and visually-natural 3D representation. The main contributions of this work are surface octahedral probes (SOP) and local lighting completion. These designs empower high-quality relightable object reconstruction and object-scene composition with significantly improved efficiency.", "questions": "1. It is recommended to add ablations or verifications on the SOP initializations. How the offset affects the final results and whether per-scene tuning is needed should be clarified.\n2. The memory consumption introduced by SOPs should be shown, as thousands of probe operations are executed in this implementation.\n3. How sensitive are results to SOP count and probe texture resolution? Is there a principled way to choose probe density per scene scale?\n4. A deduction on the limit of the proposed method regarding the assumptions is favorable. For example, what is the threshold for a normal pipeline proceeding, and how do these assumptions affect the method’s effectiveness?\n5. Is it possible to incrementally update the SOP as the object or camera moves to mitigate the calculation of re-caching for each editing? This may further enhance the pipeline completeness.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The proposed method is well-motivated. This work targets two long-standing obstacles in the object-scene composition task: relightable object reconstruction and scene lighting estimation, and introduces specific designs to tackle these challenges.\n2. The SOP design mitigates the reliance on per-point ray tracing during optimization and rendering, thus improving the inverse rendering efficiency, leading to (near) real-time relightable rendering speed.\n3. The local lighting completion builds upon the insight that only part of the scene interacts with the to-be-composed object, thus focusing on only the salient parts of the representation.\n4. The proposed pipeline achieves superior performance from both the quantitative and qualitative evaluations while demonstrating high rendering efficiency, verifying its optimal effectiveness-efficiency trade-off.", "weaknesses": "1. The SOP initialization raises concerns. It is assumed that the SOPs are initialized by ray tracing and then optimized in a supervised manner. This initial ray-tracing step and progressive SOPs require careful designs to avoid light leaks and placement offsets. Although a heuristic solution is provided for the offset = 1% object size, its robustness is not clearly verified.\n2. Assumption on the pipeline settings. The proposed pipeline focuses on scenes with low or moderate occlusions (<40%), and builds on assumptions that the object is small relative to the scene and its placement affects only its own appearance and nearby regions. These assumptions are fundamental to the overall pipeline and prompt an efficient solution. However, it is not clear whether this assumption accounts for most real-world cases, and it is interesting to investigate how the method performs for hard cases and failure cases.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4h2iSWON2H", "reviewer": "Reviewer_safG", "summary": "This paper presents a realistic 3D object-scene composition pipeline that achieves plausible shadows and lighting effects in real time.\nThe framework consists of three major stages:\n(1) Reconstruction, where both the object and scene are represented using Gaussian splatting;\n(2) Editing, which includes environment lighting estimation and Surface Octahedral Probe (SOP)-based occlusion caching; and\n(3) Rendering, performing relighting and shadow synthesis for seamless composition.\nThe proposed system is efficient and demonstrates high-quality visual realism on various examples of single object insertion and relighting.", "questions": "1. For the scene reconstruction, are the 2D Gaussians also equipped with albedo, roughness, and metallic attributes as described in lines 155–161? If so, how are these optimized since they do not appear in Step 2? For the object reconstruction, is the RGB buffer C still optimized and used? If so, how does it involve in the rendering equation?\n2. Eq. 2 seems incorrect, any typos?\n3. Is the self-occlusion term computed in Eq. (8) reused in Section 3.3 (``Rendering'')? Does the relighting step re-bake indirect illumination under new lighting conditions? From Figure 7 and explanations in Section 3.3, it seems these two terms are not re-used at the relighting stage.\n4. What if there are multiple light sources in the scene (may correspond to various peak values in the environment map)? Will the generated shadow composite of layered shadows with darker overlapped regions?\n5. Does a mismatch between the captured object’s original environment map and the target scene environment influence the final performance?\n\nI will raise the score if the major concerns are addressed.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. The proposed framework demonstrates good performance and maintains efficiency, outperforming several baselines.\n2. The proposed SOP structure is convenient, enabling real-time shadow and lighting computation.\n3. The paper is well organized and easy to follow.", "weaknesses": "1. Self-occlusion or ambient occlusion effects on the inserted object are barely noticeable (e.g., Figure 7), suggesting that the occlusion modeling may not be effective enough. (Please also see question 3).\n2. The experiments mostly showcase single-object insertions. It remains unclear how the system performs when inserting multiple objects, or when an object is placed freely in the scene (for example, partially under the shadow area). More results could further demonstrate the effectiveness of the proposed method. \n3. The performance of object inverse rendering depends on the input images. The synthetic dataset from the simulator largely aligns with the proposed method, which may be biased. More discussion about real-data capture conditions or constraints would improve clarity and generalizability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JjCDMLYAf7", "reviewer": "Reviewer_5io7", "summary": "This paper presents a novel object-scene composition framework, ComGS, which consists of an efficient 2DGS-based inverse rendering method and a diffusion-based lighting estimation model. The proposed inverse rendering pipeline utilizes Surface Octahedral Probes to cache the indirect illumination and occlusion, eliminating the need for costly ray tracing . Experiment results are provided to demonstrate the good performance of the framework, highlighting the effectiveness of SOPs.", "questions": "1. Is there a way to obtain occlusion probes even when the scene is represented by a single image? Or are there other methods to obtain the shadow?\n2. Could you provide the performance of two extra baselines MV-CoLight and GI-GS on your SynCom-Objectdataset?\n\n[MV-CoLight, NIPS'25](https://github.com/InternRobotics/MV-CoLight)\n\n[GI-GS, ICLR'25](https://github.com/stopaimme/GI-GS/tree/master)", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. This paper proposes a novel inverse rendering framework. Unlike previous methods that use expensive ray tracing to obtain indirect lighting and occlusion, this method uses probes to store indirect lighting and occlusion, significantly improving efficiency.\n2. The proposed lighting estimation model conditioned on rendered partial panorama envmap from GS demonstrates good performance.\n3. Extensive experiments show that the method achieve better performance than inverse-rendering and generative baselines.", "weaknesses": "1. It seems that the proposed methods cannot be used in single image scenario. During editing stage, since the object is inserted into the scene, the occlusion probes near the object should be baked first.  But if the scene is represented by single image, it is impossible to use GS to reconstruct it, which means we cannot obtain the occlusion SOPs.\n2. Even the position of probes are generated from the surface point clouds using FPS, the sampled probes may still be within the Gaussians’ effective ranges (mixing with the Gaussians), which will cause light leakage. But I think there shouldn't be many such examples, so this isn't a big problem.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QHqaj6iRZU", "reviewer": "Reviewer_4c68", "summary": "This paper proposes ComGS, a Gaussian-splatting-based framework that performs inverse rendering for a scene and supports inserting new objects into the scene with consistent lighting appearance. Main contributions include (1) Surface Octahedral Probes (SOPs) for capturing indirect lighting and occlusion, (2) a diffusion model to estimate HDR 360° lighting based on the Gaussian scene, (3) the occlusion caching technique to efficiently cast shadows of the new object without ray tracing. Experimental results demonstrate superior quality to baseline.", "questions": "1. Is the proposed 360° environment map lighting diffusion model (Sec 3.2.1) similar to the \"Diffusion posterior illumination\" paper? If so, the author should clarify this and weaken the contribution claim on it.\n2. Can authors provide a clarification on whether the method can handle non-Lambertian scenes? Experimental results are also preferred.\n3. How accurate are the estimated cast shadows by the occlusion caching compared to the ray-traced ground truths, given that Eq. 15 may introduce significant approximation error?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The idea of using light probes for the inverse rendering task is novel. Light probes are a technique in traditional graphics rendering that cache indirect lighting to avoid costly ray tracing, which is well-suited to the inverse rendering task for performance gains.\n2. The proposed occlusion caching further avoids costly ray tracing for the inserted object. Ray tracing has been a widely used but costly technique in recent years' inverse rendering paper, and this paper successfully bypasses it while preserving quality.\n3. The inverse rendering community can benefit from the SynCom Dataset created by this paper.", "weaknesses": "## Major\n1. Missing citations. The related work section only introduces NeRF- and GS-based inverse rendering methods, while a rich literature of mesh-based inverse rendering methods has also emerged in recent years and is worth mentioning, for example,\n```\n@article{hasselgren2022shape,\n  title={Shape, light, and material decomposition from images using monte carlo rendering and denoising},\n  author={Hasselgren, Jon and Hofmann, Nikolai and Munkberg, Jacob},\n  journal={Advances in Neural Information Processing Systems},\n  volume={35},\n  pages={22856--22869},\n  year={2022}\n}\n@inproceedings{dai2025inverse,\n  title={Inverse Rendering using Multi-Bounce Path Tracing and Reservoir Sampling},\n  author={Dai, Yuxin and Wang, Qi and Zhu, Jingsen and Xi, Dianbing and Huo, Yuchi and Qian, Chen and He, Ying},\n  booktitle={The Thirteenth International Conference on Learning Representations},\n  year = {2025},\n  url = {https://openreview.net/forum?id=KEXoZxTwbr}\n}\n```\nThe idea of using a diffusion model to estimate 360° environment map lighting is also explored by previous work\n```\n@article{lyu2023diffusion,\n  title={Diffusion posterior illumination for ambiguity-aware inverse rendering},\n  author={Lyu, Linjie and Tewari, Ayush and Habermann, Marc and Saito, Shunsuke and Zollh{\\\"o}fer, Michael and Leimk{\\\"u}hler, Thomas and Theobalt, Christian},\n  journal={ACM Transactions on Graphics (TOG)},\n  volume={42},\n  number={6},\n  pages={1--14},\n  year={2023},\n  publisher={ACM New York, NY, USA}\n}\n```\n2. Lambertian assumption. The training scheme regularizes the roughness to be near 1 and metallic to be near 0 (Eq. 12). While this may improve training stability, it also limits the expressivity of the inverse rendering pipeline, especially for scenes with specular reflections. Also, Eq. 15 further approximates the rendering equation with Lambertian assumption, which may introduce bias for non-Lambertian scenes.\n3. Paper writing. (1) The proposed Surface Octahedral Probes are one of the main contributions of this paper, but I found the corresponding description to be too concise (L224-235). The authors should expand the corresponding paragraph in the revision. (2) The paper does not contain a limitation subsection.\n\n## Minor\n1. Monte Carlo rendering. Strictly speaking, the rendering estimator in Eq. 7 with a low-discrepancy sequence should be a deterministic quasi-Monte-Carlo scheme, compared to traditional Monte Carlo methods that require stochastic importance sampling. Despite avoiding noise and variance, the shortcoming of such a scheme is the lack of ability to capture high-frequency details, such as sharp specular reflections [1]. This weakness coincides with the Lambertian assumption weakness I mentioned before.\n2. The inverse rendering results on TensoIR Dataset (Fig 14) still contain significant artifacts, e.g. the albedo map. It would also be preferred to show the inverse rendering results on the scene-level data.\n\n[1] Zhu et al. \"Learning-based inverse rendering of complex indoor scenes with differentiable Monte Carlo raytracing.\" SIGGRAPH Asia 2022 Conference Papers. 2022. Fig 4", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)