MaterialRefGS: Reflective Gaussian Splatting with Multi-view Consistent Material Inference
Authors: Wenyuan Zhang
Jimin Tang
Weiqi Zhang
Yi Fang
Yu-Shen Liu
Zhizhong Han
Venue: NeurIPS 2025 poster ()
Keywords: 3D Gaussian Splatting
Novel view synthesis
Physically based rendering
Deferred shading
Primary area: applications

Abstract: Modeling reflections from 2D images is essential for photorealistic rendering and novel view synthesis. Recent approaches enhance Gaussian primitives with reflection-related material attributes to enable physically based rendering (PBR) with Gaussian Splatting. However, the material inference often lacks sufficient constraints, especially under limited environment modeling, resulting in illumination aliasing and reduced generalization. In this work, we revisit the problem from a multi-view perspective and show that multi-view consistent material inference with more physically-based environment modeling is key to learning accurate reflections with Gaussian Splatting. To this end, we enforce 2D Gaussians to produce multi-view consistent material maps during deferred shading. We also track photometric variations across views to identify highly reflective regions, which serve as strong priors for reflection strength terms. To handle indirect illumination caused by inter-object occlusions, we further introduce an environment modeling strategy through ray tracing with 2DGS, enabling photorealistic rendering of indirect radiance. Experiments on widely used benchmarks show that our method faithfully recovers both illumination and geometry, achieving state-of-the-art rendering quality in novel views synthesis. Project Page: https://wen-yuan-zhang.github.io/MaterialRefGS.

MaterialRefGS: Reflective Gaussian Splatting with Multi-view Consistent Material Inference
Wenyuan Zhang Jimin Tang' Weiqi Zhang'
Yi Fang²
Yu-Shen Liu' Zhizhong Han
School of Software, Tsinghua University, Beijing, China" Center for Al and Robotics (CAIR), NYU Abu Dhabi, UAE Department of Computer Science, Wayne State University, Detroit, USA (zhangwen21, 2anngjm24,2wq222 2maii) @mails. tssighuun edu. cn yfangenyu. edu liuyushendtsinghun edu.
h31 h312h@wayme.eds edu
3DGS-DR
Ref-Gaussian
Ours
Ous Mesh
RGB
Nome
Diffuse Color
Sperular Color
Final Color
Ours Mesh
Ref Ref- Cauesiin
Outss
Figure I: We highlight our novel view synthesis results on real-world scenes with complex reflection effects. Our MaterialRetGS outperforms the state-of-the-art methods in producing photorealistic renderings, disentangling physical materials, and recovering accurate scene geometry.
Abstract
Modeling reflections from 2D images is essential for photorcalistic rendering and novel view synthesis. Recent approaches enhance Gaussian primitives with reflection-related material attributes to enable physically based rendering (PBR) with Gaussian Splatting. However, the material inference often lacks sufficient constraints, especially under limited environment modeling, resulting in illumi- nation aliasing and reduced generalization. In this work, we revisit the problem from a multi-view perspective and show that multi-view consistent material in- ference with more physically-basee environment modeling is key learning accurate reflections with Gaussian Splatting. To this end, we enforce 2D Gaus- sians to produce multi-view consistent material maps during deferred shading. We also track photometric variations across views identify highly reflective
Equual contribution.
'The corresponding author is Yu-Shen Liu.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

regions, which serve as strong priors for reflection strength terms. To handle indirect illumination caused by inter-object occlusions. we further introduce an environment modeling strategy through ray tracing with 2DGS, enabling pho- torealistic rendering of indirect radiance. Experiments on widely used bench- marks show that our method faithfully recovers both illumination and geometry, achieving state- of-the-art rendering quality in novel views synthesis. Project Page: https: / /wen- yuan- yuan-zhang gi github. Mo/MaterialReett.
Introduction
Learning scene appearance representations and recovering unseen views from multiple posed RGB images has been a long-standing task in computer vision and graphics [22, 36, 18, 64]. Recent advances in Neural Radiance Fields (NeRF) [36] leverage volume rendering to learn implicit scene representations for novel view synthesis. More recent efforts have been made by learning explicit 3D Gaussians (3DGS) [22] to achieve real-time rendering through differentiable splatting procedure. Despite achieving photorealistic synthesis, 3DGS shows limited performance when confronted with complex reflective environments. This limitation arises from the contradiction between the simplistic geometric representations and the intricate shading mechanisms of real-world objects.
To tackle this problem, recent methods typically separate the rendering color into diffuse and specular components, and adopt inverse rendering frameworks [55, 11] to learn illumination decomposi- tion through physically based rendering [40]. They endow each Gaussian primitive with learnable reflection-relate properties, such as metallic and roughness [20, 24]. A splatting pass rasterizes these attributes into screen-space material maps [22, 18]. followed by lighting pass that evaluates Bidirectional Reflectance Distribution Function (BRDF) [4] using material maps and environment lighting to synthesize the final image. This two-stage pipeline is known as deferred shading- -ased PBR [24, 56, 66]. It decomposes the view-dependent reflection effects into view-indepen -ed-indeenndentt material properties by considering how the light interacts with the objects and environments, thereby improv- ing the fidelity of novel views. However, the illumination decomposition poses several optimization challenges. First, inferring material properties from multi-view images is an ill-posed problem. All material parameters are optimized only through photometric loss after complex light transport, so multiple combinations of lighting and materials can explain the same pixels, which often leads to sub- optimal illumination decomposition [46, 58]. Second, the view-dependent behavior of 3D Gaussian representations conflicts with the goal of learning view-independent material properties. When the same physical attribute yields inconsistent appearances in different viewpoints, the BRDF struggles to infer accurate reflections from ambiguous observations, resulting in aliasing and degenerated illumination decomposition [10, 31].
To resolve these issues, we propose novel approach that learns illumination decomposition for modeling reflections with 2DGS through multi-view consistent material inference. We first enforce Gaussians produce multi-view consistent material buffers based their physical attributes. This is achieved by aligning the projections of geometric surfaces on material maps from different views. We find that this constraint significantly improves illumination decomposition by limiting the view- specific overfitting. To better facilitate this process, we track photometric variations on object surfaces along the camera trajectory and quantify these variations as reflection scores. A spatial reflection fusion module is then applied to aggregate these per-view reflection scores into a multi-view consistent reflection strength prior. This prior is subsequently used as a supervision for the reflection strength attribute, i.c., metallic.
In addition, we observe that secondary reflection effects caused by inter-object occlusions often lead to degraded novel view synthesis. To address this, we propose an improved environment modeling strategy via differentiable ray tracing, which combines splatted indirect radiance and queried direct radiance with an on-the-fly fly estimation of occlusion probability. This approach effectively provides physically grounded signals in occluded regions, enabling more realistic indirect illumination. Our numerical and visual evaluations on widely used benchmarks demonstrate our superiority over the latest methods in terms of material inference and novel view synthesis. Our contributions summarized as follows:
We propose a novel approach to modeling reflections through Gaussian Splatting with multi- view consistent material inference, including multi-view material consistency constraint and

---

reflection strength prior supervision. Our approach provides new perspective for modeling reflections through physically grounded illumination decomposition.
We introduce differentiable environment modeling strategy through 2DGS based ray tracing, which enhances photorealistic rendering of indirectly illuminated regions caused by inter-object occlusions.
We achieve state-of -feee--rrr performance of novel view synthesis both in numerical results and visual comparisons widely used benchmarks.
2 Related Work
2.1 Novel View Synthesis
The task of novel view synthesis aims to predict unseen views of scene from a set of posed RGB
images. Traditional methods typically rely image interpolation [45] inpainting [2] to generate novel views. With the rapid development of deep learning [70, 69, 67, 68, 5, 39, 28, 52, 32]. novel view synthesis has gradually shifted toward learning- (NeRF) [36, 62, 63, 65, 16, 51] pioneers this task by learning a mapping from 5D coordinates to volume densities the scene representations. More recently, 3D Gaussian Splatting (3DGS) [22] has emerged as a new paradigm for real-time rendering by rasterizing Gaussian ellipsoids into images in a splatting manner. Various extensions support diverse scales and scenes through novel data structures such as hierarchies [23, 34] and octrees [44]. Others address sparse-view challenges by incorporating geometric priors [7, 19, 27, 17]. Beyond static scenes, some works also explore 3DGS for dynamic scenes [50], semantic-aware manipulation [42], and content generation [71, , 60, 9]. Recent efforts aim extract high-quality surfaces from 3DGS by flattening 3D Gaussians into 2D disks [18] and leveraging differentiable kernels to rasterize them into images. To better align Gaussians with object surfaces, regularization strategies such as depth-normal consistency [18, 15] and neural gradient supervision [61, 29, 35] are applied. In our work, we adopt 2D Gaussians as the foundational representation due to their effectiveness in modeling surface geometry and normals.
approaches. Neural Radiance Fields
2.2 Modeling Reflections in NeRF and 3DGS
The view-dependent color representations used in original NeRF [36] and 3DGS [22], such as neural networks or spherical harmonics, struggle to capture high-frequency specular reflections that are commonly observed in real-world scenes. Existing solutions typically decompose the outgoing radiance into diffuse and specular components and blend them using learnable weights. To better model the specular reflections, some methods introduce directional encodings like Integrated Directional Encoding [47] and Gaussian Directional Encoding [26]. Other approaches extract accurate meshes to provide reliable normal for reflection modeling [33, 48, 13, 25]. Recent advances in 3DGS [22] offer new perspectives for addressing this challenge. Inspired by inverse rendering, Relight3DGS [11] assigns each Gaussian with physical properties such as metallic and roughness, and performs PBR on the Gaussians to synthesize the final image. Recent studies have proven that rendering Gau-Gansiinn illumination attributes into material maps followed by deferred shading- based PER [24, 56, 66, 20] yields better performance than per-Gaussian shading [11]. To improve environment interactions, some methods develop ray tracing techniques for Gaussians [37, 14, 53). However, these methods primarily focus on per-view light-material interactions and neglect the globally consistent geometric information inherent in the multi-view settings. To fill in this gap, propose leveraging multi-view cues to facilitate the disentanglermen of material properties, enabling more accurate and realistic modeling of reflections in 3D Gaussians. Notably, our method not equivalent to inverse rendering. Our goal to model specular color through illumination decomposition, while relying on the Gaussians to render the diffuse color. In contrast, inverse rendering methods evaluate all lighting effects through BRDF, making them more suitable for quantitatively evaluating material decomposition and for relighting tasks. However, due to the complexity of learning diffuse component, these methods are limited to simple object-centric scenes.
3 Method
Given a set of posed RGB images (7,}²1 that represents scene with high reflections, we aim to synthesize novel image from an unseen viewpoint. We learn set of 2D Gaussians as the scene
3

---

representations. We begin by introducing the preliminaries (Sec. 3.1), and then describe our multi- view material inference strategy (Sec. 3.2) and environment modeling strategy (Sec. 3.3). Finally, we detail the optimization procedure (Sec. 3.4). An overview of our method is provided in Fig. 2.
3.1
Preliminary
3D Gaussian Splatting (3DGS) [22] has become paradigms for learning 3D representations from multi-view images. A scene is represented by Gaussian functions (G.),1 with attributes like mean E; opacity o; and scaling 8;. We also attach several reflection-related material attributes to Gaussians, including diffuse color Ca R3, alhedo E R, metallic R and roughness R. We can then rasterize these Gaussians into images using
2 O
II j=1
-1)
(I)
where 0j Ps are the opacity and screen-space probability [72] of the i-th Gaussian, respectively, and y denotes selected attribute of G,. By choosing different attributes such Cdi. ag m, or r; th, we can render the corresponding material maps & V A L M y respectively. To facilitate surface reconstruction, 2DGS [18] flattens each 3D Gaussian into 2D disk setting scaling dimension to zero. We adopt 2DGS as our base representation for better surface and normal alignment.
Our deferred shading-based PBR adopts simplified version of the Disney BRDF model [4]. Given viewing direction wo, the rendered color on the ray-surface intersection can be computed by
dwio) = =(1 - m))a + [uswon a, m. r, rt), Lu(wos a, (,,, = [x(w (w) nidwuss
DFG
fo(wi,wo) = [W::n))[w.
where Wi, Lg. D.F.F D. G denote the incident direction, normal, outgoing specular radiance, BRDF term, normal distribution function, Fresnel term and shadowing wrwwwinngmssiinggg term, respectively. Since computing the integral of L, over the upper hemisphere Q is computationally expensive, we adopt the split-sum approximation [38, 56, 31], which separates the integral into two components,
S fa(wi, (.(wx,w))(e (ws n)dws [ Lo(ws) ((wi, (ws -.n.nn))e n)ddw::
La(wo a, m. T, n)
where the first term can be precomputed using a, m, r and stored in look-up table. The second term can be queried from a set of learnable environment cubemaps using reflected direction and r.
Similar to reflective Gaussian methods [20, 56), we decouple the diffuse component ca and the specular component L.,, assigning the prediction of ca to Gaussian rasterization. Unlike classical graphics pipelines that jointly infer diffuse and specular terms from albedo and roughness [31, 14], we find that such a design significantly increases optimization difficulty, especially in complex real- world scenes. Moreover, since the diffuse component is relatively insensitive to viewing direction, delegating its prediction to the Gaussians allows us to better disentangle reflection effects from illumination.
3.2 Multi-view Consistent Material Inference
Current methods model scene reflections by learning a set of Gaussians associated with material properties through PBR. The underlying assumption is that view-dependent specular variations can be disentangled into view-independent material attributes, while evaluating the final view siew-dependent appearance can be deputed to BRDF. However, this assumption often breaks down in practice, as shown in Fig. 8 (a), where the learned material maps exhibit significant discontinuities and inconsistency across different views. Since Gaussians cover different pixels and contribute varying weights across viewpoints during alpha blending, material parameters exhibit significant inconsistency from different perspectives. This inconsistency hampers accurate illumination decomposition, as the BRDF struggles to infer global physical reflectance effect from such inconsistent material

---

Euviromment Modeling through Ray Tracing (Sec. 3.3)
Mask
Unoecladdod
Docladed Light
Light
Ray Tracing
Alpha
Color Spleting Lictt)
Ens Integration
Spaating
Light
Splattin
Map
Radiance
2D Cauedars
Indirect Lighting
Direct
Lighting
Rosten: zataca
Bidirectional
RGB * Loss
Reflectance Distribution
Function (BRDF)
Diffuse
Roughness
Metallic
Albedo
Normal
Rendered Image
Image
Multi-view Material Consistency Loss (Sec. 3.2)
Reflection Streagth Prior (Sec. 3.2)
Plotommatno Varlinos
Near Viewe
NevrViord Vira?
Maps
Rasterization
Stutiooa Potnrs
Tojectary
Rellection Stwnigh
Bunghtres
Figure 2: Overview our method. We learn illumination decomposition by imposing multi-view material consistency constraint and reflection strength prior supervision the rasterized material maps (Sec. 3.2). To facilitate this process, we introduce an environment modeling strategy through ray tracing with 2DGS, which effectively captures photorealistic incident lighting effects (Sec. 3.3).
observations. To overcome this obstacle, we propose to learn illumination decomposition by exploring multi-view consistent geometric clues as material inference constraints and guidance.
Multi-view Material Consistency. Based on the above analysis, we constrain the 2D Gaussians to produce multi-view consistent material maps, which is essential for accurate fight-objct-environmeee interactions inference. Specifically, for surface point visible from both viewpoint v; and tij. we want the projection T (p) of p on the two material maps V (T; (p) to be the same. Drawing inspiration from multi-view -iew stereo methods [8, 6]. we impose constraints between adjacent views. We first sample 7 pixel patch ((T:(p) around n, (y),, back back-project it into 3D space along v; using the rendered depth d; at Te(p), rotate it with the rendered normal at m. (p), and reproject into V using the rendered depth at ( form a warped patch P'(TT(())
plane patches
P'(x((p)) = H.y (P))())), Hij = Ky(R.j Tymm d:
K
where K, K2 are the intrinsic parameters of camera e and ej. Raj, Tij, Hij denote the relative rotation, relative translation and homography matrix from v; to Uy. respectively. We then enforce consistency between the two patches on the material maps using MSE loss,
(5) In practice, once select reference view along with multiple source views, warp the patch from the reference view to each source view, and compute a loss for every reference-source patch pair. The multi-view material consistency constraint is imposed to diffuse, roughness and metallic maps &C2, R R T. We do not use this constraint on the albedo map 1 since albedo contributes little to most non-reflective surfaces, and enforcing consistency ineffective and potentially harmful.
Emo = || (p)) W (_(P"(Ty(P)]]l
Multi-view Consistent Reflection Strength Prior. Multi-view consistency on material maps alone is insufficient to provide clear guidance for illumination decomposition. Based the observation that highly reflective surfaces exhibit significantly different appearances across different viewpoints [3, 13], explore multi-viee photometric variations explicit supervision for reflection strength. We first apply luminance normalization [54] on the ground truth RGB images to eliminate brightness inconsistencies caused by shadows and textures. As illustrated in Fig. 3, given reference view Vr. we select M nearby views ( along the camera trajectory. For each pixel (u, v) in vr, we sample a 3 x patch P. (u, v) and warp it into the near views as [P, (u, J.M using Eq. We then compute the averaged per-pixel variance among these patches reflection score for vr, using standard deviation,
re refscore.
std (,, w)], Vnal [ (M, w)], V V PParPaa (6)
5

---

where y denotes the nor- malized RGB image and -.d(.) is per-pixel standard deviation operator. Since the reflected environments on the images may appear similar from certain view- ing angles, the obtained per- view reflection scores are often inconsistent, as shown in Fig. 3 (a). To address this, we further introduce a spa- tial reflection fusion mod- ule to aggregate multi-view reflection scores. We back- project the per-view reflec- tion scores into 3D space us- ing depth maps to form a reflection score point cloud.
(a) Multi-view Photometric Tracking
(b) Spatial Reflection Fusion
Fused Pointeloud
Back- Back-Project
Near View1
Surface Point
Near View2
Ball Queey
Top Average
Reference View
Averaged RGB StdDev
Per-view Reflection Strength (Eq.
Consistent Reflechon Strength Priors alarr
(c) Metallic Superviston
Weighting
Supervision (Eg.
Per Block Metallic Average
Filler
[(u. P)
Remby
SAM2
For each query pixel, we perform a ball query [41] around its back back-projected 3D location within the point cloud and compute the averaged top-K scores, thus yielding the final reflection strength prior Wref- as illustrated in Fig. 3 (b). The prior indicates how likely the surface has a high reflection strength, therefore can serve as weight of the constraint material maps V.
Readured Metallio Map
(2
Turget Metallic
GT Imegee
Foreground Mask Block Segmenlation
Figure 3: Illustration of computing reflection strength priors.
Cuef(u, u) -f [Tes T(u,e) v)- [Mo(u, v) T (u, v)l
if V (u, v) Mo
(7)
if v M (u, v) Md
where Mo(u, v) is a pre-computed target value, and I'uu, v) is a binary mask indicating whether the supervision is applied at pixel (u, v). To adapt our constraint to centric scene structures, we extract foreground masks using [12] and segment them into semantically meaningful regions using SAM2 [43]. For each region, I(u, v) is set to 1 if the averaged reflection prior Wref of all pixels in the region exceeds a threshold, and 0 otherwise. The target metallic value Mo [u, v) is determined according to the averaged metallic value within each region. In word, if a region more likely corresponds to a highly reflective surface, we apply a constraint that encourages higher metallic values for these pixels, as illustrated in Fig 3 (c). The intensity of the constraint is controlled by Wref, allowing the supervision to be adaptively modulated based on our confidence of the surface reflectivity. Similar observations were discussed in [24, 56, 53], where highly reflective surfaces are often reliably identified early during the optimization.
Normal Prior.
We also incorporate monocular normal priors obtained by pre-trained network [57] to supervise the normals rendered by 2DGS. We find it to be an effective cue for scene geometry inference during the early training stage.
3.3 Environment Modeling through Ray Tracing
When using split-sum approximation to model the specular illumination Eq. 3, an incident light wa fails to retrieve a plausible radiance from the environment map if it is occluded by other objects in the scene, To address this issue, we decompose the incident radiance into direct and indirect components, and introduce occlusion probability O(w;) E [0, 1] indicating how likely is occluded, (wi( = Trindirect (wi) ) + (1 O(w,)) (wi)
(8)
The direct lighting Ldirect (w:) can be obtained by querying a learnable environment map using the reflected direction and roughness. The environment map is a mip-mapped cubemap constructed with multiple roughness levels. estimate the color of the indirect light which is occluded by other objects, we perform Gaussian ray tracing [53]. Starting from surface point, we trace ray along the reflected direction and identify all intersected Gaussians, where each Gaussian has been transformed into Bounding Volume Hierarchy (BVH) with two triangles. The intersected Gaussians

---

Table 1: Numrical evaluations on all four datasets. Best results are highlighted as
1st
2nd
3rd
Datasets
ShinyBlender [47]
SyoesySymhetic
[33]
Ref-Real [47]
NoR 360
Methods
PSNR1
SSIM
LPIPS.
MAE"
PSNR1
SSIM1
LPIPS)
PSNR1
SSIMI
LPIPS,
PSNR1
SSIMI
LPIPS)
RefNeRF [47]
33.13
0.961
0.080
18.38
0.905
0.112
016645
0.239
ENVIDR [50]
32.88
0.969
0.072
2.74
29.06
0.947
0.060
23.00
0.666
0.332
3DGS [22]
30.37
0.947
0.083
26.01
0.886
0.089
23.85
0.660
0.230
27.21
0.815
0.214
StassinnSsderr [20]
31.94
0.957
0.068
7.00
27.11
0.922
0.082
23.46
0.521
0.257
2DGS
29.58
0.946
0,088
0.661
0.292
27.03
0.805
0.225
3DGS-DR
0.059
2.62
0.054
0.269
Ref-Grussian [56]
35.04
0.973
0.056
4.59
30.08
0.957
0.050
24.61
0.685
0.252
26.62
0.788
0.272
EnoG8 [53]
33.83
0.969
0.066
6.36
28.17
0.938
0.067
24.85
0.688
0.215
27.36
0.799
0.222
Ours
35.57
0.996
0,049
2.04
30.83
0.962
0.046
25.04
0.703
0.185
27,06
0.809
0.181
GaussianShedder
3DGS-DR
Ref-Gaussian
EnvGS
Ours
Ground Truth
Figure 4: Visual comparisons on Synthetic Datasets. Our method successfully recovers fine-grained reflections the helmet, as well as inter-reflection effects on the teapot.
are septh-sorted, and splatting is performed to compute both the accumulated transmittance, denoted as (w;), and the resulting lighting of the indirect illumination, denoted as Lindirect (w,))
N
i-1 II o,) + cr. O(w;) = Σ O P * II o1)
i-1
Lindirect (wi)
Cd 2
(9)
where N the number of the intersected Gaussians during ray tracing. We also incorporate a residual term e, in the indirect radiance to account for noise and higher-order lighting effects [58, 56]. The ray tracing procedure naturally handles both occluded and unoccluded cases, where unoccluded rays yield Lindirect (w,) (ww;) O. Therefore, query to obtain the full incident radiance. Compared to Ref-GGussian [56] which relies on an offline binary visibility indicator separate direct and indirect terms and estimates indirect light solely through a residual color, our method evaluates occlusion in a fully differentiable manner. Moreover, this design allows Gaussians to participate in environment illumination modeling and be jointly optimized, leading to more physically grounded modeling and improved generalization.
only need one ray tracing pass and environment
3.4
Optimization
We train our method for a total of 30k iterations. We begin by training 2DGS [18] with normal priors during the first 3k iterations to ensure geometric stability. After that, we incorporate PBR and our environment illumination modeling into the training process. At 10k iteration, we remove the normal prior to avoid potential bias from inaccurate predictions, and introduce our multi-view regularization terms. We also adopt normal propagation [24, 56] to propagate reliable normals to neighboring Gaussians for consistency and stability. The loss function can be written as
L Lo + An-aCn-d + XnCn, + AmeCmo Cref
(10)
where L. = 0.8 * Ergs + 0.2 * LD-SSIM is the photometric loss commonly used in Gaussian- based methods [22, 56), fa-dd denotes the depth-normal consistency loss used in 2DGS [18], and Ln |1 Nr N is the normal prior loss. Luv, Lies correspond our multi-view consistency loss (Eq. 5) and reflection strength loss (Eq. 7), respectively.
Experiments
4.1 Experiment Settings
Datasets & Metries. We evaluate the performance of our method on widely used benchmarks, including two synthetic datasets, ShinyBlender [47] and GlossySynthetic [33], as well as two real-
7

---

GausslanShader
3DGS-DR
Ref-Gaussian
ErvGS
Ours
Ground Truth
Figure 5: Visual comparisons in real- world Ref-Real [47] dataset. Our method accurately reconstructs reflection textures from surrounding environments highly reflective surfaces.
3DGS
3DGS-DR
Ref- Gaussian
EnvGS
Owr
Ground Truth
Ours Normal
Ours Normal
Figure 6: Visual comparisons in real-world Mip-NeRF 360 [1] dataset. Our method faithfully reconstructs highly reflective surfaces, such as the aluminum bowl and the metal plate.
world datasets, Ref-Real [47] and Mip-NeRF 360 [1]. All of these datasets contain challenging scenes with prominent reflective surfaces. To evaluate the quality of novel view synthesis, we report PSNR, SSIM [49] and I.PIPS [59]. We also evaluate the accuracy of the predicted normals using Mean Angular Error (MAE).
Baselines. We compare our method with the state-of-the-ar reflection modeling methods, including NeRF-based methods: Ref-NeRF [47], ENVIDR [30], as well GS-based methods: 3DGS [22], GaussianShader [20], 2DGS [18], 3DGS-DR [24], Gaussaan [56] and EnvGS [53].
4.2 Comparisons
Comparisons on Synthetic Dataset. We first evaluate our method on two synthetic datasets, ShinyBlender [47] and GlossySynthetic [33], and report the numerical results in Tab. 1, where we achieve the best performance across all metrics on both datasets. We further provide visual comparisons in Fig. 4, where our method accurately captures the environment reflections on the helmet. In addition, our approach effectively models secondary light reflections, such as the self- reflection of the teapot lid knob on the metallic lid, which benefits from our environment modeling.
8

---

BGB
Difluse
Metalie Roughness Albedo
Normal
Envarament Map
RGB
Driffuse
Metallic Rougtness lbodo Normal
Enviroment
G.3ada
Figure 7: Visualization of illumination decomposition results and the learned environment maps. Our method produces more uniform and physically plausible material maps, along with sharper and more detailed environment maps. Best viewed with zoom in.
(a) wlo Lup W Lyer
(b) mm Lype/
Full Modid
(6) w/o Normal (b) wio Env (c) Full Model
(d) GT
PRGB Map
Metellic Map
Viesv1
View2
View1
View2
Viewl
View2
Figure 8: Ablation study on multi-view consistent mate-
Figure 9: Ablation study normal prior and environment modeling.
rial inference strategies.
Comparisons on Ref-Real Dataset. We also report quantitative comparisons on the real-worl world dataset Ref-Real [47] in Tab. 1, where our method consistently achieves state-of-the-art performance across all metrics. Visual comparisons in Fig. 5 show that method accurately reconstructs the reflection textures from the surrounding environments on highly reflective surfaces, such as tree branches reflected
windows and ground seams reflected on the metallic sphere.
Comparisons on Mip-NeRF 360 Dataset. further evaluate our method on the more challenging real-world dataset Mip-NeRF 360 [1], as reported in Tab. 1. Existing reflective 3DGS methods often show degenerated performance on such complex environments with few reflective surfaces. In contrast, method achieves competitive results and outperforms all basclines in terms of LPIPS, highlighting our strong generalization ability. Visual comparisons are provided in Fig. 6, where our method faithfully recovers highly reflective surfaces such as the aluminum bowl and the metal plate, which even remain difficult for existing GS-based reconstruction methods.
Comparisons of Illumination Decomposition. To validate the effectiveness of our illumination decomposition, we visualize the decomposed material components and the learned environment maps on GlossySynthetic dataset [33], shown in Fig. 7. Note that our method cannot be directly compared with inverse rendering methods [11, 31, 21J, as the illumination modeling approaches are much different.
4.3 Ablation Study
Effectiveness of Each Module. We conduct ablation studies to evaluate the effectiveness of each module in our framework on both synthetic and real-world datasets. We start by analyzing the multi-view material inference strategies. Without any strategies, the result (Fig. 8 (a). "w/o Lmo. w/o Lref" row in Tab. show inconsistent material maps and weak reflections. Introducing L.e makes the multi-view material maps uniform and consistent, leading to clearer reflections on the top of the sphere (Fig. (b), "w/ L.v. w/o Lrey" row Tab. 2). Further adding Lref improves the metallic in textureless reflective regions, making subtle reflections like ground seams more visible (Fig. 8 (c), "Full Model" row in Tab. 2). We also ablate the environment modeling strategy by removing ray tracing, relying residual color and environment map overfit incident illumination. This causes

---

Table 2: Ablation study on each one of our modules.
Datasets
ShinyBlender [47]
Ref-Real [47]
Models
PSNR
SSIM
LPIPS}
PSNR
SSIM
LPIPS}
w/o Lines w/o fref
34.87
0.972
0.055
24.24
0.655
0.260
w/ Lmv, w/o Lref
35.21
0.975
0.051
24.47
0.670
0.242
w/o Ln
35.37
0.975
0.050
24.39
0.672
0.229
w/o Environment
34.69
0.976
0.049
24.76
0.681
0.199
Full Model
35.57
0.976
0.049
25.04
0.703
0.185
noticeably blurred reflections in inter-reflection regions (Fig. (b), "w/o Environment" row Tab. 2).
Lastly, removing the normal prior from full model leas to degenerated geometry and color (Fig. 9 (a). "w/o Cn" row in Tab. 2).
Normal Prior. To evaluate the necessity of the normal prior, we conduct ablation studies on ShinyBlender dataset [47] under three experi- mental settings: Full Model, Full model with- out normal prior (w/o L,), Full model without normal prior, material regularization and envi- ronment modeling (w/o f.n, w/o Reg, w/o Env). We report the normal accuracy in Tab. using
Table Ablation study on normal prior.
Models
MAE
CD
w/o C.s w/o Reg, w/o Env w/o La
3,47
0.94
2.59
0.68
Full Model
2.04
0.60
MAE, as well the geometric reconstruction accuracy compared with ground truth meshes using Chamfer Distance (CD). The results indicate that, beyond the normal prior, our material regularization and environment modeling also contribute significantly geometry learning. This is because both our material constraints and environment modeling are differentiable to the depth and normal, enabling end-to-end joint optimization of geometry, appearance, and material properties for improved overall performance.
5 Conclusion
We propose MaterialRefGS, novel approach that learns multi-view illumination decomposition for reflective gaussian splatting through multi-view consistent material inference. To this end, we enforce the Gaussians to produce consistent material maps across different views, and explore reflection strength priors from photometric variants to provide explicit supervision for specular reflectance modeling. We also introduce a novel environment modeling strategy based on Gaussian ray tracing, which compensates for the indirect illumination caused by inter-object occlusion. Extensive ablation studies justify the effectiveness of our proposed modules, loss functions. and training strategies. Our evaluations show our superiority over the latest methods in rendering photorealistic novel views and recovering accurate geometry.
6
Acknowledgement
This work was supported by Deep Earth Probe and Mineral Resources Exploration National Science and Technology Major Project (2024ZD(003405), and the National Natural Science Foundation of China (62272263), and in part by Kuaishou.
References
[1] Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Mip- NeRF 360: Unbounded anti-aliased neural radiance fields. Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 14709,479, 2022.
[2] Marcelo Bertalmio, Guillermo Sapiro, Vincent Caselles, and Coloma Ballester. Image inpainting. In Proceedings of the 27th annual conference on Computer graphics and interactive techniques, pages 417-424, 2000.
10

---

[3] Sai Bi, Zexiang Xu, Kalyan Sunkavalli, Miloš Hašan, Yannick Hold-Geoftroy, David Kriegman, and Ravi Ramamoorthi. Deep Reflectance Volumes: Relightable reconstructions from multi- view photometric images. In European Conference Computer Vision, pages 294-311. Springer, 2020.
[4] Brent Burley and Walt Disney Animation Studios. Physically-based shading at disney. ACM Siggraph, volume 2012, pages 1-7. vol. 2012, 2012.
[5] Chao Chen, Yu-Shen Liu, and Zhizhong Han. NeuralTPS: Learning signed distance functions without priors from single sparse point clouds. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47615:56--582 2025.
[6] Danpeng Chen, Hai Li, Weicai Ye, Yifan Wang. Weijian Xie, Shangjin Zhai, Nan Wang, Haomin Liu, Hujun Bao, and Guofeng Zhang. PGSR: Planar-based gaussian splatting for efficient and high-fidelity surface reconstruction. IEEE Transactions Visualization and Computer Graphics, 2024.
[7] Yuedong Chen, Haofei Xu, Chuanxia Zheng, Bohan Zhuang, Mare Pollefeys, Andreas Geiger, Tat-Jen Cham, and Jianfei Cai. MVSplat: Efficient 3D gaussian splatting from sparse multi-view images. In European Conference on Computer Vision, pages 370-386. Springer, 2024.
[8] François Darmon, Bénédicte Bascle, Jean-Clément Devaux, Pascal Monasse, and Mathieu Aubry. Improving neural implicit surfaces geometry with patch warping. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 6260-6269, 2022.
[9] Yikang Ding. Jiwen Liu, Wenyuan Zhang, Zekun Wang, Wentao Hu, Liyuan Cui, Mingming Lao, Yingchao Shao, Hui Liu, Xiaohan Li, et al. Kling- Grounding multimodal instructions for cascaded long-duration avatar animation synthesis. arXiv preprint arXiv:2509.0955 2025.
[10] Kang Du, Zhihao Liang, and Zeyu Wang. GS-ID: Illumination Decomposition on Gaus- sian Splatting via Diffusion Prior and Parametric Light Source Optimization. arXiv preprint 0et:2448..85244 2024.
[11] Jian Gao, Chun Gu, Youtian Lin, Zhihao Li, Hao Zhu, Xun Cao, Li Zhang, and Yao Yao. Relightable 3d gaussians: Realistic point cloud relighting with brdf decomposition and ray tracing. In European Conference on Computer Vision, pages 73-89, Springer, 2024.
[12] Daniel Gatis. Rembg: A tool to remove images background. httpp://// /tgg://///hhb......// thub. com/
danielgatis/rembe 2023.
[13] Wenhang Ge, Tao Hu, Haoyu Zhao, Shu Liu, and Ying-Cong Chen. Ref-NeuS: Ambiguity- reduced neural implicit surface learning for multi-view reconstruction with reflection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4251-4260. 2023.
[14] Chun Gu, Xiaofei Wei, Zixuan Zeng. Yuxuan Yao, and Li Zhang. IRGS: Inter-Reflective Gaussian Splatting with 2D Gaussian Ray Tracing. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025.
[15] Antoine Guédon and Vincent Lepetit. SuGaR: alrgaee-aligned gaussian splatting for efficient
3d mesh reconstruction and high-quality mesh rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages
2024.
[16] Liang Han, Xu Zhang, Haichuan Song, Kanle Shi, Yu-Shen Liu, and Zhizhong Han. Sparseke- con: Neural implicit surface reconstruction from sparse views with feature and depth consisten- cies. Proceedings of International Conference on Computer Vision, 2025.
[17] Liang Han, Junsheng Zhou, Yu-Shen Liu, and Zhizhong Han. Sinocular-Guuide 3D Gaussian Splatting with View Consistency for Sparse View Synthesis. In Advances in Neural Information Processing Systems, 2024.
[18] Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao. 2D Gaussian Splatting for Geometrically Accurate Radiance Fields. ACM SIGGRAPH 2024 conference papers, pages 1-11, 2024.
11

---

[19] Han Huang, Yulun Wu, Chao Deng. Ge Gao, Ming Gu, and Yu-Shen Liu. FatesGS: Fast and accurate sparse-view surface reconstruction using gaussian splatting with depth-feature consistency. In Proceedings of the AAAI Conference on Artificial Intelligence, 2025.
[20] Yingwenqi Jiang, Jiadong Tu, Yuan Liu, Xifeng Gao, Xiaoxiao Long, Wenping Wang, and Yuexin Ma. GaussianShader: 3D Gaussian Splatting with Shading Functions for Reflective Surfaces. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern 2024.
Recognition, pages
[21] Haian Jin, Isabella Liu, Peijia Xu, Xiaoshuai Zhang, Songfang Han, Sai Bi, Xiaowei Zhou, Zexiang Xu, and Hao Su. TensolR: Tensorial inverse rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 165-174, 2023.
[22] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3D gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4))1-14, 2023.
[23] Bernhard Kerbl, Andreas Meuleman, Georgios Kopanas, Michael Wimmer, Alexandre Lanvin, and George Drettakis. A hierarchical 3d gaussian representation for real-time rendering of very large datasets. ACM Transactions on Graphics (TOG), 43(4):1-15, 2024.
[24] Ye Keyang, Hou Qiming, and Zhou Kun. 3D Gaussian Splatting with Deferred Reflection. ACM SIGGRAPH Conference Proceedings, 2024.
[25] Jia Li, Lu Wang, Lei Zhang, and Beibei Wang. TensoSDF: Roughness-aware tensorial represen- tation for robust geometry and material reconstruction. ACM Transactions on Graphics (TOG), 13,,) 2024.
[26] Jiabao Li, Yuqi Li, Ciliang Sun, Chong Wang, and Jinhui Xiang. Spec-NeRF: Multi-spectral neural radiance fields. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 2485-2488 IEEE, 2024.
[27] Qing Li, Huifang Feng, Xun Gong, and Yu-Shen Liu. VA-GS: Enhancing the geometric representation of gaussian splatting via view alignment. Advances in Neural Information Processing Systems, 2025.
[28] Shengtao Li, Yudong Liu, Ge Gao, Ming Gu, and Yu-Shen Liu. I-filtering: Implicit filtering for learning neural distance functions from 3d point clouds. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2025.
[29] Shujuan Li, Yu-Shen Liu, and Zhizhong Han. GaussianUDF: Inferring Unsigned Distance Functions through 3D Gaussian Splatting, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025.
[30] Ruofan Liang, Huiting Chen, Chunlin Li, Fan Chen, Selvakumar Panneer, and Nandita Vi- jaykumar. ENVIDR: Implicit differentiable renderer with neural environment lighting. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 79-89, 2023.
[31] Zhihao Liang, Qi Zhang, Ying Feng, Ying Shan, and Kui Jia. GS-IR: 3D Gaussian Splatting for Inverse Rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21644-21653, 2024.
[32] Xinhai Liu, Zhizhong Han, Yu-Shen Liu, and Matthias Zwicker. Point2Sequence: Learning the Shape Representation of 3D Point Clouds with Attention-based Sequence to Sequence Network. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 8778 8778-8785, 2019.
[33] Yuan Liu, Peng Wang, Cheng Lin, Xiaoxiao Long, Jiepeng Wang, Lingjie Liu, Taku Komura, and Wenping Wang. NeRO: Neural geometry and brdf reconstruction of reflective objects from multiview images. ACM Transactions Graphics (ToG), 42(4):1-22, 2023.
12

---

[34] Tao Lu, Mulin Yu, Linning Xu, Yuanbo Xiangli, Limin Wang, Dahua Lin, and Bo Dai. Scaffold- GS: Structured 3D gaussians for view-adaptive rendering. Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages
2024.
[35] Baorui Ma, Zhizhong Han, Yu-Shen Liu, and Matthias Zwicker. Neural-Pull. Learning signed distance function from point clouds by learning pull space onto surface. In International Conference on Machine Learning, pages 7246-7257. PMLR, 2021.
[36] Ben Mildenhall, Pratul Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi. and Ren Ng. NeRF: Representing scenes as neural radiance fields for view synthesis. In European Conference on Computer Vision (ECCV), pages 405 4211.21.22 Springer, 2020.
[37] Nicolas Moenne-Loccod Ashkan Mirzaci, Or Perel, Riccardo de Lutio, Janick Martinez Esturo, Gavriel State, Sanja Fidler, Nicholas Sharp, and Zan Gojcic. 3D Gaussian Ray Tracing: Fast tracing of particle scenes. ACM Transactions Graphics (TOG), 43(6):1-19, 2024.
[38] Jacob Munkberg, Jon Hasselgren, Tianchang Shen, Jun Gao, Wenzheng Chen, Alex Evans, Thomas Müller, and Sanja Fidler. Extracting triangular 3d models, materials, and lighting from images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8280-8290, 2022.
[39] Takeshi Noda, Chao Chen, Junsheng Zhou, Weiqi Zhang, Yu-Shen Liu, and Zhizhong Han. Learning bijective surface parameterization for inferring signed distance functions from sparse point clouds with grid deformation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2025.
[40] Matt Pharr, Wenzel Jakob, and Greg Humphreys. Physically based rendering: From theory implementation. MIT Press, 2023.
[41] Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Advances in neural information processing systems, 30, 2017.
[42] Minghan Qin, Wanhua Li, Jiawei Zhou, Haoqian Wang, and Hanspeter Pfister. LangSplat: 3d language gaussian splatting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Putlern Recognition, pages
2024.
[43] Nikhila Ravi, Valentin Gabeur, Yuan- Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. Sam Segment anything in images and videos. arXiv preprint arXiv:2408.. 200.2400.00744 2024.
[44] Kerui Ren, Lihan Jiang, Tao Lu, Mulin Yu, Linning Xu, Zhangkai Ni, and Bo Dai. Octree- GS: Towards consistent real-time rendering with stu-strue 3d gaussians. arXiv preprint arXiv:2403.17888 2024.
[45] Daniel R Steinwand, John Hutchinson, and John Snyder. Map projections for global and continental data sets and an analysis of pixel distortion caused by reprojection. Photogrammetric Engineering & Remote Sensing, 61(1)):1487-1499 1995.
[46] Zhe Jun Tang and Tat-Jen Cham. 3iGS: Factorised tensorial illumination for 3d gaussian splatting. In European Conference on Computer Vision, pages 143-159. Springer, 2024.
[47] Dor Verbin, Peter Hedman, Ben Mildenhall, Todd Zickler, Jonathan T Barron, and Pratul P Srinivasan. Ref-NeRF: Structured view-dependent appearance for neural radiance fields. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5481-5490. IEEE, 2022.
[48] Fangjinhua Wang, Marie-Julie Rakotosaona, Michael Niemeyer, Richard Szeliski, Mare Polle- feys, and Federico Tombari. UniSDF: Unifying neural representations for high-fidelity 3d reconstruction of complex scenes with reflections. Advances Neural Information Processing Systems, 37:3157-3184, 2024.
13

---

[49] Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Ecro P Simoncelli. Image quality assessment: from error visibility structural similarity, IEEE transactions on image processing, (30):000 612, 2004.
[50] Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang. 4d gaussian splatting for real-time dynamic scene rendering. In Proceedings of the IEEE/CVF Conference Computer Vision and Patlern Recognition, pages 20310-20320, 2024.
[51] Yulun Wu, Han Huang, Wenyuan Zhang, Chao Deng, Ge Gao, Ming Gu, and Yu-Shen Liu. Sparis: Neural implicit surface reconstruction of indoor scenes from sparse views. In AAAI Conference on Artificial Intelligence, 2025.
[52] Peng Xiang, Xin Wen, Yu-Shen Liu, Yan-Pei Cao, Pengfei Wan, Wen Zheng, and Zhizhong Han. Snowflake point deconvolution for point cloud completion and generation with skip-transformer. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(5):630-6633,, 2022.
[53] Tao Xie, Xi Chen, Zhen Xu, Yiman Xie, Yudong Jin, Yujun Shen, Sida Peng, Hujun Bao, and Xiaowei Zhou. EnvGS: Modeling view-dependent appearance with environment gaussian. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025.
[54] Qingsen Yan, Yixu Feng, Cheng Zhang, Guansong Pang. Kangbiao Shi, Peng Wu, Wei Dong. Jinqiu Sun, and Yanning Zhang. HVI: A new color space for low-light image enhancement. Proceedings of the IEEE/CVF International Conference on Computer Vision, 2025.
[55] Wengi Yang. Guanying Chen, Chaofeng Chen, Zhenfang Chen, and Kwan- Yee Wong. PS- NeRF: Neural inverse rendering for multi-viww photometric stereo. In European Conference Computer Vision, pages 266-284. Springer, 2022.
[56] Yuxuan Yao, Zixuan Zeng, Chun Gu, Xiatian Zhu, and Li Zhang. Reflective gaussian splatting. International Conference Learning Representations, 2025.
[57] Chongjie Ye, Lingteng Qiu, Xiaodong Gu, Qi Zuo, Yushuang Wu, Zilong Dong, Liefeng Bo, Yuliang Xiu, and Xiaoguang Han. StableNormal: Reducing diffusion variance for stable and sharp normal. ACM Transactions on Graphics (TOG), 43(6):1-18, 2024.
[58] Kai Ye, Chong Gao, Guanbin Li, Wenzheng Chen, and Baoquan Chen. GeoSplatting: Towards geometry guided gaussian splatting for physically-basee inverse rendering. arXiv preprint arXiv:2410.24204 2024.
[59] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unrea- sonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 586-595, 2018.
[60] Weiqi Zhang, Junsheng Zhou, Haotian Geng, Wenyuan Zhang, and Yu-Shen Liu. GAP: Gaussianize any point clouds with text guidance. Proceedings of International Conference on Computer Vision, 2025.
[61] Wenyuan Zhang, Yu-Shen Liu, and Zhizhong Han. Neural Signed Distance Function Inference through Splatting 3D Gaussians Pulled on Zevo-Level Set. In Advances in Neural Information Processing Systems, 2024.
[62] Wenyuan Zhang, Kanle Shi, Yu-Shen Liu, and Zhizhong Han. Learning unsigned distance functions from multi-view images with volume rendering priors. In European Conference on Computer Vision, pages 395-415. Springer, 2024.
[63] Wenyuan Zhang, Emily Yue ting Jia, Junsheng Zhou, Baorui Ma, Kanle Shi, Yu-Shen Liu, and Zhizhong Han. NeRFPrior: Learning neural radiance field as prior for indoor scene reconstruction. In Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025.
[64] Wenyuan Zhang, Ruofan Xing, Yunfan Zeng, Yu-Shen Liu, Kanle Shi, and Zhizhong Han. Fast Learning Radiance Fields by Shooting Much Fewer Rays. IEEE Transactions on Image Processing, 32:2703-27 27118, 2023.
14

---

[65] Wenyuan Zhang. Yixiao Yang, Han Huang, Liang Han, Kanle Shi, Yu-Shen Liu, and Zhizhong Han. Monolnstance: Enhancing monocular priors via multi-view instance alignment for neural rendering and reconstruction. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, 2025.
[66] Youjia Zhang, Anpei Chen, Yumin Wan, Zikai Song, Junqing Yu, Yawei Luo, and Wei Yang. Ref-GS: Directional factorization for 2d gaussian splatting. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025.
[67] Junsheng Zhou, Baorui Ma, Shujuan Li, Yu-Shen Liu, Yi Fang, and Zhizhong Han. CAP-UDF: Learning unsigned distance functions progressively from point clouds with consistency- aware field optimization. JEEE Transactions on Paltern Analysis and Machine Intelligence, 2024.
[68] Junsheng Zhou, Baorui Ma. Wenyuan Zhang. Yi Fang, Yu-Shen Liu, and Zhizhong Han. Differentiable registration of images and lidar point clouds with voxelpoint-to- to-pxel matching. Advances in Neurul Information Processing Systems, 36:51 166-51177, 2023.
[69] Junsheng Zhou, Xingyu Shi, Haichuan Song, Yi Fang, Yu-Shen Liu, and Zhizhong Han. U-CAN: Unsupervised point cloud denoising with consistency- noise2noise matching. Advances in Neural Information Processing Systems, 2025.
[70] Junsheng Zhou, Jinsheng Wang, Baorui Ma, Yu-Shen Liu, Tiejun Huang. and Xinlong Wang. Uni3D: Exploring Unified 3D Representation at Scale. International Conference on Learning Representations, 2024.
[71] Junsheng Zhou, Weiqi Zhang, and Yu-Shen Liu. DifIGS: Functional gaussian splatting diffusion.
In Advances in Neural Information Processing Systems, 2024.
[72] Matthias Zwicker, Hanspeter Pfister, Jeroen Van Baar, and Markus Gross. Surface splatting. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques. pages 371-378, 2001.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our main claims made in the abstract and introduction accurately reflect the paper's 's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations and failure cases in the supplementary materials.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
16

---

We leverage the theory of physically based rendering that has been widely adopt by previous works, We do not make any assumptions the paper.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide the detailed information in reproducing our methods in Section of the main paper.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
17

---

The authors should provide scripts to reproduce all experimental results for the new reproducible, they Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: We do not include our code or data at this stage, we will definitely release our code and data in the future.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
/ httppsscccppbpiccccceeceeecl geides/ CodeeSbbm ssi aasonnooicyy for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
proposed method and baselines. only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide the training and testing details in the experiment section (Section
4).
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[No]
Justification:
We report the average performance the experimental results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
18

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
We report our training time with baseline methods the supplementary materials.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics attpp://neurre cccpublic//eeeececcceecceeeeeeee Ceteec////eeeeeeeeeee csQuidel ines?
Answer: [Yes]
Justification: The research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
We discuss the applications and potential impacts of our method in the introduction.
Justification:
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
19

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We use the open- sourced datasets and codes under their licenses.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperswi wwwode.hcoe...c. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
20

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Bthies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
21

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: Our method development in this research does not involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: /attps:/on /ttp///ersp////// tweurrpp.cc/oooffeeeeeeee.e /20nceee/////222//////// /LLMM for what should or should be described.
22


Reviews:
[{"review_id": "J05xPrjJAp", "reviewer": "Reviewer_LbeZ", "summary": "This paper tries to address the problem of inverse rendering with highly specular objects. Specifcally, it will reconstruction a scene with geometry represented with 2D Gaussians, and BRDF of each 2D Gaussians and an enviromentmap lighting from multiview posed images. \n\nThis problem has been long studied. The major novel techniques introduced in this paper is the multiview material consistency and reflection strength prior. Other techniques like visibility mask, normal loss, indirect lighting and split-sum approximation are not novel but it's non-trival to make it work and integraated into the proposed system. \n\nThe author report novel view synthesis results on ShinyBlender, GlossySynthetic, Ref-Real and Mip-NeRF360, achieving state-of-the-art results especially on ShinyBlender, GlossySynthetic and Ref-Real.", "questions": "A video demo of either relighting or novel view synthesis would be great!\n\nSpecularity is better visualized through videos of changing viewpoints or videos of changing env lighting. In the supp, I only find videos of changing viewpoints for a few specular objects, it would be great if you can also have a video of changing lighting. \n\n\n\nThe methods are a little bit complicated, it will be great to have a pseudocode for the major component of it.  Also, a quick question, the code for computing multiview material consistency and reflection prior could largely be the same?  if so, that would ease the cost of engineering. \n\nAbout evaluating extracted environment map lighting or extracted specular and diffuse colors. I don't think this is necessary, but it's very good to have, especially about the lighting.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength:\n\nThe results looks very impressive. The novel view rendering results, and the extrated geometry and also the visualization of the extracted environment map looks just expressive. \n\nThe author also ablated the complicated system design, especially the multiview material consistency and reflection prior. \n\n\n\nWeakness:\n\nOverall, I don't think there are some major weak point. My only feeling is that the proposed methods is a little bit too complicated.", "was_revised": true, "final_justification": "I read the author's rebuttal and it address most of my concerns. Given the promise that the authors will provide video visualization of changing lighting, I retain my score as weak accept.", "rebuttal": "Dear reviewer, thanks for your valuable comments and recognizing the **_impressive results_** and the **_novelty_** of our work. We are pleased to provide further clarifications and additional experiments to address your concerns.\n\n**1. About video visualization.** \n\nIn the supplementary video, we provide novel view synthesis results on both synthetic and real scenes, material decomposition results, and material editing results in terms of diffuse color and roughness. We also provide novel view synthesis of changing environment maps for relighting at the end of the video. Unfortunately, due to the restrictions on rebuttals, we are unable to show more environment relighting results. As for additional relighting applications such as inserting extra light sources (point lighting or parallel lighting), we can render these results based on our inferred material decomposition. We will include more visualization results in the revised version of our video.\n\n**2. About complexity of our method.** \n\nWe provide a flowchart for the two regularization strategies introduced in Section 3.2 to make it more clear. \n\n&nbsp;&nbsp;&nbsp;&nbsp;**(I) Multi-view Material Consistency.**\n\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;i) Select a reference view and M source views.\n\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ii) Warp pixel patches from reference view to source views. (Eq. 4)\n\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;iii) Compute $L_{mv}$ between patches on rendered material maps. (Eq. 5)\n\n&nbsp;&nbsp;&nbsp;&nbsp;**(II) Reflection Strength Prior.**\n\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;i) Select a reference view and M source views. (The same as above)\n\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ii) Warp pixel patches from reference view to source views. (Eq. 4) (The same as above)\n\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;iii) Compute reflection scores between patches on GT RGB images. (Eq. 6, Fig. 3a)\n\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;iv) Back-project the reflection scores into a 3D point cloud and compute the reflection strength prior. (Fig. 3b)\n\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;v) Use rendered metallic maps to compute $\\Gamma$ and $M_0$. (Fig. 3c)\n\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;vi) Use the reflection strength priors $w_{ref}$, $\\Gamma$, $M_0$ to compute $L_{ref}$. (Eq. 7)\n\nAs the reviewer observed, the i) - iii) steps of both processes share a very similar logic, which allows us to use the same implementation in practice. The only difference is which maps are applied to compute the patch errors.\n\n**3. Evaluation of environments and materials.**\n\nIt is not easy to directly evaluate the predicted diffuse and specular colors in these benchmarks, as there are no ground truths available for reference. As an alternative, we evaluate the performance of the material decomposition on Synthetic4Relight dataset which provides ground truth materials, as reported in Tab. 1 in supplementary materials. We further evaluate the accuracy of the learned environment maps on ShinyBlender dataset, as reported below. The results demonstrate that our multi-view consistent material decomposition and environment modeling contribute to learning more accurate environmental illumination.\n\n|        | GaussianShader | 3DGS-DR | Ref-Gaussian | Ours |\n|--------|----------------|---------|--------------|------|\n| **LPIPS↓** | 0.656          | 0.543   | 0.495        | **0.406** |"}, {"review_id": "hCm6ZSoKPh", "reviewer": "Reviewer_t1W5", "summary": "This paper addresses the challenge of handling reflective scenes using 3DGS. The authors argue that multi-view consistent material inference is crucial for the task. The proposed approach enforces Gaussian Splatting to generate view-consistent material maps while utilizing photometric variations across multiple views as prior information for reflective regions. Experiments on public benchmarks demonstrate that the proposed method achieves SOTA performance in novel view synthesis.", "questions": "The authors could also refer to the weakness section:\n\n- Better visualizations and explanations for Equation 1 and Figure 3 are suggested.\n- The baselines used are a bit confusing.\n- Ablation studies on window size parameters are suggested.\n\nI would consider raise my rating if the above concerns could be addressed.", "limitations": "Yes", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The proposed method performs well in real-world scenes, with high-quality mesh results on MipNeRF360 and clear visual improvements on Ref-Real. \n- The proposed method is well motivated. While view-dependent specular effects are commonly represented as view-independent material properties, the authors observe that Gaussian Splatting inherently produces view-consistent appearance. Their proposed modules effectively address this, leading to improved performance.\n- The paper is well-written and structured.\n\nWeaknesses:\n- Further verification and clearer explanation of Equation (1) should be provided, particularly regarding the undefined \"screen-space probability\" term.\n- Additional ablation studies on window size parameters (particularly for pixel patch size in Multi-view Material Consistency loss and Reflection Strength Prior calculations) are welcome.\n- It is unclear whether 3DGS or 2DGS serves as the baseline (inconsistent in the abstract and the Normal Prior sections with other parts).\n- Figure 3 is a little bit small, and it is suggested to provide clearer annotations to enhance the interpretability.", "was_revised": true, "final_justification": "The authors have addressed my concerns in the rebuttal, and I would raise my final rating to \"accept\".\n\nI think the paper would make a good NeurIPS poster paper.", "rebuttal": "Dear reviewer, thanks for your valuable comments and recognizing our **_impressive results_**, **_novel motivation_**, and **_well-organized structures_**. We are glad to provide further clarifications and experiments to address your concerns.\n\n**1. Explanation of Eq. (1).**  \n\nEq. (1) describes how Gaussian ellipsoids in 3D space are rasterized into 2D images, following the classic volume rendering formulation[1][2],\n\n$$\n\\hat{\\Psi} = \\sum_{i=1}^N \\psi_i * o_i * p_i * \\prod_{j=1}^{i-1}(1-o_j).\n$$\n\nHere, $\\psi_i$ denotes a selected attribute of the Gaussian $G_i$, such as diffuse color, albedo, and metallic. $o_i$ represents the opacity of $G_i$. The term $p_i$ indicates the probability that a given pixel lies within the projection of the 3D-space Gaussian onto the 2D screen space. The value of $p_i$ reaches 1 at the center of the projected Gaussian and decays outward. It is equivalent to evaluating a 2D Gaussian distribution function at the given pixel location. The term $\\prod_{j=1}^{i-1}(1-o_j)$ denotes the accumulated transmittance, indicating how much light passes through the preceding Gaussians before reaching $G_i$. We will add a more clear clarification in the revised version.\n\n**2. Ablation studies on patch size.**  \n\nThe patch size refers to the side length of the image patch warped from the reference view to the source view. We conduct additional ablation studies on the effect of patch size in Multi-view material consistency and Reflection strength prior on Ref-Real dataset, as reported below. As shown in the table, different patch sizes exhibit similar results, except when the patch size is too small, which makes the patch error more sensitive to outliers. These results demonstrate our robustness to the selection of patch size. In our implementation, we use path size as 7 for both multi-view material consistency and reflection strength prior. We also notice that there is a typo in line 182 of the main paper, where we mistakenly wrote the patch size as 3×3 instead of 7×7. We will correct it in the revised version.\n\n\n|  | **Multi-view** | **material** | **consistency** /// | **Reflection** | **strength** | **prior** |\n|-------------|----------------|----------------|-------------------|----------------|----------------|-------------------|\n| **Patch Size** | **PSNR$\\uparrow$** | **SSIM$\\uparrow$** | **LPIPS$\\downarrow$** | **PSNR$\\uparrow$** | **SSIM$\\uparrow$** | **LPIPS$\\downarrow$** |\n| 1           | 23.3989        | 0.6408         | 0.1900            | 23.4358        | 0.6434         | 0.1875            |\n| 3           | 23.4237        | 0.6422         | 0.1895            | 23.4372        | 0.6437         | 0.1871            |\n| 7           | 23.4378        | **0.6439**     | 0.1870            | 23.4378        | 0.6439         | **0.1870**        |\n| 11          | 23.4375        | 0.6438         | 0.1871            | **23.4380**    | **0.6441**     | **0.1870**        |\n| 15          | 23.4347        | 0.6437         | **0.1868**        | 23.4377        | 0.6439         | **0.1870**        |\n| 19          | **23.4389**    | 0.6435         | 0.1873            | 23.4375        | 0.6438         | 0.1871            |\n\n**3. 3DGS or 2DGS.**  \n\nWe exactly use 2DGS as our representation. We will carefully review the paper and ensure consistent description.\n\n**4. About Figure 3.**  \n\nWe provide a more clear clarification of the pipeline of Figure 3 in the following. We will enlarge Figure 3 and add symbols alongside their corresponding textual descriptions in the revised version to improve interpretability.\n\n&nbsp;&nbsp;i) Select a reference view and M source views.\n\n&nbsp;&nbsp;ii) Warp pixel patches from reference view to source views. (Eq. 4)\n\n&nbsp;&nbsp;iii) Compute reflection scores between patches on GT RGB images. (Eq. 6, Fig. 3a)\n\n&nbsp;&nbsp;iv) Back-project the reflection scores into a 3D point cloud and compute the reflection strength prior. (Fig. 3b)\n\n&nbsp;&nbsp;v) Use rendered metallic maps to compute $\\Gamma$ and $M_0$. (Fig. 3c)\n\n&nbsp;&nbsp;vi) Use the reflection strength priors $w_{ref}$, $\\Gamma$, $M_0$ to compute $L_{ref}$. (Eq. 7)\n\n[1]. Robert A Brebin, Loren Carpenter, and Pat Hanrahan. Volume rendering. In Seminal graphics: pioneering efforts that shaped the field, pages 363–372. 1998.\n\n[2]. Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3D Gaussian Splatting for Real-Time Radiance Field Rendering. ACM Trans. Graph., 42(4):139–1, 2023."}, {"review_id": "H6ZKKOMiQR", "reviewer": "Reviewer_Th9t", "summary": "This paper introduces MaterialRefGS, a novel approach to model reflection with multi-view material consistency and reflection strength prior supervision. Inspired by multi-view stereo methods, material maps are constrained to be view-independent by avoiding view-specific overfitting. To facilitate this process,  reflection scores are evaluated to supervise reflection strength attributes such as metallic parameter. The proposed method also proposes a differentiable environment model for photorealistic rendering with indirect illumination.  MaterialRefGS achieves state-of-the-art results of novel view synthesis for various datasets.", "questions": "1. Why does the reflection strength prior regularize the metallic parameter rather than the roughness? Roughness has a more direct impact on surface reflectivity.\n2. The improvement in geometry reconstruction seems to stem from the monocular normal prior rather than from reflection or environment modeling. Please provide further ablation studies isolating the effect of the normal prior, including MAE scores and normal visualization.\n3. The assumption that the same semantic region shares similar material properties is questionable. The SAM2 model is not trained to distinguish between different material properties.\n4. In the presence of self-occlusion shadows, is it possible to separate shadows from intrinsic material properties? Shadows may be baked into the diffuse map, complicating reflectance decomposition.\n5. The distinction between \"diffuse\" and \"albedo\" in the paper is unclear. \"Diffuse\" refers to diffuse albedo, while \"albedo\" appears to denote specular albedo. However, in the commonly used Disney BRDF model, both diffuse and specular components share the same albedo parameter.\n\nI hope authors address my concerns in Weaknesses and Questions to increase my score.", "limitations": "yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 1, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n1. Considering multi-view material consistency of view-independent material maps significantly improves performance.\n2. The reflection strength prior is effectively used to encourage higher metallic values for pixels with high weights.\n3. MaterialRefGS achieves SOTA performance in novel view synthesis\n\nWeaknesses\n1. Additional visualizations of the reflection strength prior across diverse scenes are necessary to demonstrate that highly reflective surfaces consistently exhibit higher weights.\n2. More visualizations of indirect illumination for glossy objects are needed to validate the effectiveness of the proposed environment modeling. Indirect radiance should handle the inter-reflection of the surfaces.\n3. The proposed differentiable environment modelling is an extension of prior methods, combining 2D Gaussian ray tracing [1] and Ref-Gaussian [2].\n\nReferences\n\n[1] IRGS: Inter-Reflective Gaussian Splatting with 2D Gaussian Ray Tracing, CVPR 2025\n\n[2] Reflective Gaussian Splatting, ICLR 2025", "was_revised": true, "final_justification": "The rebuttal provided detailed explanations of the proposed design and implementation, addressing most of my initial concerns.\n\nThe additional experiments were thorough and effectively demonstrated the method’s effectiveness.\n\nI acknowledge the significance and potential impact of the environment modeling component and multi-view material consistency.\n\nOverall, I raise my score to Borderline Accept, and I encourage inclusion of more comprehensive visualizations, explanations, and release of public code.", "rebuttal": "Dear reviewer, thanks for your valuable comments and suggestions. We are glad to provide further clarifications and experiments to address your concerns.\n\n**1. Visualization of reflection strength prior.** \n\nDue to the policy restriction, we are unable to show visualization of the reflection strength priors. To quantitatively evaluate the effectiveness, we binarize our reflection strength prior and uses the ground truth materials to identify highly reflective surface regions that are either predicted or the ground truth. We then compute the Accuracy and F1-score of the classification results on GT meshes between the predicted and ground truth regions, as reported below. The experiments are conducted on ShinyBlender dataset where there are ground truth meshes and materials. The results demonstrate that our reflection strength prior accurately localizes the highly reflective surfaces. We will include more visualization of the reflection strength prior in the revised version.\n\n| |ball|car|toaster|helmet|coffee|teapot|Mean|\n|-|-|-|-|-|-|-|-|\n|**Accuracy↑**|98.82%|96.79%|96.10%|95.73%|96.30%|94.69%|96.40%|\n|**F1-score↑**|0.9817|0.9564|0.9612|0.9545|0.9557|0.9459|0.9592|\n\n**2. Visualization of indirect illumination.**\n\nWe have included several examples of indirect illumination in the main paper, such as:  \n- the reflection of the flowerpot in the tray at the bottom in the first row of Fig.1;\n- the reflection of the teapot lid’s handle on the lid surface in the second row of Fig.4; \n- the reflection of ground seams on the metallic ball in the fourth row of Fig.5;\n- inter-reflections in the first and third rows of Fig.2 in the supplementary material.  \n\nUnfortunately, due to the policy restriction, we are unable to provide more visualization here. As an alternative, we evaluate the rendering performance in indirect illumination regions labeled by ground truth masks calculated on the ground truth meshes from each view. The experiments are conducted on ShinyBlender and GlossySynthetic datasets, where there are ground truth meshes. The results are reported in the table below. We do not include LPIPS metric here, as it is designed for full image evaluation. Our method shows significant improvements over the baselines in indirect illumination regions, demonstrating the effectiveness of our environment modeling module. We will include more visual examples in the revised version.\n\n|**Scenes**|**toaster**| |**coffee**| |**teapot**| |**tbell**| |**angel**| |\n|-|-|-|-|-|-|-|-|-|-|-|\n|**Methods**|**PSNR↑**|**SSIM↑**|**PSNR↑**|**SSIM↑**|**PSNR↑**|**SSIM↑**|**PSNR↑**|**SSIM↑**|**PSNR↑**|**SSIM↑**|\n|GaussianShader|18.24|0.821|32.70|0.951|12.67|0.723|22.02|0.951|17.83|0.876|\n|3DGS-DR|18.93|0.839|33.55|0.953|14.81|0.792|26.26|0.968|18.96|0.901|\n|Ref-Gaussian|20.60|0.852|34.72|0.955|17.51|0.835|28.46|0.970|21.71|0.921|\n|EnvGS|19.20|0.844|33.17|0.954|13.04|0.776|23.94|0.962|18.17|0.892|\n|**Ours**|**25.97**|**0.942**|**35.37**|**0.959**|**21.50**|**0.912**|**31.14**|**0.985**|**25.56**|**0.957**|\n\n**3. About environment modeling module.**\n\nRef-Gaussian separates surfaces into direct and indirect illumination regions and uses rendered residual colors from Gaussians to approximate indirect radiance. However, this approach has significant limitations in modeling complex inter-reflections between objects. IRGS introduces 2D Gaussian ray tracing, but it requires Monte Carlo sampling at every ray-surface intersection to evaluate the rendering equation, which significantly slows down the training and inference, and makes it less scalable to complex real-world scenes.\n\nOur method combines the strengths of both approaches. We apply the split-sum approximation in indirect illumination regions and conduct 2D Gaussian ray tracing along the reflected rays to model higher-order lighting effects. As a result, we achieves significantly better rendering quality. Qualitative comparisons can be found in the second row of Fig. 4, the fourth row of Fig. 5 in the main paper, and the first, third, and fourth rows of Fig. 2 in the supplementary material. The numerical comparisons in the indirect illumination regions, as shown in the table above, further highlight the strengths of our approach.\n\n**4. Why not supervise roughness & The use of albedo in diffuse and specular color.**\n\nThe primary goal of our work is modeling reflection effects in complex scenes for novel view synthesis. Therefore, our approach of material modeling and its usage differs from inverse rendering methods such as IRGS and GS-IR.  Following Ref-Gaussian, we directly rasterize the diffuse color $c_{diffuse}$ using per-Gaussian spherical harmonics and use other material properties such as albedo, metallic, and roughness to evaluate the specular color $c_{specular}$, rather than using albedo and metallic to compute both diffuse and specular components as done in inverse rendering methods. This design avoids the challenging task of evaluating albedo and metallic values for all locations in complex real-world scenes. Moreover, since the diffuse component is not sensitive to viewing direction, delegating the prediction of diffuse color to the Gaussians allows us to better disentangle reflection effects from illumination. In such case, our rendering equation can be specifically written as follows,\n\n$$\n\\begin{aligned}\nc_{\\text{diffuse}}&=(1-m)c_d,\\\\\\\\\nc_{\\text{specular}}&=\\left(((1-m)\\cdot 0.04+m\\cdot a)\\cdot F_1+F_2\\right)\\cdot\\text{env}(\\omega_i, r),\\\\\\\\\nF_1,\\ F_2&=\\text{lookuptable}(\\omega_i, r),\\\\\\\\\nc_{\\text{final}}&=c_{\\text{diffuse}}+c_{\\text{specular}}\n\\end{aligned}\n$$\n\nWhere $\\text{env}(\\omega_i,r)$ denotes querying from the mip-mapped environment cube maps using the incident direction and roughness. For indirect illumination, this term is replaced by a combination of the residual color and the color obtained via 2D Gaussian ray tracing. $F_1,F_2$ are obtained through table lookup based on the incident direction and roughness.\n\nFrom the above equation, we can see that roughness only influences our color modeling indirectly, whereas the metallic value directly determines the proportion of light attributed to specular and diffuse components. Therefore, we apply the reflection strength prior as a regularization term for the metallic prediction.\n\n**5. Ablation on normal prior.**  \n\nWe conduct additional ablation studies on ShinyBlender dataset under three experimental settings:  **A. Full model**;  **B. Full model without normal prior**; **C. Full model without normal prior and without other components (including material regularization and environment modeling)**.  We report the normal accuracy using MAE, as well as the geometric reconstruction accuracy using Chamfer Distance (CD) below.\n\n| Metric|A|B|C|\n|-|-|-|-|\n| MAE$\\downarrow$|**2.04**|2.59|3.47|\n| CD$\\downarrow$|**0.60**|0.68|0.94|\n\nThe results indicate that, beyond the normal prior, our material regularization and environment modeling also contribute significantly to geometry learning. This is because both our material constraints and environment modeling are differentiable to the depth and normal, enabling end-to-end joint optimization of geometry, appearance, and material properties for improved overall performance.\n\n**6. About the segmentation of SAM2.**  \n\nSorry for the misunderstanding in Fig.3 and description. We don't enforce SAM2 to perform semantic segmentation of the scene. Instead, we encourage SAM2 to distinguish patterns that exhibit different appearances due to varying material properties, which is not limited to semantic objects. It can divide a single object into multiple patches to identify regions with different material attributes. Furthermore, the segmentation granularity of SAM2 is controllable by configuring the number of anchor points per side and specifying whether to prioritize traversal based on minimum or maximum patch areas. By setting an appropriate granularity level, we can prevent regions with different materials from being merged into a single part. We provide an additional ablation study on the impact of segmentation granularity, and report the granularity level, the average number of patches predicted by SAM2, and the rendering metrics, as shown in the table below.\n\n|Granularity|PointsPerSide|Patch Traversal|Number of Patches|PSNR$\\uparrow$|SSIM$\\uparrow$|LPIPS$\\downarrow$|\n|--|--|--|--|--|--|--|\n|G1|128|From Min to Max|15|**30.63**|**0.961**|**0.046**|\n|G2|64|From Min to Max|9.5|30.54|0.961|0.047|\n|G3|64|From Max to Min|2.5|30.09|0.958|0.050|\n|G4|32|From Max to Min|1|29.85|0.957|0.052|\n\nIn the table, G1 represents the finest segmentation granularity, while G4 corresponds to the coarsest. The experimental results show that finer granularity leads to better performance, however, the difference is not significant. Our method exhibits robustness to the choice of segmentation granularity. We use the same setting as G1 in our implementation. We will add more segmentation visualizations in the revised version.\n\n**7. About self-occlusion shadows.**  \n\nOur method is able to effectively handle inter-reflection effects caused by self-occlusion between reflective objects. As for the shadow cast by one object onto another behind it, it may indeed be learned in the diffuse component. However, our method mainly focuses on modeling reflection effects in the scene for novel view synthesis. Since the light sources in the scene are fixed, shadows can be naturally interpreted as an intrinsic component of the diffuse color. Therefore, the shadows in the diffuse map do not harm the rendering quality from different views. We will add visualizations of multi-view rendered shadows in the revised version to demonstrate this. To explicitly disentangle shadows from illumination, future works may include incorporating image relighting priors to remove shadows from input images, or leveraging additional input images captured under varying lighting conditions to provide richer illumination priors."}, {"review_id": "BKPPuc5PMz", "reviewer": "Reviewer_dUyL", "summary": "This paper proposes a multi-view consistent framework for learning reflections in 3D Gaussian Splatting (3DGS), enhancing material inference and environment modeling. By enforcing consistent material maps and leveraging ray-traced indirect illumination, the method improves photorealism and achieves state-of-the-art novel view synthesis.", "questions": "Kindly refer to the [Weaknesses].", "limitations": "Yes", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "### Strengths\n* The motivation of supervising the learning of accurate reflective surfaces through multi-view material attribute consistency is well-founded, and the effectiveness of the proposed method is thoroughly validated through experiments.\n* Leveraging visual foundation models to provide additional prior input proves highly suitable for such ill-posed tasks, and the authors present sufficiently convincing visual results.\n* The paper is well-written and easy to follow.\n\n### Weaknesses\n*  There is some confusion regarding the simultaneous learning of albedo and diffuse attributes. The simplified Disney BRDF model employed in the paper treats albedo and diffuse components as independent, which is uncommon in computer graphics literature. Why not compute diffuse from albedo, rather than optimizing both separately? In L161–162, the authors claim that the contribution of “albedo” to non-reflective surfaces is minimal, which contradicts the traditional definition of albedo in graphics. If the authors could provide experiments to demonstrate that separating albedo and diffuse terms significantly benefits the optimization of 3DGS, it would help dispel this confusion.\n* The choice of values for $M_0$ and $\\Gamma$ in Equation 7 seems rather ad hoc. Whether Eq. 7 functions effectively depends heavily on the specific values of $M_0$  and $\\Gamma$ yet the paper lacks experimental validation to confirm the robustness of their computation. It is recommended to include ablation studies on the thresholding of the averaged reflection prior $w_{ref}$, as well as on the computation method for $M_0$ .\n* Visualization of occlusion and indirect illumination. Although the authors provide qualitative and quantitative comparisons with and without environment modeling, many recent works employ similar techniques involving ray tracing or residual color estimation for indirect illumination. While I believe the proposed one-bounce ray tracing with residual color is a promising solution, I am curious about how the resulting occlusion and indirect illumination buffers compare with those of other methods such as GS-IR, Ref-Gaussian, TensoIR, or Relightable 3DGS.\n* The normal prior. As illustrated in the visualizations in Fig. 9, the normal prior appears to be critical, particularly for reconstructing geometric surfaces. However, do the two regularization strategies proposed by the authors rely on the accuracy of the geometric surface in the initial optimization stage? If the monocular normal prediction model provides inaccurate or viewpoint-inconsistent prior sets, can the proposed method still serve as a robust regularizer for material learning? Additionally, in Tab. 2, two rows of metrics for ShinyBlender are identical, which may be a minor oversight.", "was_revised": true, "final_justification": "The authors' response and additional experiments have addressed my concerns. Considering the perspectives of the other reviewers, I will maintain my current score.", "rebuttal": "Dear reviewer, thanks for your valuable comments and recognizing our **_well-founded motivation_** and **_well-organized structures_**. We are glad to provide further clarifications and experiments to address your concerns.\n\n**1. About albedo term.**  \n\nThe primary goal of our work is modeling reflection effects in complex scenes for novel view synthesis. Therefore, following Ref-Gaussian, we directly rasterize the diffuse color $c_{diffuse}$ using per-Gaussian spherical harmonics and use other material properties such as albedo, metallic, and roughness to evaluate the specular color $c_{specular}$.  \nWe observe that incorporating albedo in modeling the diffuse component, as done in traditional graphics, will significantly increase the optimization difficulty, especially in complex real-world scenes. On the other hand, since the diffuse component is not sensitive to viewing direction, delegating the prediction of diffuse color to the Gaussians allows us to better disentangle reflection effects from illumination. The full rendering equations we used can be written as follows,\n\n$$\n\\begin{aligned}\nc_{\\text{diffuse}} &= (1 - m)c_d, \\\\\\\\\nc_{\\text{specular}} &= \\left(((1 - m) \\cdot 0.04 + m \\cdot a) \\cdot F_1 + F_2 \\right) \\cdot \\text{env}(\\omega_i, r), \\\\\\\\\nF_1,\\ F_2 &= \\text{lookuptable}(\\omega_i, r), \\\\\\\\\nc_{\\text{final}} &= c_{\\text{diffuse}} + c_{\\text{specular}}\n\\end{aligned}\n$$\n\nwhere $ \\text{env}(\\omega_i, r) $ denotes querying from the mip-mapped environment cube maps using the incident direction and roughness. For indirect illumination, this term is replaced by a combination of the residual color and the color obtained via 2D Gaussian ray tracing. $ F_1, F_2 $ are obtained through table lookup based on the incident direction and roughness. From the above equation, we can see that the albedo component contributes little to the specular color when the metallic value is small. That is why we claim that the contribution of albedo to non-reflective surfaces is minimal. Through this way, we do not need to devote much attention to learning albedo in non-reflective regions, which is why we argue that our modeling strategy significantly reduces the optimization difficulty of material decomposition.\n\nTo further demonstrate this, we compare our method with inverse rendering methods which use albedo to compute both diffuse and specular components on the real-world Mip-NeRF 360 dataset. As reported in the table below, our superior results highlight the advantages of our decomposition approach in complex real-world scenes.\n\n\n| Methods            | PSNR$\\uparrow$ | SSIM$\\uparrow$ | LPIPS$\\downarrow$ |\n|--------------------|----------------|----------------|-------------------|\n| GS-IR | 25.38          | 0.757          | 0.267             |\n| GS-ID   | 25.91          | 0.771          | 0.264             |\n| IRGS     | 21.93          | 0.525          | 0.428             |\n| Ours                 | **27.06**      | **0.809**      | **0.181**         |\n\n**2. The choice of $M_0$ and $\\Gamma$.**  \n\nIn our implementation, $\\Gamma$ is set to 1 or 0 according to whether the average metallic value in a region is larger than 0.5 or not, practically. And $M_0$ is set to be the top 30\\% of metallic values in a region, practically. We conduct additional ablation studies on these two thresholds, as reported below.\n\n| | $M_0$      | $M_0$      | $M_0$      | $M_0$      | $M_0$      | $\\Gamma$    | $\\Gamma$    | $\\Gamma$    | $\\Gamma$    | $\\Gamma$    |\n|-------------------|------------|------------|------------|------------|------------|-------------|-------------|-------------|-------------|-------------|\n|   Metric    | 10\\%       | 30\\%       | 50\\%       | 70\\%       | 90\\%       | 0.1         | 0.3         | 0.5         | 0.7         | 0.9         |\n| **PSNR$\\uparrow$** | 23.435     | **23.438** | 23.433     | 23.426     | 23.414     | 23.288      | **23.444**  | 23.438      | 23.437      | 23.353      |\n| **SSIM$\\uparrow$** | 0.6436     | **0.6439** | 0.6434     | 0.6427     | 0.6412     | 0.6329      | 0.6438      | **0.6439**  | **0.6439**  | 0.6378      |\n| **LPIPS$\\downarrow$** | 0.1871  | **0.1870** | 0.1875     | 0.1891     | 0.1897     | 0.1943      | 0.1874      | **0.1870**  | 0.1873      | 0.1917      |\n\nFor $M_0$, too small threshold makes the reflection strength prior ineffective, while too large threshold results in surfaces with overly strong reflection strength, both of which harm the modeling of accurate reflection effects. For $\\Gamma$, too small value causes most regions to be wrongly identified as highly reflective surfaces, while too large value fails to identify true surfaces with high reflectiveness, both of which hinders the learning of material decomposition. Overall, however, our method remains robust to the choice of these thresholds.\n\n**3. Visualization of occlusion and indirect illuminance.**  \n\nDue to the policy restriction, we are unable to show visualization of the occlusion maps or indirect illumination. To quantitatively evaluate the results, we compute ground truth binary indirect illumination masks and occlusion masks for each view on the ground truth meshes. We then evaluate the rendering quality in indirect regions and the accuracy of binarized occlusion maps on ShinyBlender and GlossySynthetic datasets. We compare our method with both reflection modeling method Ref-Gaussian and inverse rendering methods including TensoIR, Relightable3DGS, GS-IR. For fair comparison, we add monocular normal priors for all baseline methods. \n\nThe results reported in the table below significantly show our advantages in approximating indirect illumination and localizing occlusions. The reason is that our multi-view material inference and environment modeling are end-to-end differentiable with respect to geometry, which allows for joint optimization of geometry, appearance, and material decomposition. In contrast, baseline methods typically first optimize geometry and then use the fixed geometry to predict occlusion and perform inverse rendering, which makes them less flexible in handling complex regions.Additionally, the material modeling strategy adopted in inverse rendering methods is not well-suited for highly reflective scenes, as we analyzed in our response to the first question \"About the albedo term\". We will add more visualization results in terms of indirect illumination and occlusion maps in the revised version.\n\n\n| | Indirect | Illumination  //// | Occlusion | Maps   |\n|--------------------|-------------------------------------|--------------------------------------|----------------------------------|----------------------------------|\n| **Methods**            | **PSNR$\\uparrow$** |  **SSIM$\\uparrow$** |  **Accuracy$\\uparrow$** |  **F1-score$\\uparrow$** |\n| TensoIR            | 18.63                               | 0.828                                | 0.7514                           | 0.3944                           |\n| Relightable3DGS    | 17.26                               | 0.827                                | 0.7409                           | 0.3480                           |\n| GS-IR              | 15.06                               | 0.754                                | 0.7153                           | 0.3065                           |\n| Ref-Gaussian       | 24.60                               | 0.907                                | 0.9133                           | 0.8279                           |\n| Ours               | **27.91**                           | **0.951**                            | **0.9492**                       | **0.9244**                       |\n\n**4. About numerical results in Tab. 2.**  \n\nSorry for the mistake. We wrongly reported the third row (w/o $\\mathcal{L}_n$) of ShinyBlender dataset in Tab. 2. It should be 35.37 / 0.975 / 0.050. We will correct this in the revised version.\n\n**5. About normal prior.**  \n\nThe normal prior is helpful for better convergence of geometry in the early training stage, which indeed benefits subsequent multi-view material inference and environment modeling. However, our method does not heavily rely on the normal prior. As shown in the third row of Tab. 2 of the main paper, even without the normal prior, our method still achieves state-of-the-art results on the ShinyBlender dataset. In fact, even without perfect geometry, our multi-view material inference and environment modeling continue to improve the geometry in later training stages. This is because our material constraints and environment modeling are differentiable to the depth and normal, enabling end-to-end joint optimization of geometry, appearance, and material properties for improved overall performance.  \n\nWe further change different monocular normal estimation models and report our rendering results with different normal priors, as reported in the table below. We achieve similar outstanding results based on various normal priors, which demonstrates our robustness to imperfect normal priors.\n\n| Methods                 | PSNR$\\uparrow$ | SSIM$\\uparrow$ | LPIPS$\\downarrow$ |\n|-------------------------|----------------|----------------|-------------------|\n| GeoWizard[1]      | 23.37          | 0.638          | 0.193             |\n| StableNormal[2] | 23.41          | 0.640          | 0.189             |\n| Metric3D-v2[3]   | **23.44**      | **0.644**      | **0.187**         |\n\n[1]. Xiao Fu, Wei Yin, et al. Geowizard: Unleashing the diffusion priors for 3d geometry estimation from a single image. In European Conference on Computer Vision, pages 241–258. Springer, 2025.\n\n[2]. Chongjie Ye, Lingteng Qiu, et al. StableNormal: Reducing diffusion variance for stable andsharp normal. ACM Transactions on Graphics (TOG), 43(6):1–18, 2024.\n\n[3]. Mu Hu, Wei Yin, Chi Zhang, et al. Metric3d v2: A versatile monocular geometric foundationmodel for zero-shot metric depth and surface normal estimation. IEEE Transactions on PatternAnalysis and Machine Intelligence, 2024."}]

Author rebuttal:

Meta-review: This paper presents MaterialRefGS, a novel framework that significantly improves the rendering of reflective scenes in Gaussian Splatting by introducing multi-view material consistency constraints and a reflection strength prior. Reviewers were impressed by the state-of-the-art visual quality and the novelty of the core ideas. The authors provided an exceptionally detailed rebuttal with new quantitative experiments and visualizations that successfully addressed all initial reviewer concerns, leading to a clear and unanimous consensus for acceptance.
Decision: Accept (poster)