Normal-GS: 3D Gaussian Splatting with Normal-Involved Rendering
Authors: Meng Wei
Qianyi Wu
Jianmin Zheng
Hamid Rezatofighi
Jianfei Cai
Venue: NeurIPS 2024 poster (2024)
Keywords: neural rendering
3D Gaussian Splatting
neural radiance field
computer vision
computer graphics
Primary area: machine_vision
TL;DR: We propose a normal-invovled rendering strategy for 3DGS, termed Normal-GS, which help enhance both the rendering quality and the normal estimation accuracy.

Abstract: Rendering and reconstruction are long-standing topics in computer vision and graphics. Achieving both high rendering quality and accurate geometry is a challenge. Recent advancements in 3D Gaussian Splatting (3DGS) have enabled high-fidelity novel view synthesis at real-time speeds. However, the noisy and discrete nature of 3D Gaussian primitives hinders accurate surface estimation. Previous attempts to regularize 3D Gaussian normals often degrade rendering quality due to the fundamental disconnect between normal vectors and the rendering pipeline in 3DGS-based methods. Therefore, we introduce Normal-GS, a novel approach that integrates normal vectors into the 3DGS rendering pipeline. The core idea is to model the interaction between normals and incident lighting using the physically-based rendering equation. Our approach re-parameterizes surface colors as the product of normals and a designed Integrated Directional Illumination Vector (IDIV). To optimize memory usage and simplify optimization, we employ an anchor-based 3DGS to implicitly encode locally-shared IDIVs. Additionally, Normal-GS leverages optimized normals and Integrated Directional Encoding (IDE) to accurately model specular effects, enhancing both rendering quality and surface normal precision. Extensive experiments demonstrate that Normal-GS achieves near state-of-the-art visual quality while obtaining accurate surface normals and preserving real-time rendering performance.

Normal-GS: 3D Gaussian Splatting with Normal-Involved Rendering
Meng Wei!
Qianyi Wu'
Jianmin Zheng² Hamid Rezatofighi' Jianfei Cail
NNayyang Technological University {meng. vei,qianyi wu, hamid. rezatof ighi, jiinnfin. amonash. edu (AS {ASJMZheng)) @ntu. edu.
Monash Univeristy
,
Abstract
Rendering and reconstruction long-standing topics in computer vision and graphics. Achieving both high rendering quality and accurate geometry is a chal- lenge. Recent advancements in 3D Gaussian Splatting (3DGS) have enabled high-fidelity novel view synthesis at real-time speeds. However, the noisy and discrete nature of 3D Gaussian primitives hinders accurate surface estimation. Previous attempts to regularize 3D Gaussian normals often degrade rendering quality due to the fundamental disconnect between normal vectors and the ren- dering pipeline in 3DGS-basse methods. Therefore, we introduce Normal-GS, novel approach that integrates normal vectors into the 3DGS rendering pipeline. The core idea is to model the interaction between normals and incident lighting using the physically-baset rendering equation. Our approach re-parameterizes surface colors as the product of normals and designed Integrated Directional Illumination Vector (IDIV). To save memory usage and simplify the optimization, we employ an anchor-based 3DGS to implicitly encode locally-shared IDIVs. Ad- ditionally, Normal-GS, leverages optimized normals and Integrated Directional Encoding (IDE) to accurately model specular effects, enhancing both rendering quality and surface normal precision. Extensive experiments demonstrate that Normal-GS achieves
state-of-the-are visual quality while obtaining accurate surface normals and preserving real-time rendering performance.
Introduction
Radiance Fields have emerged as a prominent representation in 3D vision, largely propelled by pioneering advancements of Neural Radiance Fields (NeRF) [1. Despite their success, NeRF's are hindered by prolonged rendering times and cumbersome training processes. Recently, 3D Gaussian Splatting (3DGS) has been introduced represent Radiance Fields using millions of 3D Gaussians, each endowed with additional attributes such as opacity and color (2). 3DGS significantly enhances rendering efficiency through -UDA-accelerated rasterization [] achieving comparable superior fidelity to NeRF-based models. The advantages of 3DGS have garnered significant attention [4], prompting numerous studies aiming at enhancing its rendering capabilities (5-7) and improving the quality its underlying geometry [0]
However, critical issue continues to plague the development of 3DGS: the quality of appearance and geomeiry seemingly oscillates like a seexaw: Efforts to refine the geometry often involve the incorporation of regularization techniques [8, additional geometry supervision [2 13), which may compromise rendering fidelity. Conversely, advancements in appearance modeling, such as the integration of implicit networks (6) and sophisticated shading attributes [5) struggle to concurrently elevate both appearance and geometry quality, This raises a pertinent question: Is it feasible to achieve high-quality appearance while also capturing precise underlying geometry information in 3DGS?
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

PSNR: 33.46 11
PSNR: 33.01
83
PSNR: 29.32 46
PSNR: 33.43 50
Rendering quality
Normal
Ours
Ground Truth
SpecGaussian [5]
3DGS [2]
GShader [7]
Figure The "seesaw" characteristic between the rendering quality and the normal accuracy of
3DGS-basse methods. Our Normal-GS is able to efficiently achieve accurate normal estimation
while preserving competitive rendering quality. Our method successfully captures the normals
cover of the semi-transparent box behind the table.
Revisiting the classical rendering equation [14), we identify fundamental issue in the appearance- geometry conflict: the disconnection between surface normals and the rendering process in current 3DGS methods. Specifically, the existing methods render pixel colors by alpha-biending colored 3D Gaussians. The color of each Gaussian is queried by the function value among Spherical Harmon- ics [15] along the ray direction, which disregards the contributions of surface normals. This oversight hampers the ability achieve a balanced integration of appearance and geometry. Our framework draws inspiration from the conventional rendering equation, which incorporates surface normals into appearance calculations. Although prior attempts have explored this integration in contexts such inverse rendering [16 17 and specular appearance modeling 5), they leverage simplified model [18] or approximation approaches like split-sum [19] or global environmental map [20 21] to decompose appearance into several individual components with meticulous regularization terms, which either compromising in rendering quality or geometric accuracy. This challenge highlights the complexity of balancing various components for chysically-based rendering, prompting our design of a straightforward method to integrate normal information explicitly and simply into Gaussian appearance modeling.
this work, we propose a normal-involved shading for 3DGS, called Normal-GS, for high-quality rendering and accurate normal estimation, while maintaining real-time performance. Our method directly accounts for normal contributions in the 3DGS rendering pipeline, enabling gradient signals to backpropagate properly to better align 3D Gaussians. Specifically, Normal-Gr considers the normal and incident lighting interactions at the surface from the physically-based shading perspective and explicitly models normals' contributions for diffuse and specular components. By considering the low-frequency nature of the incident light field, we further implicitly model it with MLPs. With strategy, compared with the state-of-the-art (SOTA) 3DGS methods, achieve competitive rendering quality, more accurate normal information, and real-time rendering, see one example in Fig. Our contributions summarized below.
We propose a new formulation to involve normal information in appearance modeling for 3DGS, which strives for good balance between appearance and geometry modeling.
We leverage an effective structure to regularize our framework using anchor- based MI.s,, without introducing many extra regularization terms and achieving better quality.
Extensive experiments demonstrate our framework for obtaining superior view synthesis and normal estimation. Our design could serve as a plug-in component for 3DGS approaches.
Related Works
Radiance Fields: From Neural Radiance Fields to 3D Gaussian Splatting. Neural Radiance Field (NeRF) has significantly influenced the field of 3D vision with its impressive novel view synthesis capabilities D. NeRF catalyzed advancements in neural rendering, inspiring multitude of subsequent research focused on enhancing rendering quality under complex lighting and material conditions [2.224] 24), varying camera distributions (25-21 27), and scalability for large scenes (28

---

However, the implicit representation used in NeRF, which relies on a Multi-layer Perceptron (MLP) to compute density and radiance for any given 3D position and ray direction, necessitates extensive computations. This inefficiency has spurred efforts to enhance the practicality of NeRF, with notable strides made in accelerating its processing efficiency
32
Among these advancements, 3D Gaussian Splatting (3DGS) has emerged as a promising solution (2). By explicitly modeling 3D scenes using sets of 3D Gaussians and employing tile-based GPU rasterization, 3DGS achieves real-time rendering with competítive quality. Its success has spurred applications in areas like 3D generation 33 34
physical simulation [35,, and sparse view recon- struction [36] 371, despite its higher memory requirements for scene representation. Innovations such as feature anchoring in Scaffold-GS [6]. value pruning and quantization techniques [3 and mining spatial relationship [4 are among the efforts to reduce the memory footprint of 3DGS, thereby enhancing storage efficiency and rendering fidelity.
Geometry and Appearance in 3DGS: From Surface Reconstruction to Inverse Rendering. Geometry and appearance are central 3D reconstruction, and the discrete and explicit nature of 3DGS can result in noisy underlying geometry 88 10
44 SuGaR [8] was an initial attempt
to refine mesh extraction from 3DGS, applying regularization to align the Gaussian Splatting with the actual surface contours. To better define geometry, capturing normal information is essential, leading to innovations like transforming 3DGS into 2D Gaussian Splatting (2DGS) and Gaussian Surfels [10 44 These methods apply regularization to depth and normal rendering, ensuring that the Gaussians are appropriately distributed across surfaces. While these approaches improve surface reconstruction, they often sacrifice rendering fidelity in novel view synthesis due to the lack of a clear relationship between geometry and appearance
10 44).
In appearance modeling, the focus has shifted to inverse rendering for 3DGS, which aims to separate scene elements into materials, lighting, and geometries. This inherently unconstrained problem challenges traditional rendering equations. Techniques such as Monte Carlo integration, pased ray tracing T7 5 and baking [16] are employed handle the complex integrals, often relying
on simplified models like the Disney Bidirectional Reflectance Distribution Function (BRDF) [8]
and approximation methods such as split-sum [66
17
45
46) or environmental mapping 7 46
Despite their efforts, these simplifications generally result in lower rendering quality, as reflected in reduced Peak Signal-to-Noise Ratios (PSNR) [1
45 failing to match the original 3DGS.
It is worth noting that some concurrent works 47 48 have a similar motivation as ours. 48 simultaneously enhance geometry accuracy and rendering quality by incorporating a laborious dual- branch framework, which uses 3DGS for appearance rendering and an implicit neural surface for geometry production. While this approach integrates the strengths of each system, it hampered by slower training speeds relative to standalone 3DGS, due to its additional networks for surface representation. [47] also distills geometry information from an extra neural implicit surface network for deferred rendering. Our solution proposes new perspective from rendering standpoint to consider geometry and appearance simultaneously without an extra implicit network for the surface.
3 Method
Our goal is to simultaneously enhance both image quality and normal estimation capability of 3DGS, while maintaining real-time rendering. To achieve this, we first analyze the existing rendering pipeline in 3DGS to identify its limitations in not involving surface normals in color modeling (Sec. 3.I.. We then design normal-involved rendering strategy that aligns with the physically based rendering equation. Our method effectively models interactions between normals and incident lighting, parameterized by the proposed Integrated Directional Illumination Vectors (IDIV) (Sec. 3.2). We employ anchor- based GS to implicitly encode locally shared IDIVs to save memory and aid The training details of our framework are provided in Sec. 3.4
optimization (Sec. 3.3
3.1 Preliminary
3DGS models scenes using set of discrete 3D Gaussians, each defined by its spatial mean μ and covariance matrix Σ:
3D Gaussian Splatting (2)
G(p) = exp(- 1 (p- - -p -1))
(1)
3

---

The covariance matrix Σ is parameterized by scaling matrix S and rotation matrix R, such that RSSTR* ensuring remains positive semi-definite during optimization.
Each 3D Gaussian is associated with a color e and an opacity a. During rendering, these Gaussians are projected (rasterized) onto the image plane, forming 2D Gaussian splats G' (x:), described in (
The 2D Gaussian splats are sorted from front to back tile-wisely, and a-blending 49 is performed for each pixel æ to render its color follows:
C(x) = E co (L))
a= a
KEN ==1
where N specifies the number of 2D Gaussian splats covering the current pixel. Heuristic densification and pruning strategies [2] are employed to address potential under- and over-reconstruction and ensure multi-view consistency in rendered images.
The color of each Gaussian, c, is represented by Spherical Harmonics (SH) as kj" Y1 (Wivow) ) to provide view-dependent effects, where (1, m) the degree and order of the SH basis Yt", kj" is the corresponding SH coefficient, and wiew specifies the viewing direction. 3DGS usually uses maximum degree 1 of 3, formulating c(wviee) = -:: Em=_; (wovee).
In Eq. 2), standard 3DGS treats colors as intrinsic attributes, independent of surface normals. This independence prevents surface normals from receiving gradient signals during the backpropagation pass when optimizing surface colors. Moreover, this separation significantly undermines our goal of simultaneously enhancing image quality and normal estimation capability, since improvements in normals cannot contribute to the rendering quality in the forward shading pass of 3DGS.
3.2 Normal-Involved Shading Strategy
Physically Based Rendering. To integrate surface normals into both the backward and forward rendering passes of 3DGS effectively, we propose normallinvolveer rendering strategy. This strategy adopts principles from physically based surface rendering [14] in computer graphics, which models the out radiance T of surface point function of the incident lighting Tin and normals. Specifically, for each 3D point, its out-radiance in the outward direction wo = wview is defined as:
el c(wwiew) = (w)) = LF(W,) + [ Linlw))(w; : n (wi, wo) dws
where LE is the emitted radiance along the outward direction, the integral is defined over the upper hemisphere 2 of the surface; wi is the incident light direction; BRDF f-(-) describes how light is reflected from Wi to wo; the cosine term (wi n.) accounts for the light geometric attenuation.
Integrated directional illumination vectors for Lambertian objects. We first consider simple ideal case where the scene contains only Lambertian objects. For more complex materials, we handle them in Sec. 3.4. Given that the diffuse reflection of Lambertian objects is view-independent, the complicated BRDF function fr (-) can be reduced to a spatially varying albedo term kp. Omitting the emitted radiance Le, the rendering equation accounting for the diffuse reflectance Lp becomes:
[wn( kp (wi: n2) dwos kD [ Ln S(a
Lin [in(w))(w:: n.) dws.
We want to find way to explicitly re- pe-aammeetrrize Lp function of normal n. meanwhile avoiding complicated integrals to save time. Specifically, we want to represent Lp as a dot product between and some illumination vector I. Inspired by the traditional shape-from-shadee strategy (50),
extract the normal out of Eq. 4
LD= KD n Ug Linlwe)ae dwos
(5)
By defining a new term called "integrated directional illumination vector (IDIV)" as
I 1= / Lin (ww(w) dwi,
(6)
Ju+
we model the color for Lambertian objects as the albedo times dot product between IDIV and the surface normal: e = LD ln n L. Compared with the original 3DGS and the method 7 that directly use a constant diffuse color, our re- re-parameterization successfully accounts for the normal's contributions to the diffuse component of surface colors.

---

Splatting Attributes
pth unchor
A
Prod(s)
Geometry:
Appearance:
o: opacity
IDIV ko:albedo
Differentiable Tiled-based Rasterization
scaalling rotation
Ourput:
MLPs
Au
normal
depth
normal
nurmal from depth
Physical Baard Rendering
kth 3D Gaussian
Vivww
Lp Ls Lp = kD n
viewing direction
Ls ((410g(a)))m.f))
rendered
rendered RGB
Gl
RGB
Figure 2: Our normal-involved GS, Normal-GS, reparameterizes the original colors into the diffuse and specular components, Ln Ls (bottom). It models the diffuse component the dot product between the normal vector n and the Integrated Directional Illumination Vector (IDIV) I, and utilizes the Integrated Directional Encoding (IDE) [22] capture view-dependent specular effects. Inherent parameters are encoded implicitly by locally shared anchor Gaussian (left) and decoded using MLPs (top). Our method accounts for the contributions of normals to colors, effectively enhancing geometry accuracy and rendering quality.
Analysis. Our re-parameterizatioo of the diffuse color enables the normal vector to be computation- ally involved in the rendering pass. As such, during backpropagation, by the chain rule, the surface color could pass gradient signals to the normal vectors as follows
dL dC de dL dn de dn de
(kD 1).
Other image-basee methods [6 46) usually optimize a single environment map E and query this map using normals determine diffuse colors. Consequently, their gradients normals function by rotating the 3D Gaussians until the queried environment map color matches the pixel color, making the optimization process heavily dependent the quality of E. These methods assume the existence of a global environment map, which may not be valid for general real scenes. In contrast, our approach employs IDIVs capture local incident lighting, eliminating the reliance on such strong assumption and providing more flexible and accurate framework for optimizing surface normals.
3.3 Modeling Locally Shared Integrated Directional Illumination Vectors
Replacing 3D Gaussian colors with kp introduces additional free parameters. Therefore, is necessary to regularize the solution space of the introduced variables kp and I to stabilize the training process and avoid overfitting.
Regularizing the solution space. Xu et al. [50] address solution space constraints from opti- mization perspective. They have introduced several regularizers, such as total variation and Laplacian terms, to enhance the performance. However, their mesh-based method relies connected topology, which is incompatible with the discrete and sparse nature of 3D Gaussians. Although k-NN can be used find connected primitives, it severely hampers training speed. Moreover, since 3D Gaussians can be semi-transparent, directly applying their methods to our case unsuitable.
Instead, we approach the problem from neural rendering perspective. As observed in 6 42 51 52 3D Gaussians with similar textures usually cluster together. A similar situation applics to incident lighting, which tends to be locally shared and exhibits low-frequency variations. To avoid the redundancy of storing Integrated Directional Illumination Vectors (IDIVs) for each Gaussian and capture local coherence, them using MLPs. An additional advantage using MLPs to approximate IDIV's is their inherent smoothness in function approximations, aligning with our objectives.
propose to encode IDIVs with locally shared features and decode
In particular, we employ an anchor-based Gaussian Splatting (GS) method, Scaffold-GS 6 to implicitly represent IDIVs using locally shared features f., stored at anchors v and decode them using
5

---

global MLP, (f.)) ] where K IDIVs share a local feature f.. Therefore, we only need learn compact set per-anchor local features and use the MLP to generate per Gaussian IDIVs. significantly reducing the dimensionality of the problem. Further details on our model configurations can be found in the following Sec. 3.4 and Appendix A.2
3.4 Training Details
Our method adopts the established 3DGS pipeline 2. which first performs Structure-from-Moiinn [53] on imput images to generate sparse points. These points serve as initial 3D Gaussians and are optimized to model the scene.
Defining surface normals on 3D Gaussians. During optimization, 3D Gaussians often exhibit flatness around the surface areas, as observed in [5 45]. For a near-flat Gaussian ellipsoid, its shortest axis thus functions as the normal vector. However, the shortest axis of 3D Gaussians does not physically represent the actual surface normal. In contrast, implicit radiance fields leverage the gradient of densities naturally approximate surfaces, and the Signed Distance Function (SDF) has its gradients as surface normals by defimition. To make the shortest axis of 3D Gaussians align with the real surface normal, we employ a depth regularized loss term on normals, similar to 7
We first render the depth and normal images, (D.N), using the 3DGS tile-based rasterizer, which is achieved by replacing the color term in Eq. with depths and normals of 3D Gaussians, respectively, Then we compute the image-space gradients of the depth image V ((,,)) D and finally perform the cross product of the gradients, resulting in No. The depth regularized loss term on normals is defined as CN = 1 - -N ... Although there are other methods for improving normals, since our primary goal is to introduce normal -nnoole- shading method, we find this self-regularizer is enough for our models. Other methods are welcome to be combined with our strategy for future improvements.
Capturing specular effects. Our derivation on Sec. 3.2 is based on the Lambertian assumption. To make our framework more generalizable, similar to our parameterization of the diffuse term, we aim to express the specular term, L,, as function of the normal vector n. Unlike view-independent diffuse colors, specular reflectance is highly sensitive to the view direction due to the complex BRDF, (w4,,.). Since normal vectors are implicitly involved in the BRDF to capture effects such as the Fresnel effect [18], we cannot directly extract the normal vector as we did for IDIVs S.2 Inspired by the Ref-NeRF 22 we model the specular component as a function of the reflection direction of the viewing direction with surface normals involved: wp = 2( n.,n wa- By applying the Integrated Directional Encoding (IDE) 22 on the reflection direction, we have Ls (( (w))), n. f.). The final color of the Gaussian is the summation of LD and Ls. More details can be found in the Appendix.
in Sec. 3.2
Model architecture. Figure. 2 illustrates the whole process of our method. We follow 6 to define a set of anchor Gaussians {w). where each v associated with a position x. a local feature fa a scaling factor 8, and k offsets O for I nearby 3D Gaussians. Anchor-wiss features together with global MLPs employed predict attributes of 3D Gaussians. We utilize our normal-involver rendering to compute surface colors c = LD + Ls. Colors are finally gathered following the traditional 3DGS pipeline, together with rendered normals and depths for reffaregularization. The final loss L consists of the original photo-metric loss Lp used in 3DGS (2), the volume regularization loss Lval used in Scaffold-GS 6 and the self-regularized derth-normal loss Lx:
L = Lp + Avol + Aw
(8)
4 Experiments
We evaluated our Normal-GS method against several ofate-of-the-art 3DGS-based methods across multiple datasets, presenting both qualitative and quantitative results.
We selected the following baseline methods for comparison: 1) 3DGS 2 and Scaffold- GS 6 These served as the vanilla and baseline methods. 2) GaussianShuder (GShader) 7 and SpecGaussian [5]: These 3DGS-based methods modeled specular effects and used surface normals. Both methods computed the reflected direction w, of the viewing direction w.r.t the normal. GShader
Baselines.

---

GShader
3DGS
ScaffoldGS
SpeeCiaussian
Ours
Ground Truch
Figure 3: Qualitative comparisons the rendering quality and normal estimation. Our method produced clean normals estimation and preserved good rendering quality.
used w, query learned environment map for shading, while SpecGaussian used to query the learned Spherical Gaussians. Additionally, we designed a new baseline method based on Scaffold- GS. In the original Scaffold-GS, anchor features f, were passed into color MLP to predict 3D Gaussian colors. our new baseline method, we also fed normals into the same color MLP, implicitly incorporating normal information. We referred to this baseline method 3) ScaffoldGS w/ N.
Datasets and evaluation metries. We followed the original 3DGS 2 methodology and used the NeRF Synthetic MT. Mip-NeRF 360 (26), Tank and Temple (54). and Deep Blending [55) datasets to demonstrate the performance of our method. Due to licensing issues, we tested on out of 9 scenes in the Mip-NeRF 360 dataset. Following 2 26). we selected every 8th image for testing and used the remaining images for training. We reported PSNR, SSIM, and LPIPS to measure rendering quality and the Mean Angular Error (MAE) of normals on the NeRF Synthetic ( dataset.
Implementation details. We implemented our method in Python using the Py Torch framework [66] Our rasterization pipeline followed the CUDA-baser rasterizer described (2) We trained our models for 30k iterations, following the settings of baseline methods. Consistent with [2] 6 we set Avol = 0.001. For the depth-normal loss, we used Au = 0.01. Because the depth and normals were inaccurate at the start of training, added the depth-normal loss after training for 5k iterations. We will release our code after publication. More implementation details were included in the Appendix A.2
We tested our method and the baseline methods using their original released implementations with default hyperparameters on an NVIDIA RTX 3090 GPU with 24 GB of memory. To obtain normals for comparisons, we retrained most of the baseline methods, except for 3DGS, which provided its optimized data. For methods without defined normals, such as 3DGS 2 and Scaffold-GS [6), we used the shortest axis of 3D Gaussians as normals for evaluation, consistent with ours.
4.1 Results
We first present comprehensive comparison of rendering quality and normal estimation in Fig demonstrating that our method either matches or surpasses state-of-the-art (SOTA) approaches in metrics of PSNR/SSIM/LPIPS [T Notably, our approach significantly outperforms the bascline
7

---

Table Quantitative comparisons of the rendering quality, Our methods achieve comparable or even better results compared with the SOTA approach, SpecGaussian [5]. However, SpecGaussin performs much worse in normal estimation, while ours achieves a good balance between geometry and appearance quality. denotes the results quoted from their paper. GShader fails on the Deep Blending scene.
Mip-NoRF360
Tanks & Temples
Deep Bleading
Method
PSNR
SSIM
LPIPS
PSNR
SSIM
I.PPS
PSNR
SSIM
IPPPS
3DGS
23.142
29.405
ScaffoldGS
29.267
0.869
0.190
24,088
0.851
0.175
30.140
0.905
0.256
ScaffolaCS w/
29.177
0.870
0.192
23.976
0.852
0.176
30.163
0.905
0.263
GShader
26.060
0.825
0.233
21.262
0.785
0.254
19.159
0.769
0.453*
SpeCCanssian
29.287
0.864
0.196
24.502
0.855
0.175
30.114
0.905
0.252
Normal-GS (Ours)
29:341
0.869
0.194
24.219
0.854
0.174
30.187
0,910
0.252
a) Ground Truth
b) Ours
c) GShader
d) Environment Map
Figure 4: b) Our method successfully captures specular effects. c) GShader relying environment map performs poorly when d) its learned environment map fails. Scenes are taken from the Mip-NeRF 360 dataset 266] which have complicated lighting conditions.
method (ScaffoldGS) and its variant incorporating normals (ScaffoldGS w/ N) in almost all metrics. This comparison underscores the limitations of the implicit normal utilization in ScaffoldGS w/N, which fails to accurately model appearance. contrast, our method adopts an explicit modeling of color as function of normals, directly derived from the rendering equation. This explicit integration enhances both the forward and backward passes during training, providing clearer cues for the network to capture nuanced color details, thereby improving rendering quality.
Our approach also demonstrates superior rendering performance compared to methods like GShader [7]. which depend global environment maps to model specular effects, as evidenced in Tab. I The dependency of normal gradients on environment map quality, described by the equation dL dE dN
7 47). often leads inaccuracies in capturing detailed information, as shown in Fig and further evidenced by performance issues on challenging datasets like DeepBiending([5]] in Tab. Our framework's capability to learn local incident light field attributes tied to distinct anchors 3D space allows for more effective handling of complex lighting effects.
dL
d.
Although SpecGaussian (5] achieves competitive rendering quality through the use of Spherical Gaussian encoding and neural networks, it falls short in accurate normal modeling, as depicted in Fig This suggests the insufficiency of relying solely on neural networks for integrating normal information into geometric modeling. Our method, with its explicit integration of normals into the rendering process, not only maintains high rendering quality but also ensures the accuracy and smoothness of normals. Our approach underscores the crucial role of direct geometric involvement in the rendering pipeline for achieving both high geometric accuracy and visual fidelity. More results are available in Appendix A.3
To quantitatively evaluate normal estimation, conducted tests the Synthetic-NeRe dataset, which includes ground- -rrth normal data. As shown in Tab. a[a), our method outperforms others in the normal Mean Angular Error (MAE) metric. Visual results in Tab. (2(b) confirm our method's superior capability in capturing accurate geometry alongside high-fidelity rendering, as highlighted in the yellow box, showcasing the effectiveness of our design.
4.2 Ablation studies
Our contributions are significant in two key areas: the integration of Integrated Directional Illu- mination Vectors (IDIV) and the implementation of inchor-based regularization design. We
8

---

Table Comparison on SyntheticNeeRE dataset. (a) The numerical results on normal consistency, Our method achieved state-of otate-of-the-ar results among all methods. (b) We visualized the rendered normals for qualitative comparison. Our framework produced the best geometry.
Normals MAE.
SpecCaussian
S
45.98
ScaffoldGS o
25.56
GShader
?
23.56
Normal -S (Ours)
20.71
SpecGussian
SeaffoldGS
GShuder
Ours
Ground Truth
(a) Normals comparision
(b) Visualization of rendered normals
a) 3DGS
b) 3DGS w/ IDIV
Ours wo Ls
d) Ours Full
Ground Truth
PSNR 20.63
PSNR: $1.00
PSNRA
PSNR: $1.99
Figure 5: Ablation study about our proposed components. The IDIV together with our regulariza- tion strategy produces better quality in rendering quality and normal accuracy.
initially validate the impact of IDIV through an ablation study conducted within conventional 3DGS framework, detailed Sec. 3.2. Introducing IDIV necessitates additional parameters, requiring the adoption of regularization techniques such as Laplacian and Total- Varia-ariation loss, referenced in [50]. The outcomes, illustrated Fig. (a) and (b), demonstrate substantial enhancements in rendering quality and the smoothing of normals.
However, we found that the model's performance is highly sensitive to the tuning of loss weights. To address this, we implemented an anchor-basee IDIV regularization strategy in Sec. (3. with results illustrated in Fig. [5] (c). This approach employs neural networks model locally shared IDIVs, proving to be both robust and efficient in regularizing the solution space. Notably, the simplicity yet effectiveness of our final loss term, defined Eq. g underscores the potential of our design to enhance both geometric and rendering quality. When considering the specular component, our full (Fig. () ()) model achieves superior results.
We also performed quantitative ablation stud- ies on the DTU dataset (57) to further vali-
Table 3: Quantitative ablation studies on DTU,
date the effectiveness of our components. The DTU dataset contains 15 scenes, each with scanned geometries provided as ground truth. We conducted the ablation study by compar- ing the mean Chamfer distance (mCD) for geometry accuracy and PSNR for rendering quality evaluation. We followed 44 to pre-process the data, extract meshes and evaluate obtained results. As shown in Tab. 3, our base model Scaffold-GS (6. We add the self-regularized depth- normal loss, LN, as defined in Sec[3.4] 'hee results of (b) in Tab. 3 indicate that the depth-normal loss can significantly improve the geometry quality but ruin the rendering with more than 2dB PSNR drop. Then introduce the specular component into (b), which improve the rendering quality
Model
mCD +
PSNR f
(a): Scaffold-GS 66]
1.84
38.14
(b): (a) + Lx
0.95
35.90
(c): (b) + Lspecular
0.93
37.37
(d): Full model
0.94
37.63

---

Table 4: Rendering quality comparisons with NeRF-
Table Geometric and rendering qual- ity comparisons on the DTU dataset.
based methods on the Mip-NeRF 360 dataset.
Method
PSNR 1
SSIM
LPIPS +
Method
mCD
PSNR 1
Mip-NeRF 360 [26]
29.231
0.844
0.207
3DGS
1.96
35.76
Ref-NeRF 22
28.553
0.849
0.196
SuGaR 8
1.33
34.57
Zip-NeRF
58
30.077
0.876
0.170
2DGS
0.80
34.52
Ours
29.341
0.869
0.194
Ours
0.94
37.63
without compromising the geometry, which further assures the importance of involving normal into the rendering. Finally, we add the proposed IDIV to get our final model (d), which further improves the PSNR by more than 0.2 dB with similar geometry quality. Compared with the base model (a), our full model achieves a better balance between the geometry quality and the rendering fidelity.
4.3 Geometric reconstruction comparisons
In addition to evaluating normal accuracy, we compare the overall geometry quality of our method with existing 3DGS-based approaches 22 g 44 on the DTU dataset [57]. For fair comparison, follow the pipeline described in 44 and measure the mean Chamfer distance (mCD) and PSNR to assess both geometric reconstruction and rendering quality. The results for 3DGS 22 SuGaR (8]. and 2DGS [44] are directly adopted from [44. As Tab. billusiilsstrates, our method achieves outstanding rendering quality while maintaining better geometric quality than 3DGS and SuGaR. 2DGS, however, sacrifices some rendering fidelity in novel view synthesis due to the lack of a clear relationship between geometry and appearance, seen in Tab. 5 Unlike 2DGS, our method explicitly addresses the interactions between lighting and normals. It is also worth noting that 2DGS and our approach conceptually orthogonal and could potentially be combined for further improvements.
4.4 Comparisons with NeRF-based methods
Finally, for completeness, we conduct comparisons with existing state-of-the-ee NeRF-based methods, including Mip-NeRF 360 (26). Ref-NeRF (22), and Zip-NeRF [58]. In Tab. 4 we present their original results for 7 of the 9 scenes from the Mip-NeRF 360 dataset (26). aligning with our experimental setup. Our method performs the second best in terms of the PSNR and achieves comparable SSIM and LPIPS scores. However, the training time for Zip-NeRF exceeds 10 hours on high-performance GPUS. In contrast, 3DGS- based methods, including ours, are relatively fast and can support real-time rendering, offering a significant speed advantage over NeRF-based approaches.
5 Discussions
Conclusion. We introduce a novel shading technique within the 3D Gaussian Splatting (3DGS) framework to improve view synthesis and normal estimation. By integrating Directional Illumination Vectors (IDIV) as advanced attributes for color representation and utilizing an machor-based design with neural MLP for IDIV prediction, our approach achieves superior balance between rendering quality and geometric accuracy. This enhancement not only elevates image fidelity but also ensures precise normal estimations, pushing the boundaries of 3DGS technology in real-world applications.
Limitation and Future Work. We acknowledge existing limitations within our framework. Notably, the self-regularized normal loss proves suboptimal for certain outdoor scenes distance (as observed in the Appendix 7. negatively impacting rendering quality in these regions. This issue could potentially be addressed with more sophisticated normal guidance techniques. In future research, we plan to delve into more granular decomposition of the Integrated Directional Illumination Vectors (IDIV) and specular components. This exploration aims to enhance capabilities in texture editing and relighting, thereby broadening the practical applications of our framework.
Boarder Impact. As approaches require per-scene optimization obtain the 3D model for each scene, the computational resources for model training could be a concern for global climate change
10

---

References
Ben Mildenhall, Pratul Srinivasan, Matthew Tancik, Jonathan Barron, Ravi Ramamoorthi, and Ren Ng- Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65 (1):99-106, 2021.
[2] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkuehler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Gruphics, 42(4):1-14, August 2023. ISSN 0730-0301, 1557-7368.
doi: 10.1 10.154533
[3] M. Zwicker, H. Pfister, 2001. VIS '01., pages 29-538, 2001. doi: 10. 109IISSSLUL 964499
Baar, and M. Gross, Ewa volume splatting. In Proceedings Visualization,
[4] Tong Wu, Yu-Jie Yuan, Ling-Xian Zhang, Yang. Yan-Pei Cao, Ling-Qi Yan, and Lin Gao. Recent advances in 3d gaussian splatting. arXiv preprint arXiv:2403. 1tt.:2403.111334, 2024.
[5] Ziyi Yang, Xinyu Gao, Yangtian Sun, Yihua Huang, Xiaoyang Lyu, Wen Zhou, Shaohui Jiao, Xiaojuan Qi, and Xiaogang Jin. spee-gaussian Anisotropic view-dependent appearance for 3d gaussian splatting. arXiv preprint arXiv:2402.156 15870, 2024.
[6] Tao Lu, Mulin Yu, Linning Xu, Yuanbo Xiangli, Limin Wang, Dahua Lin, and Bo Dai. Scaffold-gs: Structured 3d gaussians for view-adaptive rendering. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
[7] Yingwenqi Jiang, Jiadong Tu, Yuan Liu, Xifeng Gao, Xiaoxiao Long, Wenping Wang. and Yuexin Ma. Gaussianshader: 3d gaussian splatting with shading functions for reflective surfaces. In Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
[8] Antoine Guédon and Vincent Lepetit. Sugar: Surface-aligned gaussian splatting for efficient 3d mesh reconstruction and high-quality mesh rendering. In Conference Computer Vision and Pattern Recognition (CVPR), 2024.
[9] Kai Cheng, Xiaoxiao Long, Kaizhi Yang. Yao Yao, Wei Yin, Yuexin Ma, Weaping Wang, and Xuejin Chen. Gaussianpro: 3d gaussian splatting with progressive propagation. arXiv preprint arXiv:2402. 14650, 2024.
[10] Pinxuan Dai, Jiamin Xu, Wenxiang Xie, Xinguo Liu, Huamin Wang, and Weiwei Xu. High-quality surface reconstruction using gaussian surfels. In SIGGRAPH 2024 Conference Papers. Association for Computing Machinery, 2024. doi: 10. 10.1145/3641519.366 11..364441..
[11] Hanlin Chen, Chen Li, and Gim Hee Lee. Neusg: Neural implicit surface reconstruction with gaussian splatting guidance. arXiv preprint arXiv:23/2.00886 2023.
[12] Jiahe Li, Jiawei Zhang, Xiao Bai, Jin Zheng, Xin Ning, Jun Zhou, and Lin Gu. Dngaussian: Optimizing sparse-view 3d gaussian radiance fields with global-local depth normalization. In Conference on Computer Vision and Pottern Recognition (CVPR), 2024.
[13] Matias Turkulainen, Xuqian Ren, laroslav Melekhov, Otto Seiskari, Esa Rahtu, and Juho Kannala. Dn- splatter: Depth and normal priors for gaussian splatting and meshing. arXiv preprint arXiv:2403.1 17822, 2024.
[14] James T Kajiya. The rendering equation. In Proceedings of the 13th annual conference on Computer graphics and interactive techniques, pages 143-150, 1986.
[15] Ernest William Hobson. The theory of spherical and ellipsoidal harmonics. CUP Archive, 1931.
[16] Zhihao Liang, Qi Zhang, Ying Feng, Ying Shan, and Kui Jia. Gs-ir: 3d gaussian splatting for inverse rendering. In Conference Computer Vision and Pattern Recognition (CVPR), 2024.
[17] Jian Gao, Chun Gu, Youtian Lin, Hao Zhu, Xun Cao, Li Zhang, and Yao Yao. Relightable 3d gaussian: Real-time point cloud relighting with brdf decomposition and ray tracing. Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
[18] Brent Burley and Walt Disncy Animation Studios. Physically- shading at disncy. In Acm Siggraph, volume 2012, pages 1-7, vol. 2012, 2012.
[19] Brian Karis and Epic Games. Real shading in unreal engine 4. Proc. Physically Based Shading Theory Practice, 4(3):1, 2013.
11

---

[20] Ned Greene. Environment mapping and other applications of world projections. IEEE computer graphics and Applications, 6(11):21-29, 1986.
[21] Ravi Ramamoorthi and Pat Hanrahan. An efficient representation for irradiance environment maps. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques, pages 497-500, 2001.
[22] Dor Verbin, Peter Hedman, Ben Mildenhall, Todd Zickler, Jonathan T Barron, and Pratul Srinivasan. Ref-nerf: Structured view-dependent appearance for neural radiance fields. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5481-5490, IEEE, 2022.
[23] Yao Yao. Jingyang Zhang. Jingbo Liu, Yihang Qu Tian Fang. David McKinnon, Yanghai Tsin, and Long Quan. Neilf: Neural incident light field for physically- material estimation. In European Conference on Computer Vision, pages 700-716. Springer, 2022.
[24] Jingyang Zhang, Yao Yao, Shiwei Li, Jingbo Liu, Tian Fang. David McKinnon, Yanghai Tsin, and Long Quan. Neilf++: Inter-reflectable light fields for geometry and material estimation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3601-3610 2023.
[25] Jonathan Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, and Pratul P Srinivasan. Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5855-5864, 2021.
[26] Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Mip-nerf' 360: Unbounded anti-aliased neural radiance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5470-5479, 2022.
[27] Wenbo Hu, Yuling Wang, Lin Ma, Bangbang Yang, Lin Gao, Xiao Liu, and Yuewen Ma. Tri-miprf: Tri-mip representation for efficient anti aliasanis neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 19774 19783, 2023.
[28] Haithem Turki, Deva Ramanan, and Mahadev Satyanarayanan. Mega-nerf: Sealable construction of large-scale nerfss for virtual thy-throughs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12922-12931, 2022.
[29] Peng Wang, Yuan Liu, Zhaoxi Chen, I Lingjee Liu, Ziwei Liu, Taku Komura, Christian Theobalt, and Wenping Wang. F2-nerf: Fast neural radiance field training with free camera trajectories. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pages 41 4150-4159, 2023.
[30] Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Zip-nerf: Anti- aliased grid based neural radiance fields. In Proceedings the IEEE/CVF International Conference on Computer Vision, pages 19797-19705, 2023.
[31] Alex Yu, Ruilong Li, Matthew Tancik. Hao Li. Ren Ng, and Angjoo Kanazawa. Plenoctrees for real-time rendering neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5752-5761, 2021.
[32] Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with a multiresolution hash encoding. ACM transactions on graphics (TOG), (14):1111,, 2022.
[33] Jiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu, and Gang Zeng. Dreamgaussian: Generative gaus- sian splatting for efficient 3d content creation. In The Twelfth International Conference on Learning Representations, 2023.
[34] Taoran Yi. Jiemin Fang. Junjie Wang, Guanjun Wu, Lingxi Xie, Xiaopeng Zhang, Wenyu Liu, Qi Tian, and Xinggang Wang. Gaussiandreamer: Fast generation from text 3d gaussians by bridging 2d and 3d diffusion models. In CVPR, 2024.
[35] Tianyi Xie, Zeshun Zong, Yuxin Qiu, Xuan Li, Yutao Feng, Yin Yang, and Chenfanfu Jiang. Physgaussian: Physics-integrated gaussians for generative dynamics. In CVPR, 2024.
[36] David Charatan, Sizhe Li, Andrea Tagliasacchi, and Vincent Sitzmann. pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction. In Conference on Computer Vision and Patter Recognition (CVPR), 2024.
[37] Yuedong Chen, Haofei Xu, Chuanxia Zheng, Bohan Zhuang, Marc Pollefeys, Andreas Geiger, Tat-Jen Cham, and Jianfei Cai. Mvsplat: Efficient 3d gaussian splatting from sparse nie images. arXiv preprint 14222 2024.
12

---

[38] Joo Chan Lee, Daniel Rho, Xiangyu Sun. Jong Hwan Ko, and Eunbyung Park. Compact 3d gaussian representation for radiance field. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, 2024.
[39] Zhiwen Fan, Kevin Wang, Kairun Wen, Zehao Zhu, Dejia Xu, and Zhangyang Wang. Lightgaussian: Unbounded 3d gaussian compression with 15x reduction and 200+ fps. arXiv preprint a1X1422311.17245, 2023.
[40] Simon Niedermayr, Josef Stumpfegger, and Rüdiger Westermann, Compressed 3d gaussian splatting for accelerated novel view synthesis. arXiv preprint arXiv:2401.0244 222446, 2023.
[41] Wieland Morgenstem, Florian Barthel, Anna Hilsmann, and Peter Eisert. Compact 3d scene representation via self-organizing gaussian grids, arXiv preprint urXiv:2312.13299 2023.
[42] Yihang Chen, Qianyi Wu. Jianfei Cai, Mehutash Harandi, and Weiyao Lin. Hac: Hash-grid assisted context for gaussian splatting compression. arXiv preprint WxXi:2243.145580 2024.
[43] Runyi Yang. Zhenxin Zhu, Zhou Jiang, Baijun Ye, Xiaoxue Chen, Yifei Zhang, Yuantao Chen, Jian Zhao, and Hao Zhao. Spectrally pruned gaussian fields with neural compensation. arXiv preprint arXiv:2405.00676 2024.
[44] Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao. 2d gaussian splatting for geometrically accurate radiance fields. In SIGGRAPH 2024 Conference Papers. Association for Computing Machinery, 2024. doi: 10. 00.1145/6411519.3554422
[45] Yahao Shi. Yanmin Wu, Chenming Wu, Xing Liu, Chen Zhao, Haocheng Feng, Jingtuo Liu, Liangjun Zhang. Jian Zhang, Bin Zhou, et al. Gir: 3d gaussian inverse rendering for relightable scene factorization. arXiv preprint ar arXiv:2312.05133. 2023.
2023.
[46] Keyang Ye, Qiming Hou, and Kun Zhou. 3d gaussian splatting with deferred reflection. arXiv preprint arXiv:2404. 18454, 2024.
[47] Tong Wu, Jia-Mu Sun, Yu- Kun Lai, Yuewen Ma, Leif Kobbelt, and Lin Gao. Deferreggs: Decoupled and editable gaussian splatting with deferred shading. arXiv preprint arXiv:2404.09411 2024.
[48] Mulin Yu, Tao Lu, Linning Xu, Lihan Jiang, Yuanbo Xiangli, and Bo Dai. Gsdf: 3dgs meets sdf for improved rendering and reconstruction. arXiv preprint arXiv:2403. 16964, 2024.
[49] Georgios Kopanas, Julien Philip, Thomas Leimkühler, and George Dreutkiis Point-based neural rendering with per-view optimization. Computer Graphics Forum (Proceedings of the Eurographics Symposium on Rendering), 40(4). June 2021. URL http::tttp:/wwtt sop. inria. /ttp:///eee///ss///////////////
[50] Di Xu, Qi Duan, Jianmin Zheng, Juyong Zhang, Jianfei Cai, and Tat-Jen Cham. Shading basedd surface detail recovery under general unknown illumination. IEEE transactions on pattern analysis and machine intelligence, 40(2):423 436, 2017.
[51] Mingqiao Ye, Martin Danclljan, Fisher Yu, and Lci Ke. Gaussian grouping: Segment and edit anything in 3d scenes. arXiv preprint arXiv:23/2.00732. 2023.
[52] Evangelos Ververas, Rolandos Alexandros Potamias, Jifei Song, Jiankang Deng, and Stefanos Zafeirion. Sags: Structure-aware 3d gaussian splatting. arXiv preprint arXiv:2404. 1944:, 2024.
[53] Johannes Lutz Schönberger and Jan-Michael Frahm. Structure-from-mmoe revisited. In Conference on Computer Vision and Puttern Recognition (CVPR), 2016.
[54] Arno Knapitsch, Jaesik Park, Qian-Yi Yi Zhou, and Vladlen Koltun. Tanks and temples: Benchmarking large-scale scene reconstruction. ACM Transactions Graphics, 36(4). 2017.
[55] Peter Hedman, Julien Philip, True Price, Jan- Man-Michael Frahm, George Drettakis, and Gabriel Brostow, Deep blending for free-viewpoin image- based rendering. ACM Transactions on Graphics (Proc. SIGGRAPH Asia), (11:6)):57:1227::::: 2018.
[56] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang. Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang. Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high- performance deep learning library. In Advances in Neural Information Processing Systems 32, pages 8024 8035. Curran Associates, Inc., 2019. URL http: htttp://papers neurips. www.wee....ppppppeeoeoeeee 9015 2016-pytorch--o an- ww--inprattietetye high- pwwformance- deep- learning- cawaanninn--ibrrry...oo. sibrary.pd.
13

---

[57] Rasmus Jensen, Anders Dahl, George Vogiatzis, Engil Tola, and Henrik Aanæs, Large scale multi-view stereopsis evaluation. In 2014 IEEE Conference Computer Vision and Pattern Recognition, pages 406-413. IEEE, 2014.
[58] Jonathan T. Barron, Ben Mildenhall. Dor Verbin, Pratul P. Srinivasan, and Peter Hedman. Zip-nerfi Anti-aliased grid-based neural radiance fields. (CCV.2023
14

---

A Appendix
A.1 Modeling the specular component using Integrated Directional Encodings (IDE)
As discussed in Scc. 3.4 utilize the Integrated Directional Encoding (IDE) of the Ref-NeRF P model the specular component. Specifically, the IDE encodes the distribution the reflection vectors using of spherical harmonics (Y?"), J. This distribution follows the von Mises- (vMF) distribution, with the reflection vector w, as the center and the inverse roughness x = 1/p as the concentration parameter. 6. describes the width of the vMF distributions, corresponding to the surface roughness. With this vMF distribution @ vMF(@,, k).
Verbin et al. [22] define the IDE
IDE K) = (RS- (V (4)) (1,m) E ML
(9)
with Mr = m). : - L,... 2 m = 0, O,...!!.. 13. Verbin et al. [22) further show that the above expectation equals A((x)))))) (w)) with
(x) exp 1(1 (1+1),
(10)
2x
For detailed derivation and illustration, we recommend referring their original paper
22
After obtaining the IDE of the reflection direction, pass dIDE (wr). n and the adjacent anchor feature f as imputs a global MI.P generating the final specular color I.g ((owe n, fa)- Details about our MLP are provided as follows.
A.2 Implementation details
Neural network architectures We employ an mecorr-baee GS method, Scaffold-GS (6. to predict tributes of 3D Gaussians. The overall architecture closely follows original implementation, with modifications made to the color modeling components. Specifically, we introduce two additional components to predict diffuse Ln and specular colors Ls respectively. Other components including the updating strategy remain unchanged. The neural network architecture of the newly introduced MLPs is consistent with those used in [6]. These MLPs fully connected networks with one hidden layer and utilize the ReLU activation function. The learning rates and feature dimensions for these MLPs are set to match original color MLP used by Scaffold-GS.
Defining normals on 3D Gaussians Recall that the covariance matrix Σ of 3D Gaussian is parameterized by a scaling matrix S and a rotation matrix R. = RS R²,, formulating a 3D ellipsoid. Let SS" diag(sf, s)), the eigenvalues of A are thus (0f, *2, sd). with the dummy eigenvectors (e1, E2, ea). In RSS Rd RAR"", the rotation matrix R [ri; T2; ra] acts like rotating eigenvectors of A and preserves its eigenvalues. Consequently, the eigenvectors of the covariance matrix indicate the principal axes of the Gaussian ellipsoid and the eigenvalues correspond to the axes' lengths. The shortest axis of the Gaussian ellipsoid thus corresponds to the ith column of the rotation matrix, where
min(s1, 83).
Loss Our loss defined as C = Lp + Aual + Ax with Ay = 0.01 and Avel = 0.001. The photometric Joss, defined in (2, is Cp = (1 ASSIM) >ssi CD-SSIM with ASSIM = 0.2. The volumetric loss, as defined Ng Prod(s:), where Prod(.) stands for the product of the values of a vector and s; is the scales of each 3D Gaussian. Our self-regularized depth-normal loss Lx has been discussed in Sec. 3.4
in (6) is Lyal Ee
A.3 More results
We include per-scene results, following the same evaluation strategies and metrics discussed in Sec. provide additional qualitative visualization results of baseline methods and ours in Fig. 6
4
We also
Table 6: Per-scene PSNR f for the Mip-NeRF360 26 dataset.
Method
bicycle
garden
stump
room
counter
kitchen
bonsai
Avg.
3DGS
25.25
27,41
26.55
30.63
28.70
30.32
31.98
28.691
ScaffoldGS
25.18
27.51
26.56
31.92
29.56
31.53
32.61
29.267
ScaffoldGS w/ N
25.20
27.47
26.56
31.84
29.47
31.41
32.29
29.177
GShader*
22.74
26.44
24.66
24.82
26.60
27.68
29.48
26.060
SpecGaussian
24.96
27.43
26.22
31.80
29.60
31.68
33.32
29.287
Ours
25.23
27.64
26.54
31.99
29.71
31.61
32.67
29.341
depth-normal normal loss, which deteriorates the performance of normal estimation and rendering at distant regions. Figure. 7 iillustrates failure case of methods.
Limitations Our method uses the regulaegl depth
15

---

GShader
3DGS
ScaffoldGS
SpecGanssian
Ours
Ground Truth
Figure 6: More qualitative comparisons the rendering quality and normal estimation. Our method produced clean normals estimation and preserved good rendering quality.
A.4 Metric definition.
Besides common-used image quality metric novel view synthesis evaluation the main context, we provide geometry metric evaluation about normal MAE scores for qualitative comparison. Here we provide the definition of it.
Denote the rendering normal as ' and the ground between these images. Assume there is mask M for object definition under each testing view, to
ground-truth truth 2D normal map Ner. we aim to evaluate the similarity
16

---

Table 7:
Per-scene SSIM f for the Mip-NeRF360 266
dataset.
Method
bicycle
garden
stump
room
counter
kitchen
bonsai
Avg.
3DGS
0.771
0.868
0.775
0.914
0.905
0.922
0.938
0,870
ScaffoldGS*
0.758
0.863
0.766
0.922
0.912
0.926
0.941
0.869
ScaffoldGS w/ N
0.759
0.862
0.767
0.923
0.912
0.927
0.943
0.870
GShader
0.692
0.843
0.702
0.846
0.875
0.897
0.917
0.825
SpecGaussian*
0.753
0.852
0.754
0.918
0.904
0.924
0.943
0.864
Ours
0.757
0.863
0.764
0.922
0.911
0.926
0.943
0,869
Table 8: Per-scene LPIPS
for the Mip-NeRF330 226
dataset.
Method
bicycle
garden
stump
room
counter
kitchen
bonsai
Avg.
3DGS
0.205
0.103
0.210
0.220
0.204
0.129
0.205
0.182
ScaffoldGS
0.232
0.118
0.236
0.213
0.198
0.128
0.204
0.190
ScaffoldGS w/ N
0.236
0.121
0.241
0.213
0.201
0.128
0.206
0.192
GShader
0.278
0.130
0.277
0.305
0.243
0.160
0.242
0.233
SpeeGaussian*
0.235
0.131
0.242
0.220
0.211
0.131
0,200
0.196
Ours
0.243
0.120
0.244
0.216
0.202
0.129
0.209
0.194
Table 9: Quantitative comparisons of the rendering quality the Tanks and Temples dataset 54
Truck
Train
Method
PSNR f
SSIM f
LPIPS
PSNR f
SSIM f
LPIPS
3DGS
25.187
0.879
0.148
21.097
0.802
0.218
ScaffoldGS
25.825
0.881
0.145
22.350
0.820
0.206
ScaffoldGS w/ N
25.738
0.882
0.147
22.214
0.821
0.205
GShader
23.696
0.843
0.193
18.828
0.727
0.315
SpecGaussia**
26.112
0.885
0.145
22.893
0.825
0.205
Ours
25.858
0.882
0.145
22.580
0.826
0.203
Table 10: Quantitative comparisons of the rendering quality on the Deep Blending dataset 55
Playroom
Dr. Johnson
Method
PSNR f
SSIM t
LPIPS
PSNR 1
SSIM 1
LPIPS
3DGS
30.044
0.906
0.241
28.766
0.899
0.244
ScaffoldGS
30.655
0.906
0.256
29.624
0.904
0.257
ScaffoldGS w/
30.785
0.907
0.263
29.541
0.903
0.263
"Shader
21.007
0.807
0.406
17.312
0.730
0,499
SpecGaussian*
30.589
0.905
0.252
29.639
0.904
0.252
Ours
30.846
0.913
0.260
29.529
0.906
0.257
(a) Ground Truth
(b) Rendered
(c) Normal
Figure The failure case of our method. Our method performs suboptimally for scenes at a distance.
17

---

mask out the rendering normal and ground-truth normal by (M) and ((Nr), we then calculate the average angle between the normals vectors for all valid pixels, Le., Normal MAE= Evem arcross M, M,M, (Ner) )>
Eie Mi
B
Assets License
In Table I1
we list the licenses of all the existing assets including the code and data we have used in this work.
Asset
License Link
3D Goussian Splatting
2
bitps///gtthub coobcooooomo com//roaponeeeeeeeeeeooeoeeoeeee caussian blatting/boo//miiil..... md
Seaffold-GS
https /i thol wtwpp//sstattttoooooooooooo
ShassianShader i
htpp: // wulbuubbbbbbboooom comn
1N SSGuussnnnnnee wtwpssssssnnnteeeba/aaaaaamaamm muw ENSE.
SpecCaussian
http:://guuuu...cmnnnngggg Smecuu air Wunssaan/hhtyymmii..........
COLMAP
intpe//cllmpp githaab..lleccns..iii.
Synthesie-NoRF
https://drive https://drivgggooll.ooo/// www.gooooooooooooooooorr..oo 28yroolddeee///yyy/y/y 28yBr IG 3NIRpp2APS774ldlddo (CC-BY 3.0)
MipNeRF360 26
hilpps Jonbarroooooooooooooooo
5607
DeepBlending
htip:/visull inn hag.a//mw...uuuuuuuuuuuuuueeaann hhtse 4.0)
Tank and Tamples
Table 11: Licenses of existing assets we have used in this work.
18

---

NeurIPS Paper Checklist
1. Claims
Question: the main claims made the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract and introduction demonstrate the paper's contribution to the scope of 3D Gaussian Splatting, considering the issue of appearance-geometry conflict and proposing solution for it.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can he expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
This paper discusses the limitations the Discussion Section
Guidelines:
The answer NA means that paper has no limitation while answer means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification,
asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-ter system might not be used reliably to provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicabll,, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might that reviewers discover limitations that aren't I acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: In the theory derivation in Sec. 3.2 we point out the assumption of Lambertian surface for proposed components.
Guidelines:
The answer NA means that the paper does not include theoretical results.
19

---

All the theorems, formulas, and proofs in the paper should be numbered and cross tross-reference..
All assumptions should he clearly stated or referenced in the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The information reproduce the main experimental results is provided in Sec. S aand appendix, including network configuration, and hyper-parameters training.
Guidelines:
The answer means that the paper does not include experiments.
the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible important, regardless whether the code and data are provided or not.
the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on contribution, reproducibility can be accomplished various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others replicate the model with the same dataset, or provide access the model. general. releasing code and data often one good way to accomplish this, reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of a large language model), releasing of model checkpoint, or other means that are appropriate to the rescarch performed.
While NeurIPS does not require releasing code, the conference does require all submissions provide some reasonable avenue for reproducibility. which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make clear how reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g.. a large language model), then there should either be a way to access this model for reproducing the results a way to reproduce the model (e.g.. with open-source dataset instructions for how construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source source models, it may he that access to the model is limited in some way (e.g.. registered users), but it should be possible for other rescarchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does paper provide open access to the data code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [No]
Justification: The details for implementation are provided in the appendix and we will release our code upon publication.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines httt:::://iips.c//pubb for more details.
guides/Codee Cowe/ooodeeeeeee aondeSubaissiooPolicioo.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
20

---

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines htttps://niss.cc/ppin
Buzdee//DodeSubmiissiooooPo
for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for new proposed method and baselines. If only a subset experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
Experimental Setting/Detais
Question: Does the paper specify all the training and test details (e.g.. data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: The training and testing details are elaborated in the appendix and Sec.
Guidelines:
The answer NA that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined other appropriate informa- tion about the statistical significance of the experiments?
Answer: [No]
Justification: We report the per-scene results the appendix, which can be used to calculate statistical significance.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanicd by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to library function, bootstrap. etc.)
The assumptions made should given (e-g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error than state they have 96% CI, if the hypothesis of Normality errors not verified.
asymmetric distributions, authors should be careful not show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). error bars are reported tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification:
We provide the related information in the experiments and appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
21

---

The paper should indicate the type of compute workers CPU or GPU, internal cluster. or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics htttps://neuttee
Answer: [Yes]
Justification:
The research is carefully conducted with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
the authors answer No, they should explain special circumstances that require deviation from the Code of Ethics.
The authors should make sure preserve anonymity (e.g.. if there is special consideration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
The discussion about potential negative social impacts is mentioned in Sec.
Guidelines:
The answer NA means that there is no societal impact of the work performed. If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative socictal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (c.g., deploy- of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is a direct path to any negative applications. the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it not needed point out that generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release models, providing defenses in addition to attacks, mechanisms for monitor- ing misuse, mechanisms to monitor how system learns from foodback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible relcase of data or models that have high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: This paper aims to support novel view synthesis and geometric reconstruction, which poses such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines restrictions to access the model or implementing safety filters.
22

---

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Answer:
[Yes]
Justification:
We include the assets used in our paper (code and dataset) in Sec.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include a URL.
The name of the license (c.g., CC-BY 4.0) should be included for cach asset. from particular source (e.g.. website), the copyright and terms of service of that source should provided.
For scraped
assets are released, the license, copyright information, and terms of use the package should be provided. For popular datasets, paperswi thcode. www.ccom//aaacc /attttttsttt has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.
For existing datasets that are re-pac re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not release new assets during the time of submission.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the datasct/code/model as part of their sub- missions via structured templates. This includes details about training, license, limitations,
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an amonymized URL or include anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelincs:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should included the main paper.
According to the NeurIPS Code of Fthics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals Equivalent for Research with Human Subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
23

---

Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer means that paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, approval (or equivalent) may be required for any human subjects research. If you ohtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applica- ble), such the institution conducting the review.
24


Reviews:
[{"review_id": "Gfzxxojp3p", "reviewer": "Reviewer_5i8r", "summary": "This paper addresses the challenge of achieving high rendering quality and accurate geometry in computer vision and graphics. While recent advancements in 3D Gaussian Splatting (3DGS) have enabled real-time high-fidelity novel view synthesis, the discrete and noisy nature of 3D Gaussian primitives hinders accurate surface estimation. The authors propose Normal-GS to integrate normal vectors into the 3DGS rendering pipeline. Surface colors are re-parameterized as the product of normals and a specially designed Integrated Directional Illumination Vector (IDIV).", "questions": "I have expressed my concerns regarding the paper above, and I strongly recommend that the authors address them by enhancing the motivation, technical contributions, and exposition. These improvements are necessary to enhance the quality of the paper. As it stands, I am inclined to reject the current form due to the unclear motivation, absence of baseline methods, and limited applicability. I am also open to revising my rating based on the response and other reviewers' feedback.", "limitations": "NA", "rating": 3, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The writing of this paper is good. It is easy to follow and understand the technical details.\n- The usage of PBR for optimizing normals of 3DGS is interesting, though it is not new.\n- Although the experimental results show only slightly better quantitative measurements compared to SpecGaussian, the quality of the normals is significantly improved.", "weaknesses": "- L102-103, the authors claim that PBR-based 3DGS methods show lower rendering quality than original 3DGS, which is not correct, please refer to the tables in GaussianShader [7].\n- The most closely related work to this paper is 2DGS [44], but it is surprisingly not mentioned or included in the experiment section, which is quite perplexing. 2DGS simplifies 3DGS by aligning it with the surface normal direction, allowing the scale parameters to be directly optimized based on the appearance loss. This omission raises confusion since 2DGS has direct relevance and could potentially provide insights or comparisons for the proposed method.\n- The integration of Physically Based Rendering (PBR) techniques, specifically Bidirectional Reflectance Distribution Function (BRDF), with 3D Gaussian Splatting (3DGS) is not a novel concept. As mentioned by the authors, previous works have already explored this combination and achieved satisfactory results. Moreover, existing methods mostly enable \"relighting\" while the proposed method is not. In the proposed method, the authors introduce a simplification of the integral over the up half-space of a point, referred to as IDIV, which represents the relationship between illumination and incident light direction. They decode this IDIV using a Multi-Layer Perceptron (MLP) under the assumption of Lambertian reflectance. However, the authors also draw inspiration from Ref-NeRF and incorporate a specular component to account for specular effects. This additional complexity and the combination of different techniques have resulted in confusion and difficulty in understanding the underlying purpose, raising doubts about the technical novelty of the method, as it appears to be a combination of existing approaches (referred to as an \"A+B\" manner).\n- In order to assess the quality of the geometry (indirectly optimized by normals, which is the motivation stated by the authors), there are established metrics and datasets available, such as DTU, that can be utilized. It is perplexing that the authors solely evaluate the Mean Angular Error (MAE) of normals, as the quality of normals is not always the primary factor when evaluating geometry. Instead, the overall geometry quality holds greater importance and should be the focus of evaluation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your review and valuable feedback. Below we address the concerns raised in the review.\n\n**Q1: L102-103, the authors claim that PBR-Based 3DGS methods show lower rendering quality than original 3DGS, which is not correct, please refer to the tables in Gaussian Shader [7].**\n\nWe appreciate the reference to Gaussian Shader [7], which indeed demonstrates improvements in rendering quality on synthetic datasets.  We agree that [7] demonstrates better performance under controlled, synthetic conditions. However, our primary concern is the Gaussian Shader’s reliance on a global environment map, which does not align with real-world scenarios with analysis on L165-174 of our paper. In our experiments, as illustrated in Figure 4 and Table 1, [7] faced difficulties in modeling complicated real-world scenes. These experiments are re-run directly using their released codes.\n\n**Q2: 2DGS [44] is not mentioned or included in the experiment section.**\n\nWe would like to clarify that the omission of 2DGS [44] from our experiments was not intentional. We discussed and cited 2DGS on L94-95 of our paper. However, it is important to note that 2DGS was recently released (with codes available on May 3rd). This timing is the primary reason for its exclusion from our experiments. Additionally, as noted on L94-95, 2DGS sacrifices some rendering fidelity in novel view synthesis due to the lack of a clear relationship between geometry and appearance, as shown in Table 3 and Table 4 of [44]. We also want to emphasize that 2DGS and our method are conceptually orthogonal and could potentially be combined. Unlike 2DGS, our method explicitly addresses the interactions between lighting and normals. We have also added comparisons with 2DGS, focusing on geometric reconstruction quality and rendering quality. Please refer to our response to Q3 and Table A for these comparisons.\n\n| Method       | Mean Chamfer Distance | PSNR  |\n|--------------|------------------------|-------|\n| 3DGS [2]     | 1.96| 35.76 |\n| SuGaR [8]    | 1.33| 34.57 |\n| 2DGS [44]    | 0.80 | 34.52 |\n| Ours         | 0.94| 37.63 |\n\nTable A. Geometric and rendering quality comparisons on the DTU dataset.\n\n**Q3: The quality of normals is not always the primary factor when evaluating geometry. Instead, the overall geometry quality holds greater importance and should be the focus of evaluation.**\n\nWe agree that overall geometry quality is important, but we believe that accurate normal is also very useful for many applications and has nice properties such as strong generalization capability [A, B]. We appreciate the reviewers' recognition of the accuracy of our normal predictions. To address the importance of overall geometry quality, we followed your suggestion and further evaluated our method's performance on the DTU dataset. We focused on both the reconstruction and rendering quality. We assessed the mean Chamfer distance and PSNR, adopting results for 3DGS [2], SuGaR [8], and 2DGS [44] directly from Table 3 of 2DGS [44].\nFor a fair comparison, we followed the process from 2DGS. As shown in Table A, our method achieves outstanding rendering quality while with a better mean Chamfer distance than 3DGS.\n\n[A] Bae, Gwangbin, and Andrew J. Davison. \"Rethinking inductive biases for surface normal estimation.\" CVPR. 2024.\n\n[B] Guangyao Zhai et.al Monograspnet: 6-dof grasping with a single rgb image. ICRA. 2023.\n\n**Q4: The integration of PBR … with 3DGS is not a novel concept. ...existing methods mostly enable “relighting” while the proposed method is not.**\n\nWe acknowledge the significant contributions of previous PBR-based methods [7, 16, 17, 45], especially as they enable “relighting” editability. However, as we discussed on L44-51 and L98-101, prior attempts often require approximations and meticulous regularization terms, which compromise either rendering quality or geometric accuracy. This motivates us to design a straightforward method to integrate normal information explicitly and simply into 3DGS. We also directly compared with the PBR-based method, Gaussian Shader [7], in our experiment section and our method achieves better rendering results and normals.\n\n**Q5: The motivation and novelty of our work.**\n\nThank you for your feedback. We respectfully disagree with the characterization of our method as merely a combination of existing approaches. Our primary motivation is to **address the balance issue between rendering and geometry quality**, rather than focusing on relighting or intrinsic decomposition.\nFrom a physically-based rendering perspective, we identify that the core issue arises from the lack of interaction between appearance and normal estimation. Solving this problem is challenging, and our approach aims to simplify the model while avoiding the complexities of highly unconstrained inverse rendering problems, which often require numerous additional regularizers.\nOur method introduces only a few additional IDIV and reflection parameters, yet achieves remarkable improvements in normal estimation and maintains rendering quality.\nWe believe our approach provides valuable insights into the issue and establishes a new baseline that could benefit the community.\n\n**Q6: ... under the assumption of Lambertian reflectance. ... incorporate specular components to account for specular effects.**\n\nWe would like to clarify that our approach does not assume objects are Lambertian. As stated on L151-153, our initial consideration is for a simple ideal case with Lambertian objects. For more complex materials, including those with specular effects, we address this in Section 3.4 of our paper.\nAs illustrated in Figure 2 of our paper, our method is designed to handle both diffuse and specular components. This allows us to account for a range of material properties beyond the Lambertian assumption.\n\nWe thank you again for your valuable feedback and we will include suggested experiments in the final version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZXwAjIbKFx", "reviewer": "Reviewer_cnbL", "summary": "Normal-GS successfully combines color estimation with surface normal optimization, achieving remarkable surface normal prediction without sacrificing view synthesis quality compared to previous methods. In calculating diffuse color, the traditional method that extracts the surface normal components from the incident light integration is used, thus diffuse color is expressed as a simple product of the diffuse albedo, normal, and integration of the incident light (referred to as IDIV in the paper). Based on Scaffold-GS, which predicts multiple nearby 3DGS parameters by passing a single anchor neural Gaussian to a global MLP, IDIV is calculated for each anchor and used for diffuse color prediction. Additionally, by integrating the IDE proposed in RefNeRF into Scaffold-GS, the specular component is successfully represented with respect to the normal. Finally, geometry is regularized by comparing the rendered surface normal with the normal obtained from the depth map.", "questions": "How is \"3DGS w/ IDIV\" in the ablation studies (section 4.2) (b) implemented? Did you create an additional anchor neural Gaussian to the original 3DGS and use an global MLP to predict IDIV, or did you place it as an optimization component like opacity in each 3DGS and use it for rendering? Additionally, in (c), \"ours\" generally refers to the complete Normal-GS, so the meaning of \"w/ IDIV\" is unclear. If (c) refers to Normal-GS without the specular component, it would be better to label it as \"Ours w/o L_s.\"\n\nIn Figure 2, it seems that there is a typo where the representation of the \"vector\" in the \"IDIV vector\" is duplicated.", "limitations": "As mentioned in the paper, learning the depth of distant objects, like the sky, consistently is extremely challenging. Therefore, it seems that learning surface normal based on depth will also be very difficult.\n\nAdditionally, as described in the weaknesses section, clearer network outputs would be beneficial.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The idea of approximating diffuse color equation in terms of surface normal by extracting the normal from the integral equation is both simple and powerful.\n\nAdditionally, the geometry inaccuracy that can occur due to view-dependent effects is effectively handled by using IDE to manage specular artifacts.\n\nThe proposed method compares well with other recent methods that demonstrate robust surface normal in view synthesis.\n\nDetailed explanations of IDE and surface normal estimation are well-documented in the supplementary materials, making it easy to follow.", "weaknesses": "Due to the lack of explanation of Scaffold-GS in the related work section, the description of the network output is confusing. While it is unnecessary to provide additional details about Scaffold-GS, it is important to clearly explain the network output.\n\nAs I understand it:\nIn Scaffold-GS, an anchored feature f_v is fed into a global MLP to initially predict the parameters of ‘k’ adjacent 3DGS. In Normal-GS, according to line 194, global MLP theta_l is used to additionally predict IDIV. For diffuse color, it is directly calculated using the predicted IDIV and diffuse albedo. For specular color, it is calculated by passing IDE, normal, and feature into the color MLP theta. However, there is no mention of how components like opacity and diffuse albedo are calculated. It is unclear whether these components are embedded and stored like the original 3DGS, or if they are calculated through a separate MLP. Therefore, it would be helpful to specify which MLP is responsible for each component, similar to IDIV, to clarify the process.\n\nThere is no quantitative ablation results. Additionally, there is no ablation of the depth-normal loss.\n\nPersonally, I think this paper, due to its contributions through the refactorization of the rendering equation, would be more suitable for a computer vision or graphics conferences.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for appreciating our work and making constructive comments.\n\n**Q1: Clearly explain the network output, and how components like opacity and diffuse albedo are calculated.**\n\nWe would like to clarify it here and in the final version. The whole procedure about how to calculate IDIVs and the specular color is the same as your comments in the second paragraph of the weakness section and as our explanation on L194-197. As for the opacity, because the updating strategy of the Scaffold-GS depends on the value of opacity, we followed the Scaffold-GS and predicted it using a global MLP. As for the diffuse albedo, we reused the color MLP of the Scaffold-GS to predict it.\n\n**Q2: How is “3DGS w/ IDIV” in the ablation studies (section 4.2) (b) implemented?**\n\nFor “3DGS w/ IDIV” in Fig. 5 (b), we attached the IDIV as an additional attribute to each 3D Gaussian without using global MLPs. As described on L299, in this setting we applied the Laplacian and Total-Variation loss to IDIVs. This was done by first rendering IDIVs following the 3DGS rendering pipeline and then applying image-space regularizers. Please note that this setting was “sensitive to the tuning of loss weights” as discussed on L302 and motivated us to use the locally shared structure, such as Scaffold-GS, to regularize IDIVs.\n\n**Q3: Quantitative Ablations.**\n\nThanks for the constructive suggestions. To better verify the effectiveness of the components, we add the quantitative results on DTU by comparing the mean chamfer distance and PSNR for geometry and rendering quality evaluation. Please refer to Table B. Our base model is the Scaffold-GS. 1) We add depth-normal into the Scaffold-GS, the results of (b) in Table B shows that the depth-normal loss can significantly improve the geometry quality but ruin the rendering with more than 2dB PSNR drop. 2) Then we introduce the specular component into (b), which can improve the rendering quality without compromising the geometry, which further assures the importance of involving normal into the rendering. 3) We add the proposed IDIV into the model to get our final model (d). Our full model further improves the PSNR by more than 0.2 dB with similar geometry quality. Compared with the base model (a) and the results of 3DGS (CD: 1.96, PSNR: 35.76), our full model achieves a better balance between the geometry quality and the rendering fidelity.\n\n\n| Model | Details                   | Mean Chamfer Distance | PSNR  |\n|-------|----------------------------|------------------------|-------|\n| (a)   | Scaffold-GS                | 1.84                   | 38.14 |\n| (b)   | (a) + Depth-normal loss    | 0.95                   | 35.90 |\n| (c)   | (b) + $L_\\text{Specular}$             | 0.93                   | 37.37 |\n| (d)   | (c) + $L_\\text{Diffuse}$ (Full model)    | 0.94                   | 37.63 |\n\nTable B. Quantitative ablation studies on the DTU dataset.\n\n**Q4: Some typos.**\n\nThank you for pointing out these typos. We will fix them in the final version.\n\nWe will also refine our writing as suggested and include the mentioned experiments in the revised version. We appreciate the reviewer's valuable help and comments.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hxqxKlT9sg", "reviewer": "Reviewer_68er", "summary": "This paper proposes a novel appearance modeling of 3D Gaussian Splatting (3DGS) for both accurate appearance representation and geometry reconstruction. Existing 3DGS methods suffer from the trade-off of the accuracy of appearance and geometry due to the disconnection between surface geometry and rendering. To address this problem, this paper reformulates the rendering equation for fast physically-based 3D Gaussian (Normal-GS) rendering, directly connecting surface normal and rendering color. For stable optimization, Normal-GS uses anchor-based MLPs that implicitly consider the local smoothness of local illumination. The experimental results show that Normal-GS reconstructs accurate surface normal while retaining the rendering quality of 3DGS.", "questions": "- How plausible are the optimized diffuse and specular reflection components? What if only either of them is rendered?\n- Can the IDIV be visualized to validate that it captures local incident light? For example, in outdoor scenes, it is oriented to the direction of the sun and other directions outside and within the shadow region, respectively.\n- Scaffold-GS uses the enhanced view-dependent features $\\hat{f}_v$ instead of directly using a local feature $f_v$. Which features does this method use? Regarding both geometry and appearance modeling, view-independent Gaussian attributes would be desirable.", "limitations": "Yes.", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "+ Propose a novel physically-based rendering method for 3D Gaussian Splatting that can use shading cues for normal estimation, backpropagating photometric loss to the surface normal of 3D Gaussian.\n+ Exploit anchor-based 3DGS to represent and stably optimize local incident light without time-consuming ray tracing.\n+ Experimentally show that the proposed method can achieve both high-quality rendering and accurate normal estimation even in complex lighting and specular scenes where existing methods degrade.", "weaknesses": "- The illumination of diffuse and specular components is independent of each other. This is physically implausible.\n- The anchor-based regularization would prevent the reconstruction of detailed geometry like bicycle spokes in Fig. 6. Furthermore, since IDIV depends on the surface normal through $\\Omega^+$, it also requires the smooth surface normal.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for appreciating our work and providing constructive comments.\n\n**Q1: The illumination of diffuse and specular components is independent of each other. This is physically implausible.**\n\nWe agree that diffuse and specular components cannot be independent physically. However, our main purpose is to strive for a better balance between rendering and geometry quality. Our insight is to keep the model simple without solving the highly-unconstrained inverse rendering problem, which needs too many extra regularizers. We do believe our framework could be further improved by introducing additional parameters like metalliciness [16, 17, 18, 45] for downstream applications.\n\n**Q2: Visualization of Diffuse, Specular and IDIVs.**\n\nThanks for the helpful suggestions. We are glad to provide more results to show the plausibility of our method. We show the visualization of our specular and diffuse components in Figure A of the one-page PDF. We choose scenes containing specular materials to better demonstrate the effectiveness of our method. From the visualization, we can observe that our method successfully disentangles the diffuse and specular components. We also visualize IDIVs in Figure B of the one-page PDF.  We choose an outdoor scene for better visualization. We can observe that IDIVs approximately align with the sunlight in bright regions (on the table) and roughly diverge in the shadow region (under the table), which indicates that our IDIVs capture local incident lighting information. \n\n**Q3: The anchor-based regularization would prevent the reconstruction of detailed geometry like bicycle spokes in Fig. 6.**\n\nWe acknowledge the detailed geometry like bicycle spokes is not fully modeled by our method. The thin structure is a challenging problem in discrete Gaussian representation which also related to the aliasing issue. We leave it to the future work.\n\n**Q4: Furthermore, since IDIV depends on the surface normal through $\\Omega^{+}$, it also requires the smooth surface normal.**\n\nWe agree with your comments. A smooth and correct normal is important for many applications, including estimating lighting and supporting capturing specular effects. We applied the depth-normal loss to regularize estimated normals.\n\n**Q5: Scaffold-GS features.**\n\nWe agree that view-independent Gaussian attributes are desirable. However, during our experiments, we found that view-dependent features would produce slightly better rendering quality (~0.1-0.2 dB). We believe that the viewing direction could help determine the inside-outside direction of the surface, thus benefiting the rendering quality.\n\nWe are glad to include the suggested results in the revised version. Thank you again for your valuable feedback.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xrFbMuaUdD", "reviewer": "Reviewer_Gwpd", "summary": "The authors propose a novel appearance modeling technique for an chor-based 3D Gaussian Splatting representation based on normal information and an incident lighting parametrization with MLPs. In addition to representation several regularization techniques are used to stabilize normal and incident light optimization.  The central claim of the paper is to achieve competitive the rendering quality while achieving more detail geometric information. The authors show experiments on the MIP-NeRF360, Tanks&Temples, Synthetic-NeRF and the deep blending dataset and provide a quantitative comparison in rendering quality and normal-accuracy.", "questions": "I'd would be great if the authors could provide explanations to weaknesses.", "limitations": "The authors addressed valid limitations in the dicussion part.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1) The paper is easy to read and all components are well explained.\n2) The conducted experiments use the most relevant datasets and metrics to assess the image quality, LPIPS, SSIM, PSNR.\n3) The underlying research problem of reconstructing accurate appearance and geometry is relevant for the field of neural rendering and an very active research area.", "weaknesses": "1) The experimental evaluation only contains 3DGS based methods. Even though there are inherent advantages of 3DGS methods, I'd still expect a comparison to NeRF-based methods on the same research problem, e.g. Ref-NeRF and follow-ups. The central question here would be, how good is the proposed method compared to the best Neural Field based method?\n\n2) The general definition of normals in the context of 3D gaussians sounds a bit vague. The authors say that they define the normal of the a gaussian primitive as the shortest axis of the 3D gaussian without actually defining a surface. How do you define the normals when gaussians overlap or are semi-transparent?\n\n3) The authors claim to improve geometric quality and show the comparison in normal accuracy in table 2. From a 3D reconstruction perspective, the normal accuracy indeed can indicate high quality, however metrics on the reconstructed geometry, like chamfer distance, F1-score, might be a better indicator for geometric quality.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for helpful comments and suggestions. We are glad to address the issues raised in the review.\n\n**Q1: Comparison to NeRF-based methods**\n\nWe would like to include the comparisons to NeRF-based methods here and in the final version. It is important to highlight that 3DGS-based methods, including ours, are relatively fast and could support real-time rendering compared with NeRF-based methods. As illustrated in Table C, on the Mip-NeRF 360 dataset, our method performs the second best in terms of the PSNR and achieves comparable SSIM and LPIPS scores. The involvement of the proposed IDIV could provide better novel view synthesis quality compared with Ref-NeRF, which further assures the novelty of our design. Please also note that the training time for Zip-NeRF is > 10 hours on high-performance GPUs, which is much longer than 3DGS-based methods, including ours.\n| Method            | PSNR   | SSIM  | LPIPS |\n|-------------------|--------|-------|-------|\n| 3DGS [2]          | 28.691 | 0.870 | 0.182 |\n| Mip-NeRF 360 [26] | 29.231 | 0.844 | 0.207 |\n| Ref-NeRF [22]     | 28.553 | 0.849 | 0.196 |\n| Zip-NeRF [30]     | 30.077 | 0.876 | 0.170 |\n| Ours              | 29.341 | 0.869 | 0.194 |\n\nTable C. Rendering quality comparisons on the Mip-NeRF 360 dataset.\n\n**Q2: Definition of normals in the context of 3D Gaussians.**\n\nWe acknowledge that we did not define normals for true surfaces but rather established a normal direction for each Gaussian. As discussed on L202-203, we define the normal for a 3D Gaussian based on its geometric properties. Specifically, during optimization, 3D Gaussians often exhibit flatness around the surface areas, as observed in [5, 7, 45]. Therefore, for a near-flat Gaussian ellipsoid, its shortest axis thus functions as the normal vector. Consequently, we use the shortest axis of each Gaussian as its normal attribute. For areas overlapped by multiple Gaussians, the normal is the weighted average of overlapped Gaussians. As shown in Fig.1, we found with our IDIV design, the model can also render a reasonable surface normal for the semi-transparent cover in this definition. Moreover, defining physically correct normals for semi-transparent areas remains an interesting and open problem and we leave it for future work.\n\n**Q3: Metrics on the reconstructed geometry other than the normal accuracy.**\n\nWe agree that the overall geometry quality is also important, in addition to the normal accuracy. We follow your suggestions and add experiments to evaluate the reconstruction quality of our method on the DTU dataset in terms of the mean Chamfer distance. We also report the rendering quality indicated by PSNR to demonstrate the advantages of our method. The results of 3DGS [2], SuGaR [8], and 2DGS [44] are adopted from Table 3 of 2DGS [44]. For a fair comparison, we conduct experiments under the same setting as 2DGS [44]. As Table 9 illustrates, our method achieves outstanding rendering quality while maintaining better geometric quality than 3DGS.\n\n| Method       | Mean Chamfer Distance | PSNR  |\n|--------------|------------------------|-------|\n| 3DGS [2]     | 1.96                   | 35.76 |\n| SuGaR [8]    | 1.33                   | 34.57 |\n| 2DGS [44]    | 0.80                   | 34.52 |\n| Ours         | 0.94                   | 37.63 |\n\nTable A. Geometric and rendering quality comparisons on the DTU dataset.\n\nWe appreciate your recommendations regarding the comparisons to NeRF-based methods, clarification of defining normals, and measurements of the overall geometry quality. We will add them in the revised version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We sincerely thank all the reviewers for their constructive and insightful comments, especially for recognizing our work by

- **Clear presentation.** "easy to follow and understand the technical details" (5i8r), "well-documented" (cnbL), "excellent presentation" (68er), "easy to read and all components are well explained" (Gwpd).
- **Soundness of design.** "Approximating diffuse color in terms of the surface normal by extracting the normal from the integral equation is both simple and powerful", "handle view-dependent effects by using IDE" (cnbL), "a novel method that can use shading cues for normal estimation, backpropagating photometric loss to the surface normal of 3D Gaussian", "stably optimize local incident light without time-consuming ray tracing." (68er). "Good soundness" (5i8r, Gwpd).
- **Superior experimental results.** "the quality of the normals is significantly improved" (5i8r), "demonstrate robust surface normal in view synthesis", "achieving remarkable surface normal prediction" (cnbL), "the proposed method can achieve both high-quality rendering and accurate normal even in complex lighting where existing methods degrade." (68er).

We will polish and revise the paper according to the reviewer's suggestions. Below please find our detailed responses. We sincerely welcome additional feedback and further discussions.

Best regards,

Authors of Submission 8220

Meta-review: The paper introduces a new approach to integrating surface normals into the 3D Gaussian Splatting pipeline, aimed at improving both rendering quality and geometric accuracy while maintaining real-time performance. Reviewers generally found the paper well-written and the proposed method innovative, particularly in how it balances rendering and geometry quality without the need for complex regularization.

However, some reviewers initially criticized the paper for being incremental, questioning its novelty and the clarity of certain aspects. There were also concerns about the lack of comparisons with NeRF-based methods and the absence of detailed ablation studies. The authors addressed these points by providing additional experiments, comparisons, and clarifications in their rebuttal, which helped alleviate most concerns.

Despite one reviewer maintaining a stance against acceptance, the other reviewers appreciated the improvements made during the review process, leading to an increased rating from them. Given the solid technical contribution, clarity, and relevance to the field, the paper is recommended for poster acceptance.
Decision: Accept (poster)