Effective Rank Analysis and Regularization for Enhanced 3D Gaussian Splatting
Authors: Junha Hyung
Susung Hong
Sungwon Hwang
Jaeseong Lee
Jaegul Choo
Jin-Hwa Kim
Venue: NeurIPS 2024 poster (2024)
Keywords: 3D reconstruction
3D Gaussian Splatting
NeRF
Surface reconstruction
3DGS regularization
Primary area: machine_vision
TL;DR: Effective Rank Analysis and Regularization for Enhanced 3D Gaussian Splatting

Abstract: 3D reconstruction from multi-view images is one of the fundamental challenges in computer vision and graphics. Recently, 3D Gaussian Splatting (3DGS) has emerged as a promising technique capable of real-time rendering with high-quality 3D reconstruction. This method utilizes 3D Gaussian representation and tile-based splatting techniques, bypassing the expensive neural field querying. Despite its potential, 3DGS encounters challenges such as needle-like artifacts, suboptimal geometries, and inaccurate normals caused by the Gaussians converging into anisotropic shapes with one dominant variance. We propose using the effective rank analysis to examine the shape statistics of 3D Gaussian primitives, and identify the Gaussians indeed converge into needle-like shapes with the effective rank 1. To address this, we introduce the effective rank as a regularization, which constrains the structure of the Gaussians. Our new regularization method enhances normal and geometry reconstruction while reducing needle-like artifacts. The approach can be integrated as an add-on module to other 3DGS variants, improving their quality without compromising visual fidelity. The project page is available at https://junhahyung.github.io/erankgs.github.io/.

Effective Rank Analysis and Regularization for Enhanced 3D Gaussian Splatting
Junha Hyung'
Susung Hong'
Sungwon Hwang' Jin-Hwa Kim 2.31
Jaeseong Lee'
Jaegul Choo
KAIST 2NAVER Lab
SNU AIIS
4Korea University
Abstract
3D reconstruction from multi-view images is one of the fundamental challenges in computer vision and graphics. Recently, 3D Gaussian Splatting (3DGS) has emerged as a promising technique capable of real-time rendering with high-quality 3D reconstruction. This method utilizes 3D Gaussian representation and tile-based splatting techniques, bypassing the expensive neural field querying. Despite its potential, 3DGS encounters challenges such as needle-like artifacts, suboptimal geometries, and inaccurate normals caused by the Gaussians converging into anisotropic shapes with one dominant variance. We propose using the effective rank analysis examine the shape statisties of 3D Gaussian primitives, and identify the Gaussians indeed converge into needle-like shapes with the effective rank 1. To address this, we introduce the effective rank as a regularization, which constrains the structure of the Gaussians. Our new regularization method enhances normal and geometry reconstruction while reducing needle-like artifacts. The approach be integrated as add-on module to other 3DGS variants, improving their quality without compromising visual fidelity. The project page is available at htttp:://junaahyung gi thub. htt.//rankgg.ii... gitgub.io////
Introduction
Creating 3D models from multiple images is central challenge in computer vision and graphics. Neural Radiance Fields (NeRF) (20) have revolutionized this area by demonstrating remarkable capa- bilities in novel view synthesis through implicit neural fields and differentiable rendering techniques. Despite their impressive 3D reconstruction quality, the training and rendering processes of NeRF- based methods are computationally intensive, posing significant challenges for real-time applications. To improve training and rendering efficiency, various acceleration techniques, such as baking with have been introduced. While these solutions enhance efficiency to some extent, there are still limitations for real- time interactive scenarios.
shell 12 22
and grid representations
Recently, 3D Gaussian Splatting (3DGS) has emerged time rendering with high-quality results. This method utilizes 3D Gaussian representations and tile-based splatting techniques instead of expensive neural field querying, making it feasible apply the technique in practical applications. This opens up new possibilities in areas that require faster rendering, such virtual and augmented reality, gaming, and real-time avatars.
promising technique capable of real-
However, despite its potential, 3DGS encounters several challenges in terms of geometry reconstruc- tion, including noisy rendering results with needle-like artifacts, especially in novel and extreme views far from the training images. These issues stem from the primitive-base based nature 3DGS, where individual primitives lack geometric constraints.
Corresponding authors
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

(a)
Figure 1: (a) Qualitative results on novel view synthesis and normal reconstruction on the DTU [4 dataset. (b) and (c) show novel view synthesis comparisons on the Mip-NeRF360 [2] and DTU datasets, respectively. The top row shows novel view renderings of 3DGS, and the bottom row shows renderings of 3DGS with effective rank regularization. While naive 3DGS presents needle-like artifacts, our regularization term mitigates these artifacts in novel views.
For accurate geometry reconstruction, is well known that the density field should be concentrated near the surface [30]. To this end, previous efforts, such as SuGaR (TO), have focused on regularizing the 3D Gaussians to be flatter, i.e., regularizing the primitives into anisotropic Gaussians with one of its variance very small. Similarly, 2DGS [3] utilizes 2D Gaussians instead of 3D Gaussians to force this effect.
While the flatness of Gaussians is necessary for proper alignment with the surface, argue that flatness alone is insufficient for accurately representing surface geometry. Gaussians should also avoid being needle-like highly anisotropic, where variance dominates the others. Needle-like Gaussians hinder accurate reconstruction as they cover only negligible portion of the surface and produce spiky artifacts. Instead, disk-like Gaussians are preferable they cover meaningful areas and contribute effectively to surface reconstruction.
However, existing methods fail to adequately differentiate between disk-like and needle-like Gaus- sians, as both can exhibit one scale that is near or exactly zero. Empirically, we find that most Gaussians converge into anisotropic forms, becoming needle-like with small scales along axes due to the absence of appropriate regularization mechanisms.
To directly examine the shape statistics (whether their geometries are disk-like or needle-like) of 3D Gaussian primitives and understand their structural changes during training, we first propose performing the effective rank analysis the covariance matrices of Gaussians. The effective rank (25), which real-valued and differentiable extension of the integer rank, can be used to monitor the training dynamics and structural transformations of Gaussian primitives. Indeed, our analysis reveals that the effective ranks Gaussians approach an effective rank (erank-1), 1), resulting in needle-like like shapes in 3DGS and other methods, such as SuGaR [10 and 2DGS [3]
Additionally, we propose using the effective rank as a regularization term constrain the structure of the Gaussians. The differentiable nature of effective rank, with its concave logarithmic term providing stable gradients, makes directly applicable continuous optimization problems. Our new regularization method enhances normal and geometry reconstruction while reducing needle-like

---

Baselines
Baselines erank reg.
25000
25000
25000
3DGS
15000
15000
1.5
25000
25000
SuGaR
15000
15000
15000
5000
5000
1.0
1.5
2.0
3.0
1.0
1.5
2.0
2.5
3.0
25000
25000
25000
2DGS
15000
15000
15000
5000
5000
5000
1.0
1.5
2.0
2.5
3.0
1.0
1.5
2.0
2.5
3.0
1.0
1.5
2.0
2.5
3.0
Iteration
Iteration 14000
Iteration 30000
(Green): The effective rank histograms for baseline methods 3DGS [T6], SuGaR [00. and 2DGS [13 showing that Gaussian ranks are not optimally constrained for geometry reconstruction. (Purple): The regularization term properly constrains the Gaussians, flattening them while preventing convergence into needle-like shapes.
Figure 2:
artifacts, particularly in novel view scenarios. Furthermore, our effective rank regularization can be applied as an add-on module to other 3DGS variants, improving their quality.
The main contributions of our work are as follows:
We are firstly analyzing the dynamics of Gaussian primitive structures using the effective rank in the optimizing process, discovering that Gaussians converge into anisotropic forms with one dominant variance.
We propose effective rank regularization method that alleviates needle-like artifacts in 3DGS rendering and improves geometric reconstruction.
Our approach is an add-on module that can integrated with other 3DGS variants, and demonstrate that our method enhances 3D geometry reconstruction without compromising visual quality.
2
Related work
Novel view synthesis NeRFs 20 have revolutionized photo-realistic rendering from novel view- points by introducing neural implicit representation 3D scenes. This approach uses high-frequency positional encoding and differentiable volume rendering to achieve unprecedented realism. Enhance- ments to NeRF address challenges like anti-aliasing 3 parameterizing unbounded scenes (7] and training from mn-the-we wild images
[19
29 through probabilistic transience modeling. Further

---

erank 3
erank 2
erank 1.65
erank 1.25
erank 1.08
Figure 3: Real-scale visualization of a 3D sphere and 2D disks and their effective ranks.
improvements reduce training time and enhance rendering quality by incorporating low-rank tensor components [
Other research efforts have aimed for real-time rendering using alternative implicit models that do not rely on MLPs. Notable examples include sparse voxel grids [9] and multi-resolution hash encod- ing [21]. Despite these advancements, ray tracing methods inherently slower than rasterization. To address this, 3DGS [16] introduced point-based rasterization technique for real-time,
view synthesis. Inspired by EWA Volume Splatting [39], 3DGS fully differentiable pipeline, representing 3D scenes with 3D Gaussians and performing volume splatting to known camera poses for rastcrization.
Surface reconstruction Surface reconstruction is a critical area in computer vision and graphics, aiming to recreate shapes and structures from 2D images or other data forms. Among recent innovations, NeuS (30) leverages volume rendering and signed distance functions (SDF) for high- fidelity reconstructions. NcuS2 (31) significantly improves training speed and extends modeling capacity to dynamic scenes. UNISURF [22) integrates implicit surface models and radiance fields for both surface and volume rendering. VoISDF [33] models volume density as a function of geometry, achieving high-quality geometry reconstructions. Neuralangelo [18] uses multi-resolution hash grids and neural surface rendering to recover detailed structures. BakedSDF 34 introduces a hybrid neural oolume-surface representation optimized for mesh extraction.
3D Gaussian Splatting Recent advancements in 3DGS have further propelled surface reconstruc- tion. NeuSG 6 refines surface details using 3DGS and neural implicit models. SuGaR [0] focuses on mesh extraction with SDF-based regularization and Poisson reconstruction. 2DGS [3] collapses 3D volumes into 2D Gaussian disks for sew--conssttent geometry and detailed mesh reconstruction. GaussianShader [15] enhances rendering quality in reflective surfaces using a shading function on
3D Gaussians. GOF 36 utilizes aay-Gaussian intersection for density estimation and geometric
regularization. GIR 28 employs 3D Gaussians for inverse rendering, enabling accurate estimation of material properties, illumination, and geometry. These advancements showease the potential of 3DGS for high-speed, detailed, and versatile surface reconstructions.
3 Preliminaries
3.1 3D Gaussian splatting
3DGS 16 represents a scene with a of learnable 3D Gaussian primitives (Ge k: = 1,... K}.
where each 3D Gaussian GA consists of mean 14g E R3311 covariance Σ₁ E RR333 point opacity og
[0, and view-dependent color C₁ in spherical harmonies. Covariance matrix Σg R. R positive semi-definite, where Sy diag(s)) is scaling matrix, Sş = (SH1: Sk2; $4:)) E 3 is scale parameter, and R₃ E xx3 is a rotation matrix parameterized by a quaternion. A 3D Gaussian primitive can be represented in 3D space as:
(x) e - (xx- " "x- (x-Me))
The primitives then rasterized via differentiable volume splatting. Specifically, 3D Gaussian is projected screen space as Σ. JWE, wTJ² where is world-to-camera transform and J is the Jacobian of the affine approximation of the projection matrix [39]. The covariance and mean

---

of the projected Gaussian ( and simply projecting μr to screen space, respectively. Finally, the Gaussians are alpha-blended in the order of depth as:
(x) are then obtained by removing the third column and row of Σx
k-1
c(u) Σ
CLOR
(1-a,93P(n)) u)),
(2)
j=1
where u is screen space coordinate. The rendered images supervised with photometric loss L for 3D primitive optimization similar to NeRF [20].
As Gaussians are initialized by sparse SfM points, Adaptive Density Control (ADC) is designed for densification during optimization. Specifically, ADC subsamples and splits Gaussians that satisfy the condition:
aL
aL Op:
Σ
T.
Tu
op: du
2
where P and p, denote set of pixel indices and the i-th pixel, respectively, and is a predefined threshold. The intuition behind Eq. Bs that regions not yet well reconstructed exhibit large view- space positional gradients. This occurs because the optimization process attempts move the Gaussians to correct these areas, so densifying such Gaussians can effectively increase expressibility.
3.2 Effective rank
Consider a real-valued non-all aan-all-zero zero M N matrix A. The singular value decomposition (SVD) of A can be expressed as A UDV, where U and V are unitary matrices of sizes M x M and N N respectively, and is a diagonal matrix of size N containing the real positive singular values in descending order:
o 22
0,
where L. min ( NM. The singular value distribution is then defined as
d; for 1. L,
as
(5)
| where (0), (2,,02,"" ". "r,""" and || denotes -noom.
Definition (Effective rank). The effective rank 25 of the matrix A concisely defined as
erank(A) = exp( (g), (q,, "2, ,q.)), (q))) where H(q, 02, ,4.) ) is the Shannon entropy given by
H(q 02. ,41) = Σ qi log qu.
(6)
4 Method
In Section 4.1 we introduce the effective rank analysis to inspect the geometries of Gaussians of 3DGS and its variants, shedding light their underlying structures. Based on the findings from effective rank analysis, we propose a novel effective rank regularization method in Section 4.2
4.1 Effective rank analysis of 3D Gaussians
We propose to analyze the effective rank to investigate the structural dynamics of individual 3D Gaussians by calculating the effective rank of the covariance matrix of the Gaussians. The covariance matrix of the 3D Guassians is defined as Σ, = R.S.S, and the diagonal matrix after SVD D == S with real positive singular values in a descending order as follows:
> 0.
where omit subscript of sy for brevity.
5

---

Accordingly, we can derive the effective rank of a 3D Gaussian GR with the covariance matrix Σn- The entropy term is (G:) = ((1 92; 9z) == -- % log Gi, with
0t 22 $3
q = (91, 92 93)
where S
Σ S
(8)
S
S
and the effective rank of 3D Gaussian GR with the covariance matrix Eg defined as follows:
erank(Gx) exp (G+))
The effective rank, being a differentiable extension of an integer rank, is a suitable tool for geometric analysis of 3D Gaussians since jointly considers all of the scale parameters and can identify the relative scales of the three axes. The advantage of effective rank becomes more apparent when compared to recent works that only analyze individual or pair-wise variances of the 3D Gaussians [TS] Such approaches do not fully represent the geometry of Gaussians, potentially leading to planar and needle-like Gaussians being categorized together. For better understanding, we visualize the effective ranks of sphere and 2D disks in Fig.
With the distinct advantage of approach, we can differentiate between needle-like Gaussians, which have effective ranks close to I, and planar disk-like Gaussians. To reconstruct a scene with an accurate surface, need Gaussians that represent plane that aligns and concentrates well with the surface [30]. Ideally, 3D Gaussians with erank(G*) 2 2 are preferred, but Gaussians with an effective rank smaller than 2 are also required for representing thin and elongated objects and patterns. However, the needle-like Gaussians with erank(Gs) Rs 1 are undesirable because they account for a negligible region of the surface and produce degenerate results in novel views.
The first row of Fig. (green graph) shows the effective rank histogram for 3DGS during training. As the model converges, the number of 3D Gaussians with crank(Ge) R= increases, indicating overfitting without improvements in PSNR and Chamfer distance metrics (metrics are provided in the Appendix A.5] Table This indicates that the majority of "flat" Gaussians (singular values close to 0) are actually needle-like (erank R 1), rather than disk-like (erank x 2). is also interesting to note that 3DGS naturally forms small mode at erank (rank(Gu) 2, indicating an observed preference that can be further strengthened with our regularization.
Despite having different geometric constraints on the Gaussians, SuGaR 100 (the second row in Fig. 2) and 2DGS [13] (the third row in Fig. ( also exhibit a similar tendency to have a large amount of needle-like like Gaussians with a single dominant variance along an axis. Notice that all Gaussians in 2DGS start with an effective rank of exactly 2, but the majority still fail to remain disk-shaped and instead become needle-like 2D Gaussians.
4.2 Optimization
The real-valued and differentiable nature of the effective rank allows us to utilize it as regularization objective to impose structural constraints 3D Gaussians. Specifically, our goal is to keep the effective rank of 3D Gaussians below 2, thereby promoting planar shapes, while penalizing Gaussians with effective rank close to 1 to minimize needle-like artifacts. Although disk-like Gaussians with crank(Gk) Rv 2 are preferred, shapes with erank(g.) < 2 are also essential for representing complex geometries. We propose effective rank regularization term that increases exponentially as the effective rank nears 1, strongly penalizing such Gaussians:
Perank Σ
Aerank max( log(erank 1 c),0) + 83.
(10)
where e = x 10-5 ensures numerical stability, and is the smallest scale parameter of Gk. The regularization effectively constrains the effective rank of Gaussian primitives when added to the baselines, as shown in the purple graphs of Fig. 2 Also, the regularization is scheduled to be applied from 7000-iteration, adhering to the coarse-to-fine training paradigm, which enables stable training Gaussians.
upon early iterations with erank
ADC algorithm We adopt the revised version of the densification algorithm presented in 36 which densifies Gaussians based on the summation of norms instead of the norm of the summation in

---

Table I: Chamfer distance and PSNR report on DTU dataset. +c denotes the erank regularization.
Mothod
24
37
40
55
63
65
69
83
97
105
106
110
114
118
122
Mcan
Std.
PSNR
3DGS
2.14
1.53
2,08
1.68
3,49
2.21
2.07
2.22
1.79
2.55
1.52
1.50
1.96
32.82
3DGS+c
0.86
0.77
0.88
0.52
1.29
1.44
0.96
1.30
2,09
0.72
0.87
1.40
0.88
0.94
0.66
1.04
0.39
33.09
SuGaR
1.47
1.33
1.13
0.61
2.25
1.71
1.15
1.63
1.62
1.07
0.79
2.45
0.98
0.88
0.79
1.33
0.52
31.59
SuGaRte
0.78
0.89
1.28
1.45
1.60
1.45
0.66
1,00
0.33
2DG8
0.48
0.91
0.39
0.39
1.01
0.83
0.81
1.36
1.27
0.76
0.70
1.40
0.40
0.76
0.52
0.80
0.33
32.43
2DGS+c
0.46
0.86
0.39
0.40
0.96
0.84
0.81
1.29
1.19
0.72
0.70
1.32
0.40
0.75
0,50
0.77
0.30
32.57
GDF
0.50
0,82
1.12
0.78
1.18
1.29
0.71
0.77
0.90
0.49
0.74
32,88
GO
0.45
0.66
0.32
0.42
0.97
0.78
0.64
1.13
1.22
0.64
0.62
0.70
0.40
0.53
0.48
0.66
0.26
33.01
Table 2: Ablation study result of our method on DTU dataset. (a): the fixed densification (ADC)
algorithm, (b): erank regularization, (c): optional bag of tricks discussed in the Appendix.
Method
37
63
110
114
PSNR
3DGS
2.14
1.53
2.08
1.68
3.49
2.21
1.43
2,07
2.22
1.75
1.79
2.55
1.53
1.52
1.50
1.96
32.82
1.24
0.97
1.09
0.62
1.45
1.55
1.14
1.58
2.31
0.92
1.08
1.72
1.02
0.97
1.26
32.97
+a+b
0.85
0.77
0.88
0.51
1.21
1.45
0.96
1.30
2.09
0.72
0.86
1.45
0.87
0.94
0.66
1.03
33.09
+a+b-c
0.45
0.66
0.32
0.42
0.97
0.78
0.64
1.13
1.22
0.64
0.62
0.70
0.40
0.53
0.48
0.66
33.01
Eq. 3 (further details in Appendix A.4). This change is particularly important for our regularization method. Disk-like Gaussians, unlike needle-like ones, often fail to satisfy the splitting criterion set by Eq. because their axes with larger variances produce smaller gradient signals per pixel. Moreover, since disk-like Gaussians typically cover more pixel space, unaligned signals tend to cancel each other out. In contrast, the revised densification algorithm facilitates the splitting of disk-like Gaussians. Notably, due to their superior ability to reconstruct surfaces compared needle-like Gaussians, our method requires approximately 10% fewer Gaussians than the baseline [16]
Experiments
We evaluate the effective rank regularization, comparing its performance as an add-on to baseline models. Additionally, we analyze the contributions of different components of the method.
5.1
Implementation
The regularization hyperparameter Aerank 0.01 is used for all training. For other components belonging to the baselines, we use the same settings as described in the corresponding papers. All experiments are conducted on a Tesla V100 GPU. For mesh extraction, the truncated signed distance function (TSDF) fusion with Open3D 38 is used, with details in the Appendix A.3
5.2 Comparison
Dataset We evaluate our model on the DTU [4 and Mip-NeRF360 (2 datasets. The DTU dataset consists of 15 forward-facing bounded scenes with a resolution of 1600 x 1200. Following prior standards [13 (3). we downsample the images
resolution of 800 600. The DTU dataset is used for evaluating both geometry reconstruction (using Chamfer distance) and novel view synthesis. The 606 dataset comprises 9 indoor and outdoor scenes with images a resolution of 1600 x 1050 and is used exclusively for novel view synthesis evaluation. For novel view synthesis, the images split into training and test sets, while the entire set of images is used for geometry reconstruction. COLMAP 26 27 is used to initialize point clouds for the baselines.
Baselines Our method is applicable to other baselines as an add-on term. Therefore, we compare baselines with and without our regularization. We choose SuGaR, 2DGS. and GOF as our baselines, works that focus on better geometry reconstruction, along with the original 3DGS. All of the experiments are performed with the proposed setting of the original paper.
Geometry reconstruction Table presents the quantitative results of geometry reconstruction on the DTU dataset. We report the Chamfer distance for each scene, along with the mean Chamfer
7

---

SuGaR
ZDGS
GOF
GOF+erank reg.
Figure 4: Visualization of the reconstructed mesh using TSDF. Baseline methods often exhibit empty holes, while our regularization term enforces disk-like Gaussians, reducing such artifacts and improving surface reconstruction.
G.T.
3DGS needles
GOF
GOP erank reg.
Figure 5: Normal reconstruction results on the DTU dataset. Needle-like Gaussians often leave empty holes or transparent regions, resulting in hollow incomplete reconstructions, as seen on the pear surface. The effective rank regularization significantly mitigates these artifacts, leading to more accurate geometry reconstruction.
distance and mean PSNR. The "+e" symbol indicates the addition of the effective rank regularization (with fixed densification) to the baseline methods.
The results show that methods enhanced with our add-on term outperform the baselines. Notably, applying our regularization to 3DGS (3DGS+e) results in a significant improvement in geometry reconstruction, demonstrating the effectiveness of the regularization. This supports our hypothesis that reducing needle-like like Gaussians and achieving flatness as in Fig. 2 improves performance. Additionally, the figure shows that SuGaR contains both needle-like and non- non-planar Gaussians with effective ranks greater than 2. By attaining flatness and removing spikes through effective rank regularization, we achieve substantial performance gain for SuGaR (SuGaR+e).
GOF and 2DGS already incorporate well-designed regularization terms, such as depth distortion loss [13 22, to align Gaussians with surfaces and enhance geometry reconstruction. Furthermore, 2DGS explicitly uses 2D Gaussians as their primitive, inherently achieving planarity. Nonetheless, our method prevents Gaussians from converging into needles in both approaches (and enforces flatness in GOF), resulting in performance gains.
Fig. 4 shows mesh reconstruction results, where baseline methods often exhibit empty holes in the reconstructed meshes. Our regularization term enforces disk-like Gaussians, reducing such holes and proving advantageous for surface reconstruction.
Fig. Fig.5land the first row of Fig. display normal reconstruction results. Fig. the resulting image from GOF shows spiky artifacts and hollow surface on the pear. Similarly to the mesh results, needle-like Gaussians often fail to cover the entire area, leaving empty holes or transparent regions, resulting in hollow or incomplete reconstructions. The effective rank regularization mitigates these noisy artifacts, leading to more accurate reconstruction the underlying geometry.
Novel view synthesis Since 3D reconstruction from 2D images is an ill-posed problem, Gaussians tend to overfit to the training views, converging into needle-like shapes and causing spiky artifacts in test views, as shown in Fig. (b). (c), and Figl6) For better understanding. we visualize Gaussians with erank(Gk) 1.02 (scale ratio of approximately 20:1 or larger) in red. Our method mitigates overfitting and the resulting artifacts enforcing structural priors on the Gaussians.

---

3DGS needles
3DGS
3DGS + crank reg.
Figure 6: Qualitative comparison on DTU dataset. Gaussians with erank < 1.02 are visualized in red. Our regularization term mitigates needle-like artifacts in novel views.
3DGS ( PSNR: 25.1 114 GB)
3DGS crank reg. (PSNR: 25.5 / 1.2 GB)
Figure 7: Qualitative comparison on Mip-NeRF360 dataset. Our method effectively represents thin objects, achieving better visual quality and compactness
Furthermore, as seen in Fig. our method adaptively preserves some elongated Gaussians when necessary, allowing the representation of slender structures. 'he results indicate that while 3DGS heavily relies on needle-like Gaussians to represent the scene, our method limits their use only when required, leading to improved novel view synthesis performance.
We also provide quantitative results in Table where we report the average PSNR for the DTU dataset. Results for Mip-NeRF360 are reported in Table 3 in the Appendix A.5) While many geometry regularization techniques degrade visual quality, our method does exhibit this trade-off and actually shows slight improvements by properly constraining the shape of the Gaussians.
Efficiency As shown in Fig. 7 and Table the efficacy of disk-like Gaussians in 3D reconstruction, compared to needle-like Gaussians, leads to better memory footprint. The average storage usage for the DTU and Mip-NeRIF36 datasets is reported Table (Appendix A.5))
5.3 Ablations
Our method comprises two key components: (a) the fixed densification (ADC) algorithm and (b) the effective rank regularization. We performed an ablation study on these components to observe their performance gains compared to the naive 3DGS method. Table 2 shows the Chamfer distance and PSNR measured on the DTU dataset. The results indicate that both components contribute to performance gains in geometry reconstruction and novel view synthesis tasks. Additionally, incorporating techniques such as depth distortion loss [3] can further enhance the best best-performing model (row +a+b+e). These techniques are discussed in the Appendix A.2
6 Conclusion
Limitations Our regularization term constrains individual Gaussians but does not account for the local and global structure of the scene. Thus, may beneficial pair our method with structure-aware regularizations, such as the depth distortion loss [13), which considers the Gaussians

---

along the ray collectively. Another limitation is the manual selection of the hyperparameter Aerank- While our chosen hyperparameter works well for the scenes used in our evaluation, it may not be optimal for extreme scenes dominated by thin objects and structures.
Acknowledgments and Disclosure of Funding
Junha Hyung and Susung Hong conducted this work during the internship at NAVER AI Lab. The NAVER Smart Machine Learning (NSML) platform [T7] had been used for experiments. This work was supported by KAIST-NAVER hypercreative AI center. This work was partly supported by Institute for Information communications Technology Promotion (IITP) grant funded by the Korea government (MSIT) No.RSS-2019 911075 Artificial Intelligence Graduate School Program (KAIST)). This work was also partly supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. NRF-2022R IAB5500201911)
References
[1] Barron, B. Mildenhall, M. Tancik, Hedman, Martin-Brualla, and P. P. Srinivasan. Mip-nerf: A multiscale representation for anti- neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5855-5864, 2021.
[2] J. T. Barron, B. Mildenhall, D. Verbin. P. P. Srinivasan, and P. Hedman. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. In Proceedings the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 1470-5479, 2022.
[3] Barron, B. Mildenhall, D. Verbin, P. P. Srinivasan, and P. Hedman. Zip-nert Anti-aliased grid-based neural radiance fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 19697-19709 2023.
[4] S. R. Buld, L. Porzi, and P. Kontschieder. Revising densification in gaussian splatting, arXiv preprint arXiv:2404.066 2024.
A. Chen, Z. Xu, A. Geiger, J. Yu, and H. Su. Tensorf: Tensorial radiance fields. In European Conference on Computer Vision, pages 333-350. Springer, 2022.
[6] H. Chen, C. Li, and G. H. Lee. Neusg: Neural implicit surface reconstruction with 3d gaussian splatting guidance. arXiv preprint arXiv:2312
2023.
[7) Z. Chen, T. Funkhouser, P. Hedman, and A. Tagliasacchi. Mobilenerf: Exploiting the polygon rasterization pipeline for efficient neural field rendering mobile architectures. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16569-16578, 2023.
[8] C. Dudai. M. Alper, H. Bezalel, Hanocka. Lang, and Averbuch-Eloc Halo-nerf: Learning geometry- guided semantics for exploring unconstrained photo collections. In Computer Graphics Forum, page c15006. Wiley Online Library, 2024.
[9] S. Fridovich-Ked A. Yu, M. Tancik, Q. Chen, B. Recht, and A. Kanazawa. Plenoxels: Radiance fields without neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5501-50 5501-550, 2022.
[10] A. Guédon and Lepetit. Sugar: Surface-an aeftace-lliged gaussian splatting for efficient 3d mesh reconstruction and high-quality mesh rendering. arXiv preprint arXiv:2311.1277, 2023.
[11] P. Hedman, J. Philip, T. Price, J.M. Frahm, G. Dreutakis, and G. Brostow. Deep blending for free-viewpoint image-based rendering. ACM Transactions on Gruphics (ToG), 37(6):1-15, 2018
[12] P. Hedman, P. P. Srinivasan, B. Mildenhall, T.T. Barron, and P. Debevec. Baking neural radiance fields for real-time view synthesis. In Proceedings of the IEFF/CVF International Conference Computer Vision, pages 5875-5884 2021.
[13] B. Huang, Z. Yu, A. Chen, A. Geiger, and S, Gao. 2d gaussian splatting for geometrically accurate radiance fields. arXiv preprint arXiv:2403.1 17888, 2024.
[14] R. Jensen, A. Dahl, G. Vogiatzis, E. Tola, and H. Aanæs Large scale multi-view stereopsis evaluation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 406-413, 2014.
10

---

[15] Y. Jiang, J. Tu, Y. Liu. X. Gao, X. Long. W. Wang, and Y. Ma. Gaussianshader: 3d gaussian splatting with shading functions for reflective surfaces. arXiv preprint arXiv:2311.1997,. 2023.
[16] B. Kerbl, G. Kopanas, T. Leimkühler, and G. Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions Graphics, 42(4):1-14, 2023.
[17] H. Kim. M. Kim, D. Seo, J. Kim, H. Park, S. Park, H. Jo, K. Kim, Y. Yang. Y. Kim, et al. Nsml: Meet the mlaas platform with real-world case study. arXiv preprint 07X19:1810.09957, 2018.
[18] 7. Li, T. Müller, A. Evans, R. H. Taylor, M. Unberath, M.-Y. Liu, and C.-H. Lin. Neuralangelo: High- fidelity neural surface reconstruction. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition. pages 8456-8465, 2023.
[19] R. Martin-Brualla N. Radwan, M. S. Sajjadi, J. T. Barron, A. Dosovitskiy, and D. Duckworth. Nerf in the wild: Neural radiance fields for unconstrained photo collections. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7210-7219, 2021.
[20] B. Mildenhall, P. Srinivasan, M. Tancik, T. Barron, R. Ramamoorthi, and R. Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1):99-106, 2021.
[21] T. Müller, A. Evans, C. Schied, and A. Keller. Instant neural graphies primitives with a multiresolution hash encoding. ACM transactions on graphics (TOG), 41(4):1-15, 2022.
[22] M. Oechsle, S. Peng, and A. Geiger. Unisurf: Unifying neural implicit surfaces and radiance fields for multi-view reconstruction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5589-5599, 2021.
[23] C, Reiser, S. Garbin, P. P. Srinivasan, D. Verbin, R. Szeliski, B. Mildenhall, J. T. Barron, P. Hedman, and A. Geiger. Binary opacity grids: Capturing fine geometric detail for mesh-based view synthesis. arXiv preprint arXiv:2402. 12377, 2024.
[24] C. Reiser, R. Szeliski, D. Verbin, P. Srinivasan, B. Mildenhall, A. Geiger, J. Barron, and P. Hedman. Merf: Memory-efficient radiance fields for real-time view synthesis in unbounded scenes. ACM Transactions on Graphics (TOG), 42(4):1-12, 202..
[25] O. Roy and M. Vettelli. The effective rank: A measure of effective dimensionality. In 2007 15th European signal processing conference, pages 606-610. IEEE, 2007.
[26] J. L. Schönberger and J.-M. Frahm. Structure-from-mon mottonn revisited. In Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
[27] J. I.. Schönberger, F., Zheng, M. Pollefeys, and J.-M. Frahm. Pixelwise view selection for unstructured multi-view stereo. In European Conference on Computer Vision (ECCV), 2016.
[28] Y. Shi, Y. Wu, C. Wu, Liu, C. Zhao, H. Feng. Liu, L. Zhang, Zhang, B. Zhou, al. Gir: gaussian inverse rendering for relightable scene factorization. arXiv preprint arXiv:2312.05133 2023.
[29] J. Sun, X. Chen, Q. Wang, 7. Li, H. Averbuch-Flor X. Zhou, and N. Snavely. Neural 3d reconstruction in the wild. In ACM SIGGRAPH 2022 conference proceedings, pages 1-9, 2022.
[30] P. Wang, Liu, Y. Liu, C. Theobalt, Komura, and W. Wang. Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. arXiv preprint arXiv:2 10689, 2021.
[31] Y. Wang, Q Han, M. Habermann, K. Daniilidis, C. Theobalt, and L. Liu. Neus2: Fast learning of neural implicit surfaces for multi-view reconstruction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3295-3306 3306, 2023.
[32] Z. Wang. Т. Shen, M. -Dvid, N. Sharp. J. Gao, A. Keller, S. Fidler. T. Müller, and Z. Gojeic. Adaptive shells for efficient neural radiance field rendering. arXiv preprint arXiv:2311. 2at::23111100111 10001,, 2023.
[33] L. Yariv, J. Gu, Y. Kasten, and Y. Lipman. Volume rendering of neural implicit surfaces. Advances in Neural Information Processing Systems, 34:4805 4815, 2021.
[34] L. Yariv, P. Hedman, C. Reiser, D. Verbin, P. P. Srinivasan, R. Szeliski, J. T. Barron, and B. Mildenhall. Bakedsdf: Mcshing neural sdfs for real-time view synthesis. In ACM SIGGRAPH 2023 Conference Proceedings, pages 1-9, 2023.
[35] Z. Yu, A. Chen, B. Huang, T. Sattler, and A. Geiger. Mip-splatting: Alias-free 3d gaussian splatting. arXiv preprint arXiv 2023.
11

---

[36] Z. Yu. T. Sattler. and A. Geiger. Gaussian opacity fields: Efficient and compact surface reconstruction in unbounded scenes. arXiv preprint arXiv:240.11077, 2024.
[37] K. Zhang, G. Riegler, N. Snavely, and V. Koltun. Nerf++: Analyzing and improving neural radiance fields. arXiv preprint arXiv:2010.07492, 2020.
[38] Q.-Y. Zhou. J. Park, and V. Koltun. Open3d: A modem library for 3d data processing. arXiv preprint 1rXiv:880100944 2018.
[39] M. Zwicker, H. Pfister, J. Van Baar, and M. Gross, Ewa volume splatting. In Proceedings Visualization, 2001. V15' pages 29-538. IEEE, 2001.
12

---

A Appendix supplemental material
A.1 Broader impact
The broader impact of our work on 3D reconstruction lies in its potential advance various fields such virtual and augmented reality, medical imaging, and digital content creation by enabling more efficient and high-quality 3D model generation. However, as with any advanced technology. also presents potential risks and avenues for misuse. For instance, enhanced 3D reconstruction techniques could be exploited to create deepfakes or unauthorized reproductions of proprietary designs, posing ethical and legal challenges. To mitigate these risks, propose implementing strict usage guidelines to ensure the integrity and rightful use of 3D models. We aim to maximize the positive impact of our research while minimizing potential negative consequences.
A.2 Additional regularization
For rendering normals, we add other regularization terms, such as depth distortion loss 13 and normal regularization, as proposed in [13] 36). (We do not utilize these regularization terms for evaluating effective rank regularization as an add-on module in Tab. [ The depth distortion loss, which concentrates splats surface and mitigates floater artifacts, given
La Ad
wew 2 2z11
(11)
where w; a; (x) og (x))) for the blending weight and z; is the depth of the intersection point of the th Gaussian, and i are indexes over Gaussians contributing to a certain ray.
The normal regularization minimizes the difference between the rendered normal map n the splats and the gradient normals ñ derived from the rendered depth map,
Cn An Al|,
(12)
which locally aligns the 3D Gaussians with actual surfaces. Since the effective rank regularization does not account for the local and global structure of the scene, it is beneficial to pair our method with these structure-awar regularizations.
A.3 Mesh extraction
We utilize the Truncated Signed Distance Function (TSDF) fusion for mesh extraction. The algorithm encodes the distance of any point the voxel grid the nearest surface, with the distance being truncated to a maximum value to limit the influence of faraway points. The sign of the distance func- tion indicates whether the point is inside (negative) or outside (positive) the object. Multiple TSDFs are combined from different viewpoints to create more accurate and complete 3D reconstruction, forming coherent and comprehensive 3D model. The Marching Cubes algorithm then used for triangulation.
A.4
ADC fix
We adopt the revised version of the densification algorithm presented in 4 (36. which densifies Gaussians based on the summation of the norm instead of the norm of the summation in Eq.
aL 0p: 0p. ou
T.
(13)
As discussed in the main paper, this approach is crucial to our regularization because disk-like Gaussians typically cover more screen space and receive gradient signals from various pixels, which can cancel out when summed. The revised algorithm ensures the effective splitting of Gaussians with our regularization. However, due the efficiency disk-like Gaussians in surface reconstruction, method requires about 10% fewer Gaussians compared to the baseline [16].
13

---

Table 3: Quantitative results on Mip-NeRF 360
dataset.
Outdoor Scene
Indoor scene
PSNR
SSIM
LPIPS 0
PSNR
SSIM 1
LIPPS
Mobile-NeRF 7
21.95
0.470
0.470
BakedSDF 34
22.47
0.585
0.349
27.06
0.836
0.258
BOG [23
23.94
0.680
0.263
27.71
0.873
0.227
NeRF 20
21.46
0.458
0.515
26.84
0.790
0.370
Deep Blending
21.54
0.524
0.364
26.40
0.844
0.261
Instant NGP 21
22.90
0.566
0.371
29.15
0.880
0.216
MERF 224
23.19
0.616
0.343
27.80
0.855
0.271
MipNeRFF30 2
24.47
0.691
0.283
31.72
0.917
0.180
3DGS 16
24.64
0.731
0.234
31.13
0.920
0.189
3DGS+e (Ours)
24.93
0.757
0.221
31.16
0.953
0.181
Table 4: Storage usage of our method, along with Chamfer distance, PSNR, and optimization time.
Dataset
Method
CD 4
PSNR f
Time +
MB (Storage) 4
3DGS
1.96
32.82
11.2m
113
DTU
3DGS+e
1.03
33.09
11.1m
98
3DGS
27.52
41m
734
Mip-NeRF360
3DGS+e
27.70
40m
646
A.5 Additional quantitative results
We report novel view synthesis results on the Mip-NeRF360 dataset in Table The results show that our add-on regularization term improves the visual quality of 3DGS in terms of PSNR, SSIM, and LPIPS. Also, the method even shows comparable or slightly better performance compared the NeRF variants with slow and computationally intensive rendering.
We report the training time of our method in Table 4 The training time for 3DGS on the DTU [14] dataset averages 11.2 minutes per scene. Adding the effective rank regularization with the densifica- tion fix incurs no overhead since the additional computation compensated with a reduced number of Gaussians. Total training time is on average 1 1.1 minutes for the DTU dataset and 40 minutes for the Mip-NeRF360 dataset, on single V100 GPU, reported in Table
Also, with a reduced number of Gaussians, our method requires less memory and storage for scene representation, in Table 4 While being more compact, method outperforms baselines in terms of Chamfer distance and PSNR.
Table 5 demonstrates Chamfer distance and PSNR changes during the course of training, for the baselines shown in Fig. 2 Results are reported for the scene 37 of the DTU dataset. Needle-like Gaussians increase, but the performance plateaus, indicating overfitting. Additionally, different Gaussian structures with similar metrics suggest the heterogeneous nature of Gaussians in 3DGS and its variants. Also, the reported "Number of needles" corresponds to Gaussians with effective rank smaller than 1.04. The results suggest that our regularization term effectively minimizes the number of needles without visual quality trade-off.
We present per scene PSNR on the DTU dataset in Table 6 The mean PSNR is already shown in Table and Table of the main paper.
A.6 Cause of needle-like Gaussians
While not directly related to our methodology, we investigate some reasons for the convergence of 3D Gaussians into anisotropic Gaussians with dominant variance.
First, the scale the 3D Gaussians is not properly constrained due to the dilation operation, which adds small constant screen space Gaussians [16] to ensure minimum scale, noted in Mip-
14

---

Table 5: Chamfer distance and PSNR changes during the course of training for the baselines shown in Fig. 2 for scene 37 of DTU dataset. Needle-like Gaussians increase, but the performance plateaus, indicating overfitting. Additionally, different Gaussian structures with similar metrics suggest the heterogeneous nature of Gaussians in 3DGS and variants. Reported "Number of needles" correspond to Gaussians with effective rank smaller than 1.04.
CDL
PSNR
Method
15k
30k
15k
30k
3DGS
1.5
1.53
27.00
26.98
SuGaR
1.21
1.23
23.64
23.52
2DGS
0.89
0.88
24.89
24.87
Number of needles
PSNR1
Ok
15k
30k
30k
3DGS
0
3170
16320
26.93
3DGS+e
0
28
23
27.21
Table 6: Additional ablation DTU dataset, reporting PSNR for each scene. (a): the fixed densification (ADC) algorithm, (b): crank regularization.
Method
24
37
40
3DGS
30.45
26.93
29.79
31.92
35,42
31.09
28.34
38.00
30.69
27.14
30.31
32.01
35,93
31.23
28.04
37.95
+a+b
30.90
27.21
30.42
32.23
35.81
31.62
28.41
38.00
Method
97
105
106
110
114
118
122
Mean
3DGS
30.20
34.32
35.00
34.65
30.86
37.25
38.07
32.82
30.25
34.30
35.11
34.59
31.10
37.65
38.21
32.97
+a+b
30.27
34.41
35.22
34.69
31.20
37.69
38.23
33.09
Splatting [35]. Combined with the inherent implicit shrinkage bias of 3DGS [16 35), this results in the underestimation of the scale parameters during the optimization process.
Second, the densification along the longer axis does not occur effectively since the longer axes, or the axes with large variance, have smaller gradients. When Gaussians move the direction of the shorter axis, pixel values change abruptly. In contrast, there are only small changes in pixel values ?u
when moving along the longer axis. Specifically, when 0p: aligns with the direction of the longest axis, the gradient values are typically small. Consequently, the norm of the final gradient often falls below the densification threshold @L < Tx preventing effective densification. We visualize aG, âx dp. for better understanding. Therefore, the splats in arrows in Fig. 8 (a), which is proportional to Bu biased towards adjusting their scale parameters (Fig. g (b)) rather than splitting along the longer axis, converging into needle-like Gaussians.
Third, scale parameters kept the same after splitting, needles not shortened after densifica- tion.
It will be interesting future work to delve deeper into these reasons and address the problem with other approaches.
A.7 Additional qualitative results
We present normal rendering of our method results. Fig. 9 are results of the scene 122, with the depth distortion and normal regularization loss used together. Fig. TO shows the results of scene 55. Fig. [ shows the rendering results of the Mip-NeRF360 dataset of our method. We visualize Gaussians with an effective rank smaller than 1.02 in red. The effective rank regularization adaptive to the scene, reducing the number of needle-like Gaussians, while effectively representing the required regions.
15

---

Gradient a 20 Gaussian
040
(x)
(a)
(b)
Figure 8: (a): Visualization 09 in arrows, which is proportional Pu towards adjusting its scale parameters rather than splitting along the longer axis, converging into needle-like Gaussians.
8p. (b) The splats are biased
8x
Figure 9: Normal rendering results of DTU dataset (scene 122) of our method, with depth distortion and normal regularization loss.
16

---

Figure 10: Normal rendering and visual rendering results of DTU dataset (scene 55) of our method, with depth distortion and normal regularization loss.
17

---

Figure 11: Rendering results of
dataset of our method. We visualize Gaussians with an effective rank smaller than 1.02 in red. The effective rank regularization is adaptive to the scene, reducing the number of like Gaussians, while effectively representing the required regions.
18

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Abstract and introduction has main claims and overall explanation of the paper included.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We include limitations and discussion.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
19

---

Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We fully disclose information including hyperparameters and the code base.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
20

---

Answer: [Yes]
Justification:
We release the code and the data is publicly available.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ /pippe.cc/pullic/guuii wwwiii/gguie///eeeeeeoeic CoodeSubeiiieeee ssi wsionnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Yes, we provide all the details necessary.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We provide mean and standard deviation of the results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
21

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of exccution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We specify the GPU and amount of time takes for training, along with memory footage.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: htttps://neurpps. httparrpp...ppppp.ccccccccecee.d
Answer: [Yes]
Justification:
Yes our paper conform to NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations their jurisdiction..
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss the societal impacts in the Appendix.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
22

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification:
We discuss such information in the Appendix.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We are the original owners of code and the models.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
23

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We provide code and the model.
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
24


Reviews:
[{"review_id": "vbpYmHa3t7", "reviewer": "Reviewer_biCn", "summary": "The paper identifies a common problem in 3D Gaussian splatting where Gaussians converge to needle-like shapes, with its variance mostly contained in one axis. These needle-shaped Gaussians can cause artifacts and inaccurate surface reconstruction. \nTo quantify this phenomenon, the paper uses the concept of effective rank, a continuous generalization of the rank of a matrix.  During scene reconstruction, while 3D gaussians may begin with an effective rank of 3, they tend to converge to an effective rank of 1 after several thousand iterations of training.  Based on effective rank, the paper proposes a regularization method to reduce needle-shaped Gaussians. On the DTU dataset, the authors show that 3DGS with rank regularization causes better chamfer distance and PSNR metrics.", "questions": "What happens if you do not regularize erank(G) < 2? Do Gaussians with erank(G) > 2 hurt reconstruction quality? \n\nWhat are x and o in Fig. 8?", "limitations": "Yes the authors have discussed the limitations and potential societal impacts.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper introduces a method to prevent needle-like Gaussians in 3DGS, an important challenge in view synthesis, and presents a clear way to quantify the problem. The authors propose a new regularization term that is simple and can be used alongside other Gaussian splat variants such as 2DGS and SuGaR. On the DTU dataset, the authors report better chamfer distance and PSNR metrics than prior work.\n\nOverall I found the paper very interesting and its claims well justified. Because of the widespread popularity of Gaussian Splatting, the method can be useful to both researchers and 3D designers.", "weaknesses": "While the paper introduces a new way to quantify needle-like Gaussians, this is not the first paper to propose a regularization term for them.  PhysGaussian [A] introduces an anisotropic regularizer to prevent skinny Gaussians. This regularization term has already been implemented in Nerfstudio’s Splatfacto method [B]. This seems like the most relevant and important comparison to the proposed method, and is completely missing from the paper. This missing comparison is the main reasoning behind my score. \n\nIn addition, I wish that the paper did a deeper investigation into what causes spikey Gaussians. I found the discussion in Appendix A.4. very fascinating, and think it could be very valuable to the community if elaborated on. Maybe a simple experiment validating the hypothesis could strengthen the paper. \n\nMinor Comments: \nI found that Fig. 1 was hard to parse. I think it would help to include labels for the figure’s images. \n\n[A] https://arxiv.org/pdf/2311.12198\n[B] https://docs.nerf.studio/nerfology/methods/splat.html", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer for highlighting the importance of the task, as well as the effectiveness and clarity of our method. We are also grateful for suggesting potential baselines and ablations to strengthen our work.\n\n### PhysGaussian\n>Initially, we did not consider PhysGaussian due to its focus on physics-grounded deformation, with regularization primarily aimed at reducing spiky artifacts caused by deformation. However, we acknowledge the similarity between our method and PhysGaussian and agree that it could be used to enhance 3D geometry. We have conducted additional experiments using PhysGaussian’s regularization and plan to include these results in the camera-ready version of our paper (possibly in the appendix).\n\n### vs. PhysGaussian and variants\n>Before presenting the results, we want to clarify that PhysGaussian’s regularization method differs from ours. Some works, including PhysGaussian, have tried regularizing Gaussian primitives, but none focus on all three axes of the Gaussians. PhysGaussian considers two axes (max and min scale axes) to remove spiky Gaussians, which reduces spiky Gaussians but does not enforce disk-like (flat) Gaussians. We present the erank histogram of PhysGaussian in the attached PDF Fig. 3 (b), showing reduced spiky Gaussians (erank(G) $\\approx$ 1) but not explicitly enforcing disk-like Gaussians (erank(G) = 2) or penalizing Gaussians with erank(G) > 2.\nAdditionally, methods like GaussianShader only consider the axis with the minimum scale, minimizing it to make the Gaussian primitive “flat”. 2DGS uses 2D surfel as a primitive, achieving similar effects. But these methods does not handle spiky Gaussians. Our method accounts for all three axes, enforcing disk-like Gaussians without needle-like Gaussians, using effective rank. This approach leads to better quantitative results, as shown in the table below.\n\n> Also, our approach provides interpretability of the Gaussian shapes via effective rank (erank(G)=3: sphere, erank(G)=2: disk, erank(G)=1 needle), and logarithmic loss is known for its stability in optimization problems (we emprically prove this in below table, also refer to results in reviewer #4). Please refer to the response to reviewer #4 for more justification and experimental results.\n\n### deeper investigation into what causes spiky Gaussians \n> please refer to general rebuttal response above.\n\n\n### Additional comments\n>- We appreciate the comment and will update Fig. 1 with labels.\n>- Yes, non-disk-like Gaussians (erank(G) > 2) negatively impact the geometry metric. Prior to submission, we conducted various experiments targeting erank(G) = x, where x > 2, and consistently found that the results were worse compared to our method. As an example, we present a case where the loss is applied with a target erank(G) = 2.5. The results are shown in the table below.\n>- In Fig. 8(b) of the main paper, we are showing a toy experiment where a Gaussian is not split into two Gaussians (x), and instead, the scale is adjusted (o), which is not optimal in this case. We apologize for the confusion and will improve the figure and caption. We provide more 2D toy experiment in the attached PDF.\n\n\n| DTU scan | 24 | 37 | 40 | 55 | 63 | 65 | 69\n|:--------:|:--------:|:--------:|:--------:|:--------:|:--------:|:--------:|:--------:|\n| 3DGS |   2.14 |   1.53 | 2.08 | 1.68 | 3.49|2.21|1.43|\n| 3DGS+e |   0.85 |  0.77 | 0.88 | 0.51 | 1.21|1.45|0.96|\n| PhysGaussian |   0.87 |   0.81 | 0.86 | 1.36 | 2.99|1.97|1.46|\n| 3DGS+erank(G)=2.5 |  1.08  | 1.26   | 1.34 | 0.97 |2.34 |2.31|1.06|\n\nThank you again for your valuable feedback.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0R6V3zUEYU", "reviewer": "Reviewer_w1Sn", "summary": "This paper performs a statistical analysis on 3DGS for its effective rank distribution of the learned Gaussians. It claims that most of the Gaussian learned are close to rank 1 effectively, giving needle-like artifacts in novel view synthesis and reconstruction. Hence, this paper proposes a regularization loss to discourage low range Gaussians from forming, showing improved reconstruction results.", "questions": "To summarize my opinions mentioned in the weakness section into questions, it would be:\n1. what is really the cause for the low-rank Gaussian formation and how does the proposed method address it?\n2. why is the proposed method better than the simple regularization on the ratio of the long/short gaussian axis?", "limitations": "The paper does not have potential negative societal impact and there is no significant limitation of the method that needs to be addressed other than the ones mentioned in the previous sections.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 1, "strengths": "**Motivation**\n* This paper starts with a clear and strong motivation, where the Gaussian learning with 3DGS creates needle-like artifacts. This reduces both the reconstruction and novel view synthesis quality.\n\n**Method**\n* This paper proposes an intuitive and effective method of regularizing the effective rank of the Gaussians, to regularize on the shape of individual Gaussian. The experiment results also demonstrate the effectiveness of the proposed method on various baselines, with considerable improvement in reconstruction results and minor improvement in novel view synthesis results.", "weaknesses": "I do not doubt the motivation and effectiveness of the proposed method. In fact, needle-like artifacts have been a well-known issue for 3DGS in the community. However, I feel reluctant to accept this paper due to the following reasons:\n1. Although the phenomenon is well known by many, the root cause of this issue is not clear. This paper tries to explain the cause in its appendix, and gives two hypotheses: a) incorrect dilation observed by MipGaussian, b) Gaussians are more likely to densify along their short axis instead of their long axis. Based on this, I would like to point out the following issues:\n- This part should be put into the main paper instead of the statistical analysis. The causes of the low-rank Gaussians would be much more valuable than reiterating the phenomenon that many people already know.\n- This cause analysis needs to be more rigorous. The version provided in the appendix is more of a hypothesis than an analysis. \n- If it is because of the dilation error, is it already fixed by MipGaussian? What is the rank distribution like for MipGaussian? Can the proposed method improve the MipGaussian performance as well? If yes, then something else is still causing the issue. If not, why shouldn't MipGaussian be used instead of the proposed method?\n- If it is because of the densification bias, does this problem still occur without densification at all? In fact, I have personally observed many cases of needle-like artifacts that are so long that extend across the entire image. In this case, the rendering loss should be enough to prevent this from happening without anything to do with the densification. But it is not happening, why? Is there something wrong with the rendering loss supervision we have with 3DGS? I am not saying that the densification bias is not a cause, but it requires more evidence and experiment results to be proved.\n\n2. Continuing from the previous point, the method proposed does not address the cause hypothesized. It is not fair to say that regularization methods are just bad, but there have been even simpler regularization methods used by other papers (e.g. PhyGaussian). The most common method is to regularize the ratio between the long axis and the short axis of the 3DGS. It takes only a few lines of code and works reasonably well. This paper does not mention this simpler regularization and does not compare with it as a baseline. However, I think it is very important to justify the more complicated method proposed in this paper, with either rigorous analysis as well as empirical results (preferably both). \n\nIn summary, I believe that this paper lacks a rigorous analysis of the cause of the low-rank Gaussian formation, and the proposed method lacks novelty and performance comparison against the simple naive regularization used in other papers.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your high-quality review and detailed understanding of our work. We are also grateful for acknowledging the motivation and effectiveness of our method.\n\nWe agree that shedding light on and analyzing the causes of low-rank Gaussians would add value to our paper. Therefore, we conducted additional experiments, including 2D toy experiments, Mip-Splatting (MipGaussian) analysis, and comparisons with PhysGaussian. Please refer to the general comment for the 2D toy experiment Mip-Splatting and reviewers #1 and #4 for the comparison with PhysGaussian and its variants.\n\nWhile we conducted all the experiments, we also would like to explain our approach and the reasoning behind it. Specifically, we want to address the points about the lack of “rigorous analysis of the cause of the low-rank Gaussian” and the “method not addressing the cause.” While a detailed analysis of the root cause is one way to approach the problem, we believe our approach of writing a paper is also valid.\n\nFor instance, consider a scenario where low test accuracy (phenomenon) is caused by overfitting due to insufficient data (root cause). One could address this by using more training data. However, directly addressing the phenomenon and focusing on improving test accuracy through different techniques and regularizations (without increasing data) is also a plausible approach. Many impactful papers focus on solving phenomena without rigorous root cause analysis, and they significantly contribute to the field. For example, Mip-NeRF 360, focuses on solving phenomena (floater artifact) without rigorous root cause analysis.\nOur approach is similar. We had a clear target for the optimal shape of Gaussian primitives—disk-like (flat) and non-needle-like. This made our direct approach to tackling the phenomenon more straightforward and convincing.\n\nWe do not argue that our approach is the best, but we believe it is a plausible one. Therefore, we believe that the focus should be on evaluating the logical and technical aspects of our paper rather than suggesting an alternative approach.\n\nStill, we tried our best to analyze the cause of needle-like Gaussians, presented in general rebuttal section above.\n\nAdditionally, we do not believe we are reiterating known phenomena. While many have observed spiky (needle-like) Gaussians and some papers, like PhysGaussian, address them, their impact on geometric reconstruction (evaluated with Chamfer Distance) has not been thoroughly explored. We are the first to introduce the tool of effective rank to 3DGS, enabling `interpretable` statistical evaluation of the shapes of individual Gaussians. We discovered that spiky or non-disk-like shapes of Gaussians significantly impact the geometric quality of the reconstructed scene, beyond just causing artifacts. This focus on geometric quality and analysis is a new contribution.\n\nMoreover, no previous works have focused on all three axes of the Gaussians. For more details on this aspect, please refer to our responses to reviewers #1 and #4.\n \n\n## The cause of spiky Gaussians\n> Please refer to the general comment\n\n## Comparison and justification with other variants & novelty\n> Please refer to our responses to reviewers #1 (`PhysGaussian` and `vs. PhysGaussian and variants` section) and #4 (`Justification of Using Effective Rank Regularization` and `Comparison & Ablation study` section). \n\n\nThank you once again for your valuable feedback.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FK3OEyLJga", "reviewer": "Reviewer_7LDG", "summary": "3D Gaussian Splatting is a remarkable technique in novel view synthesis. However, it usually degenerates into noodle-like shapes which sometimes bring visual artifacts and inaccurate geometry. This paper analyzes the phenomenon and proposes an effective rank loss to regularize the Gaussians. The technique seems to be a plug-and-play module for 3DGS variants. Extensive experiments suggest the effectiveness.", "questions": "1. How the normal is constructed for 3D Gaussian and how it is visualized?\n2. What is the effect of densification on geometry reconstruction in 3DGS? In Table 1. Does 3DGS+e indicate with ERANK only or plus densification? It should be clearly ablated where the performance comes from.", "limitations": "Overall, the paper did a good job with minor issues required to clarify. The limitation of tuning the hyperparameter is also discussed and the potential negative effect is reasonable. \n\n**minors:**\n\nThe normal visualization seems not to be normalized witn [0,1]", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The observation and analysis are strong and the solution of using effective rank is simple yet effective.\n2. The results show clear improvements over representative works including 3DGS, SuGaR, and 2DGS, which cover a broad range of scene representations.\n3. The paper is well-written and easy to follow. I have no issue with the written. Most of the related works are adequately discussed. \n4. The technical details are sufficient and I believe an expert in 3DGS can easily implement it.", "weaknesses": "1. There is no supplemental video, which is essential for nvs tasks. \n2. The improvement of e-rank for Novel view synthesis is limited. It would be beneficial to highlight how this could improve novel view synthesis.\n3. Although the paper analyzes that the noodle-like artifacts are due to the training bias, they can also be created to represent high-frequency details. The paper should highlight the PSNR in Figure.1 to give readers a sense that such regularization indeed improves novel view synthesis.\n4. It would be beneficial to add the number of Gaussians since enforcing the Gaussian to 2D from 1D should reduce the number of Gaussians in representing a scene.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer for acknowledging the effectiveness and simplicity of our method. Additionally, we are grateful for pointing out the strong analysis and concise writing.\n\n### Supplemental Video\n>We promise to share the video results in the project page. Please stay tuned!\n\n### Novel view synthesis\n>The primary focus of our paper is geometry reconstruction, which we evaluate using Chamfer Distance, rather than novel view synthesis. Unlike other geometry reconstruction works that has geometry and visual quality tradeoff, our method maintains high visual quality without compromise, which we later found is an additional strength of our model. Also it reduces spiky artifacts in novel views. In constrained scenes with well-prepared data, these spiky artifacts appear in very small regions, potentially explaining the limited improvement in visual quality metrics. However, in few-shot settings and renderings from extreme viewpoints (far from the training view), we observe larger needle artifacts. It would be interesting to see results with such datasets.\n\n### Figure 1 Visibility\n>Thank you for pointing out the visibility issue in Figure 1. We will improve the figure as per your suggestion. You are correct that anisotropic Gaussians are necessary for representing high-frequency details. The method should, therefore, only encourage a tendency towards disk-like Gaussians as regularization, rather than “eliminating” them.\n\n### Adding the number of Gaussians\n>We appreciate your idea. Actually, we had similar thoughts and have tested several related settings previously. Specifically, we tried lowering the densification threshold (causing more frequent densification) and lowering the pruning threshold (resulting in less frequent pruning). Neither of these experiments improved the metrics. Surprisingly, they resulted in worse PSNR in test views, despite higher PSNR for training views. This indicates that more Gaussians may lead to overfitting of the training views without improving the visual quality of novel test views. Our method, in fact, reduces overfitting, as evidenced by generally lower training PSNR but higher PSNR for testing views, as presented in the main paper.\n\n### Q: Normal Calculation \n>There are several ways to calculate the normal. “Depth normal” is derived from 2D rendered depth and is used for depth normal consistency loss (used in 2DGS and GOF). For individual Gaussians, 3DGS and SuGaR use the shortest axis as the normal. GOF uses ray-dependent normal (the normal of the intersection plane; please refer to the GOF paper for more details). These normals are rendered into screen space using the same alpha blending method used to render the RGB image.\n\n### Q: Ablation Studies \n>Please refer to Table 2 of the main paper for ablation results. Both erank regularization and densification bring performance gains. (+e indicate both).\n\nWe again appreciate the reviewer for taking the time to read through the paper and this lengthy response. Your reviews are immensely helpful in strengthening our work. We will add more results (including the supplemental video) during the discussion period.\nThank you.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "e0eqFGngHL", "reviewer": "Reviewer_n78p", "summary": "The authors present a new regularization for Gaussian splatting (GS) method that increases the Shannon entropy of the scale parameters (it was named erank as while rendering the scale is presented as a diagonal matrix, and the aforementioned calculation turns into the effective rank), to better reconstruct 3d mesh out of the GS point cloud. The paper argues that a majority of the gaussians in previous methods are close to erank of 1, which ruins the 3d reconstruction because of needle-like gaussians that effectively add noise to the reconstructions. Given that regularization as an add-ons to other methods it improves the 3D reconstruction, as shown in the experiments on DTU dataset. Additionally, it shows in the appendix that the image rendering abilities of the GS is not impaired because of the added regularization.", "questions": "see weaknesses", "limitations": "see weaknesses", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper dives into the attributes of the GS and impressively recognizes an issue relating to the GS ability to reconstruct 3d mesh. The authors exhibit an analysis of the amount of 1-ranked gaussians that strengthen their claim, and showed improved results on DTU dataset.", "weaknesses": "There are several concerns regarding this paper, first it should include more benchmarks for 3d reconstruction, for example Tanks and Temples dataset which was used as benchmark in many other works in the field, including the works that was cited in the paper. Secondly, there are two works that the authors did not mention [1,2] and should be compared to in the DTU experiment. The first [1] has the same idea as 2DGS and was already published. It might be that their regularization solves part of the issue of the GS that this paper presents. The second [2], does not use the gaussian splatting directly for reconstruction and instead renders stereo images for depth estimate. Thus, it does not share the reconstruction error because of the needle-like gaussians. Comparing both of them will strengthen this paper. Lastly, there is no adequate justification for the erank regularization in comparison to other possibilities, e.g. linear, parabolic, or exponential loss on the size of the smallest magnitude scale. Finally, an ablation study showing why erank-regularization is superior compared to other losses is missing. \n[1] Dai, P., et al. \"High-quality surface reconstruction using gaussian surfels.\" arXiv preprint arXiv:2404.17774 (2024).\n[2] Wolf, Y., et al.. \"Surface Reconstruction from Gaussian Splatting via Novel Stereo Views.\" arXiv preprint arXiv:2404.01810 (2024).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer for acknowledging the contribution of our work. We are grateful for the helpful reviews that could strengthen our paper.\n### Tanks and Temples \n>Thank you for the suggestion. We conducted the experiments after the submission, and the table below shows a general improvement when our method is used as an add-on module. Also we want to note that, unlike other papers, we needed four times more experiments because we tested our method as an add-on module on four different models: 3DGS, SuGaR, 2DGS, and GOF. For a single DTU evaluation alone, we ran 15 scenes * 5 repeated experiments per scene * 4 models, totaling 300 training and evaluation runs, which took an extremely long time.\n\n| Methods | 3DGS | 3DGS+e | GOF | GOF+e |\n|:--------:|:--------:|:--------:|:--------:|:--------:|\n| Barn |   0.13 |   0.32 | 0.51 | 0.61 |\n| Caterpillar |  0.08 |  0.19 | 0.41 | 0.42 |\n| Courthouse |  0.09 |  0.13 | 0.28 | 0.28 |\n| Ignatius |  0.04 |  0.37 | 0.68 | 0.74 |\n| Meetingroom |  0.01 |  0.15 | 0.28 | 0.30 |\n| Truck |  0.19 |  0.24 | 0.59 | 0.59 |\n\n\n### Related Works \n>Thank you for suggesting relevant papers. We appreciate the recommendations to strengthen our paper. We were aware of these papers and are also curious about the results. However, we believe these works are concurrent, and not comparing them should not be considered a weakness. Notably, [2] did not have code released at the time of our submission (the code was recently released, during the rebuttal period), and it was not published when we submitted to NeurIPS (ECCV24's final decision was in July). Therefore, it was not possible to compare it with our work. Similarly, [1] was published on arXiv on April 30 (initial version on April 27), and our final draft for NeurIPS was completed around the first week of May. Still, we believe we have shown our efforts to track and include the latest papers, GOF was on arXiv on April 16, and 2DGS on March 26. We are willing to conduct more experiments and share results during the discussion period (if requested, following the rules of discussion period), and the rest in the camera-ready version of our paper.\n### Justification of Using Effective Rank Regularization\n>- Interpretability: Effective rank provides interpretable and meaningful numbers for regularization. Using other variants (linear, exponential) would yield different values without explicit meaning. We know that Gaussians with erank(G)=2 is disk-like, erank(G)$\\approx$1 is needle-like, and we aim to eliminate non-disk-like Gaussians (erank(G) > 2) or needle-like Gaussians (erank(G) $\\approx$1).\n> - Comprehensive Regularization: Our regularization considers all three axes. Some works, including PhysGaussian, have tried regularizing Gaussian primitives but do not focus on all three axes. PhysGaussian considers two axes (max and min scale axes) to reduce spiky Gaussians, but not enforcing disk-like Gaussians. The attached PDF Fig.3(b) shows the erank histogram of PhysGaussian, where it reduces spiky Gaussians (erank(G) $\\approx$ 1) but does not enforce disk-like Gaussians (erank(G) = 2) or penalize Gaussians with erank(G) > 2. Methods like GaussianShader only consider the axis with the minimum scale, minimizing it to make the Gaussian primitive “flat”. 2DGS uses 2D surfel as a primitive, achieving similar effects but not handling spiky Gaussians. Our method considers all three axes, enforcing disk-like Gaussians without needle-like ones, using effective rank.\n>- Elegant Loss Definition: Effective rank of Gaussian covariance enables a single scalar term representation of the Gaussian geometry, which is an elegant way to define loss, avoiding the need to consider all three combinations (s1-s2,s2-s3,s1-s3) of three axes. Additionally, logarithmic loss is known for its stability in optimization problems (we emprically prove this below).\n\n### Comparison & Ablation Study\n>We conducted an ablation study on different variants as per your suggestion. We denote s1, s2, s3 as the largest, second largest, and smallest scale.\n>- vs. PhysGaussian (minimize s1 / s3)\n>- vs. linear (minimize s1 / s2, minimize s3)\n\n| DTU scan | 24 | 37 | 40 | 55 | 63 | 65 | 69\n|:--------:|:--------:|:--------:|:--------:|:--------:|:--------:|:--------:|:--------:|\n| 3DGS |   2.14 |   1.53 | 2.08 | 1.68 | 3.49|2.21|1.43|\n| 3DGS+e |   0.85 |  0.77 | 0.88 | 0.51 | 1.21|1.45|0.96|\n| PhysGaussian |   0.87 |   0.81 | 0.86 | 1.36 | 2.99|1.97|1.46|\n| 3DGS+linear | 0.89   | 0.80   |0.91  | 1.21 | 2.03| 1.84| 1.44|\n\n\n>The results (DTU scan, Chamfer distance $\\downarrow$) show that our method is superior to other variants, empirically proving our justification.\n\n\n\nOnce again, we appreciate the reviewer for taking the time to read through the paper and our response. Your reviews are immensely helpful in strengthening our work. We will add more results (including the supplemental video) during the discussion period.\nThank you.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank the reviewers for acknowledging the effectiveness of our work and highlighting the importance of the task. We are also grateful to all the reviewers for taking the time to read through the paper and providing detailed feedback. Your reviews are immensely helpful in strengthening our work.

We have responded to each review individually and use this general rebuttal space to share more details about the cause of needle-like Gaussians, as requested by reviewers #1 and #2.

While investigating and analyzing the cause of spiky Gaussians is not the main contribution of our work (thus it is included in the appendix), we do agree that further analysis on this aspect is interesting and could potentially strengthen our work. More discussion on this is provided in our response to review #2.

We suggested three reasons for the causes of needle-like Gaussians in the paper: 1. Dilation, 2. Densification trigger along the longer axis, and 3. unadjusted scale after densification.

### (1) Mip-Splatting and Dilation Operation

> As requested by reviewer #2, we first present the Mip-Splatting (MipGaussian) erank histogram in the attached PDF, Fig.3 (a), along with the Chamfer Distance metric (table below). To clarify, we did not claim that the dilation operation solely causes needle-like Gaussians. The dilation operation, combined with the implicit shrinkage bias (cite) of 3DGS, can cause the scale of some axes to be small. Densification trigger issue (2) causes needle-like Gaussians, and dilation can further boost this phenomenon by making the smallest scale even smaller.

>Mip-Splatting focuses only on Gaussians smaller than the pixel size, which has a negligible impact on effective rank measurement and geometry reconstruction. For instance, given two needle-like Gaussians with 10x scale differences: scales (1,0.01,0.01) and (1, 0.001, 0.001), the eranks are 1.002 and 1.000, respectively, indicating a small difference.

>Moreover, Mip-Splatting does not constrain Gaussians to be flat (disk-like), which is crucial for geometry reconstruction. The results show that Mip-Splatting produces a similar effective rank distribution as the original 3DGS and does not improve geometry reconstruction. Thus, while dilation may contribute to the formation of needle-like Gaussians, it is not the main reason, and its impact is small.

| DTU scan | 24 | 37 | 40 | 55 | 63 | 65 | 69
|:--------:|:--------:|:--------:|:--------:|:--------:|:--------:|:--------:|:--------:|
| 3DGS |   2.14 |   1.53 | 2.08 | 1.68 | 3.49|2.21|1.43|
| 3DGS+e |   0.85 |  0.77 | 0.88 | 0.51 | 1.21|1.45|0.96|
| Mip-Splatting |   2.45 | 2.21 | 1.66 | 1.51 | 3.28 |2.32| 1.46|


### (2) Densification Trigger
>Densification issue 2 is straightforward. The densification trigger is the norm of $\sum_{i \in \mathcal{P}} \frac{\partial L}{\partial p_i} \frac{\partial p_i}{\partial u}$ larger than the densification threshold, and $\frac{\partial p_i}{\partial u}$ is small when u moves in the direction of the longest axis (Gaussians have small gradients along the longer axis). Thus, densification along the longer axis is difficult to trigger.


### (3) Scale After Densification
> When a Gaussian is split along the longest axis, instead of halving the largest scale, it is kept the same (scale is copied), resulting in two spiky Gaussians.

### Experiments
>We conducted toy experiments to empirically demonstrate that:

>Densification along the longer axis is less preferred, and Gaussians tend to elongate, even when initialized with multiple Gaussians.

>Fig. 1 and 2 in the PDF show a 2D toy setting of Gaussian splatting. The first row indicates the target image (left) and the initial Gaussian(s) (right). The second row shows the fitted Gaussian(s) and the absolute difference between the target and the fitted.

>Fig. 1 (a) in PDF suggests that Gaussians are not properly densified when they should densify along the longer axis. We also numerically observe that dL/du is very small in this case, hindering densification along the longer axis compared to (b). Lowering the densification threshold might be suggested, but this increases densification in all directions, which does not reduce the spikes. Also in practice, it is not feasible because the number of Gaussians would explode even with a small decrease in the threshold. For example, as presented in Fig. 3 (c) of the PDF, lowering the threshold from 0.0002 to 0.0001 results in twice as many Gaussians, increasing training or GPU memory requirements and training time.

>Similarly, Figure 2 (a) and (b) of the PDF suggest that Gaussians tend to elongate instead of splitting, even when initialized with multiple Gaussians.

>Our method addresses issue #2 by limiting the anisotropic Gaussians, but tackling issue #3 and modifying the scale after splitting would be an interesting future work.

Meta-review: This paper was reviewed by four experts in the field.  Based on the reviewers' feedback, the decision is to recommend the paper for acceptance.  The reviewers did raise some valuable concerns that should be addressed in the final camera-ready version of the paper. The authors are encouraged to make the necessary changes to the best of their ability.    We congratulate the authors on the acceptance of their paper!
Decision: Accept (poster)