AbsGS: Recovering fine details in 3D Gaussian Splatting

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-07 16:39:46

Abstract: Recent advances on neural rendering have shown photo-realistic results in novel view synthesis. As one of the most promising method, 3D Gaussian Splatting (3D-GS) couple 3D Gaussian primitive with differentiable rasterization to obtain high-fidelity 3D scene reconstruction and achieve real-time rendering. The exceptional performance of 3D-GS is attributed to the carefully designed adpative density control strategy, which progressively populate empty areas by splitting/cloning more Gaussians throughout the optimization process. While 3D-GS offers significant advantages, it frequently suffer from over-reconstruction issue in intricate scenes containing high-frequency details,  consequently leading to blur. This issue's underlying causes have still been under-explored. In this work, we present an comprehensive analysis of the cause of aforementioned artifacts and we call it gradient collision, which prevent large Gaussians that cover small-scale geometry from splitting. To address this issue, we further propose novel homodirectional gradient as the guidance for densification. Our strategy efficiently identifies large Gaussians in over-reconstructed regions, and recovers fine details by splitting. We evaluate our proposed method on various challenging datasets, and our approach achieves best rendering quality with reduced memory consumption and yields better distributions of 3D Gaussians in world space. Our method is also easy to implement with just few lines of codes and can be incorporated into a wide variety other Gaussian Splatting-based methods. We will open source our codes upon formal publication.

Main paper: 
 introduction: 
 High quality novel view synthesis from multiple unordered images is a long-standing problem for 3D vision researchers. Recent advances on neural rendering have revolutionized this task by learning a neural implicit representation instead of explicit point clouds or meshes. One of the most effective approach with in this paradigm has been reconstructing a set of 3D Gaussian primitives of the scene [17] . Coupled with splat-based rasterization, 3D Gaussian Splatting (3D-GS) [17] produces compelling real-time rendering results with unprecedented fidelity. The remarkable performance of 3D-GS is closely tied to the adaptive density control strategy. Initialized solely from a set of sparse point clouds derived from Structure from Motion (SfM), 3D-GS gradually populate empty areas by split/cloning existing Gaussians, ultimately covering whole scenes with compact and precise representation. There have been many interests on extending 3D-GS to other applications, e.g., dynamic modeling [9, 29, 37, 42] , single-view or text-to-view generation [30, 39, 43, 44] , mesh extraction [12, 15, 32] , SLAM [8, 14, 40] and so on.However, applying 3D Gaussian Splatting to complex scenes encounters the issue of over-reconstruction, where regions containing high frequency details are covered by only a small number of large Gaussians. Consequently, the rendering results become blurry and 𝑖 to 2D coordinate 𝜇 𝑖 in pixel-space. The number of covered pixels by Gaussian 𝐺 𝑖 is 𝑚. (b) By backpropagating, the view-space gradient 𝑔 𝑖 of Gaussian 𝐺 𝑖 under viewpoint 𝑘 is caculated as the sum of all view-space gradients of pixels that are covered by 𝐺 𝑖 . Since the gradients 𝜕𝐿 𝑗 𝜕𝜇 𝑖 have different directions, the overall sum 𝑔 𝑖 will have a small scale, which do not satisfy the gradient threshold for densification. cannot accurately reflect the appearance and geometry of the scene as validated in Fig. 1 . The cause of over-reconstruction is that the adaptive density control strategy cannot effectively identify large Gaussians in over-reconstructed areas and split them to represent details. The deficiency of the strategy has not been well explored.In this paper, we find that the deficiency of the original strategy lies in its failure to consider the negative impact of pixel-wise sub-gradient directions on the identification of large Gaussians in over-reconstructed areas. Specifically, original 3D-GS proposes to use the view-space positional gradient to determine whether if a large Gaussian requires split. We observe that for each Gaussian primitive the pixel-wise sub-gradients of the view-space positional gradient may have different directions. Therefore, the sub-gradients cancel each other out during the process of summation, namely gradient collision. Especially for large Gaussians covering many pixels, maintaining consistent gradient directions for each pixel becomes exceptionally challenging which results in a small-scale view-space positional gradient. Consequently, the magnitude of view-space positional gradient fails to surpass the densification threshold, thereby hindering the split of over-reconstructed Gaussians.Based on the above analysis, we propose homodirectional viewspace positional gradient as criteria for densification. Homodirectional view-space positional gradient is designed as the sum of the absolute values of pixel-wise sub-gradients covered by a Gaussian primitive, based on the rationale that the representation quality is solely dependent on the magnitude of the gradient, irrespective of its direction. The absolute operation can mitigate the influence of gradient direction while retaining the influence of gradient magnitude. The homodirectional view-space positional gradient therefore avoids the gradient collision and facilitates the split of large Gaussians in over-reconstructed regions that were unrecognized by original strategy. An overview of our method (dubbed as AbsGS) is shown in Fig. 2 .We evaluate AbsGS on previously published real-world datasets. The experiment results show that our method consistently yields high quality novel view synthesis and exhibits better results on PSNR, SSIM, and LPIPS. At the same time, our method keeps similar or less memory consumption compared with 3D-GS. From the visualization of Gaussian ellipsoids, we observe that our method eliminates over-reconstruction areas and recovers fine details while 3D-GS fails and leads to blur, as illustrated in Fig. 1 . In summary, our contributions are as follows:• First, we analyze the deficiency of the original strategy that results in over-reconstruction is caused by gradient collision. • Second, a straightforward yet effective strategy is proposed to utilize homodirectional view-space positional gradient as guidance for densification throughout training. • Third, our proposed method can effectively eliminate large Gaussians in over-reconstructed regions, and achieves better novel view synthesis quality with similar or less memory consumption. 
related works: 
 Neural Implicit 3D Representation. In contrast to widely adopted classic explicit 3D representaiton, e.g., point cloud, voxels and mesh, more recent learning-based neural implicit representations do not require complex regularization, attract more attention and achieve more accurate rendering results. As a revolutionary pioneer, Neural Radiance Fields (NeRF) [20] couples differentiable ray-marching with continuous radiance field to enable end-to-end optimization from images. NeRF has been broadly receiving massive interest towards more photo-realistic novel view synthesis [1, 2, 4] and its follow-up methods have been providing impressive results to other applications, e.g, dynamic modeling [11, 24, 25] , surface reconstruction [35, 36, 38, 41] , 3D asset generation [16, 26, 34] . However, with expensive volumetric ray-marching which densely sample point locations along the camera rays, NeRF is inefficient for training at the beginning of design. Though notable NeRF-variants [5, 10, 21, 27] have been proposed to alleviate the training/inference computation burden by introducing spatial datastructures to store neural features instead of large MLPs, the plenty of sampling and queries can not be avoided due to the inherit requirement of volumetric rendering. Instead, point based representations with learnable attributes support more efficient forward rasterization for realtime rendering. More recently, 3D Gaussian Splatting (3D-GS) [17] , revisit fast point-based rendering engine with learnable Gaussian primitives. Starting from initial sparse point clouds from Structure from Motion (SfM) [23, 28, 33] , the optimization procedure moves the Gaussians to correct positions, creates new Gaussians to cover empty space, removes invalid Gaussians and finally produces a set of Gasussions to precisely represent underlying scenes. As an unstructured and discrete representation that supports forward rasterzation, it fundamentally avoid the shortcomings of expensive sampling and queries and provide real-time rendering performance, along with high quality for novel-view synthesis. Nowadays there have been many subsequent extensions of 3D Gaussian splatting, e.g, surface reconstruction [6, 12, 15] , generation [7, 22, 31, 39] and dynamic modeling [19, 37] . 
method: 
 In this section, we first review the basic background of 3D-GS in Section 3.1; then, we describe the gradient collision phenomenon that prevent large Gaussions in over-reconstructed regions from splitting in Section 3.2; finally, we propose the homodirectional gradient as guidance for splitting and present the details in Section 3.3.3D Gaussian Splatting (3D-GS) [17] proposes to represent scenes by a set of learnable 3D Gaussians 𝐺 0 , 𝐺 1 , ..., 𝐺 𝑁 . Each 3D Gaussian primitive 𝐺 𝑖 is explicitly parameterized via center position 𝜇 3𝑑 𝑖 and full 3D covariance matrix Σ 3𝑑 𝑖 :EQUATION3D Gaussian primitive also has two additional learnable attributes: opacity 𝑜 𝑖 and spherical harmonics coefficients 𝑆𝐻 𝑖 to model view dependent color. To render an image, the 3D Gaussian primitive 𝐺 𝑖 is first transformed into the camera coordinate and projected onto image plane, resulting the 2D Gaussian 𝐺 2𝑑 𝑖 with center position 𝜇 𝑖 and 2D covariance matrix Σ 2𝑑 𝑖 : then differentiable alpha blending is employed to integrate colors from front-to-back:EQUATIONEQUATIONEQUATIONwhere 𝜎 (•) is the sigmoid function and N is the number of Gaussians that participate in alpha blending. 3D-GS initializes 3D Gaussians with the free sparse point clouds produced from SfM [28] , and then applies adaptive density control to populate empty areas. There are two forms of densitification: split and clone. Split operation is designed to split large Gaussians that represent smallscale areas in two, which corresponds to over-reconstruction. Clone operation aims at clone more Gaussians to sufficiently cover underreconstruction region For each Gaussian 𝐺 𝑖 , 3D-GS uses the average magnitude of view-space positional gradients to determine whether to apply densification. Specifically, for Gaussian 𝐺 𝑖 which have the pixel-space projection point 𝜇 𝑘 𝑖 = (𝜇 𝑘 𝑖,𝑥 , 𝜇 𝑘 𝑖,𝑦 ) under viewpoint 𝑘 and corresponding loss 𝐿 𝑘 , the average view-space positional gradient ∇ 𝜇 𝑖 𝐿 is calculated every 100 training iterations as follows:EQUATION)where 𝑀 is the total number of viewpoints that Gaussian 𝐺 𝑖 participates in calculation during 100 iterations. Split for 𝐺 𝑖 is performed when it satisfies:EQUATIONwhere 𝜏 𝑝 is the the gradient threshold (default 0.0002) and 𝜏 𝑆 is the scale threshold (default 0.01). (e) Select by AbsGS Figure 4 : An example to demonstrate the difference between densification strategy of 3D-GS and AbsGS. From (b) and (c), we observe that large-scale Gaussians are used to represent cement ground, which contains fine details and indeed should be represented by many small-scale Gaussians. In (d) and (e), we show the Gaussians that satisfy densification criteria of 3D-GS and ours respectively, where identified Gaussians' colors are set to white. When using 𝑔 𝑖 as 3D-GS, the large Gaussians that represent cement ground are not identified while our selection strategy based on ĝ𝑖 can find those Gaussians.3D-GS relies on gradient descent to optimize the scene, so the magnitude of the gradient can reflect the quality of representation. However, the computation of ∇ 𝜇 𝑖 𝐿 includes gradient directions that are irrelevant to the representation state, weakening the effectiveness of the gradient magnitude. In this section, we analyze the negative impact of gradient directions. Specifically, there exists gradient collision in the calculation of and 𝜕𝐿 𝜕𝜇 𝑖,𝑦 respectively. Take the x-axis gradient 𝑔 𝑖,𝑥 as an example. We further decompose this gradient into the sum of multiple sub-gradients contributed by each pixel:EQUATIONwhere 𝑚 is the number of pixels covered by 𝐺 𝑖 , 𝐿 𝑗 is the loss computed by 𝑗-th pixel. Our key observation is that the per-pixel gradients 𝜕𝐿 𝑗 𝜕𝜇 𝑖,𝑥 may have different directions. The proof is as follows.Proof. We simplify the proving goal to demonstrating that the signs of gradients may differ, which is equivalent to proving that the gradient directions are different. Per-pixel gradient 𝜕𝐿 𝑗 𝜕𝜇 𝑖,𝑥 can be calculated as:EQUATIONThe sign of 𝜕𝐿 𝑗 𝜕𝜇 𝑖,𝑥 is determined by the multiplication of signs of three individual terms.For the first term, since 𝐿 𝑗 is L 1 loss, the sign of first term𝜕𝐿 𝑗 𝜕𝑐 𝑗 𝑙depends on the comparison result of rendered and real RGB values.For the second term𝜕𝑐 𝑗 𝑙𝜕𝛼 𝑖 , according to Equ.3, it can be further calculated as follows:EQUATIONEQUATIONThus, the two additions to Equ.9 have opposite signs, and the sign of𝜕𝑐 𝑗 𝑙𝜕𝛼 𝑖 is uncertain. Here 𝑐 𝑖 and 𝑐 𝑝 are Gaussian colors, 𝑐 𝑗 𝑙 is the color of the pixel.For the third term 𝜕𝛼 𝑖 𝜕𝜇 𝑖,𝑥 , according to Equ.4, it is calculated as:EQUATION)where p 𝑗 is the coordinate of 𝑗-th pixel. Therefore, the sign of the third term 𝜕𝛼 𝑖 𝜕𝜇 𝑖,𝑥 is determined by the relative x-axis coordinate difference (𝜇 𝑖,𝑥 -p 𝑗 ).Overall, the sign of first term is determined by to the rendered pixel value, the sign of second term is related all the Gaussians's that participate in calculation, and the sign of third term is related to the projection point, so these three terms do not always have the same sign, and the per-pixel gradient 𝜕𝐿 𝑗 𝜕𝜇 𝑖,𝑥 may have different directions for different pixels.We design a simple experiment to verify the above analysis, as illustrated in 3. An image is randomly selected from the 𝑓 𝑙𝑜𝑤𝑒𝑟𝑠 scene from Mip-NeRF360 [3] and is reduced to a resolution of 100×65 in Fig. 3 (a) . Then we optimize only one Gaussian to fit this image, and the final rendering result is shown in Fig. 3 (b ). The overall L1 loss is large and this Gaussian leads to a typical over-reconstruction issue, in Fig. 3 (c ). we show the x-axis gradient direction 𝜕𝐿 𝑗 𝜕𝜇 𝑖,𝑥 in Fig. 3 (d) , where the pixel color means the x-axis direction of the gradient contributed by this pixel, and red is for positive x-axis direction and green for negative x-axis direction. The different directions of per-pixel gradient 𝜕𝐿 𝑗 𝜕𝜇 𝑖,𝑥 results the sum ∇ 𝜇 𝑖 𝐿 may have a small-scale magnitude . Especially for large Gaussians that covering many pixels, maintaining consistent gradient directions for each pixel becomes exceptionally challenging. Consequently, Table 1 : Quantitative results on Mip-NeRF 360 [1] , Tanks & Temples [18] , and Deep Blending [13] . All scores of the rest of the baselines are directly sourced from the original 3D-GS paper [17] to make a fair comparison. INGP-Base and INGP-Big are Instant-NGP versions with default settings and increased network size respectively. The 1st, 2nd, and 3rd-best performances are indicated by red, orange, and yellow highlights respectively. Based on the above analysis, we design AbsGS, which can accurately reflects the representation state and identify Gaussians in over-reconstructed regions. The overall review of our method is shown in Fig. 2 . AbsGS aims to eliminate gradient collision by erasing the influence of gradient direction while retaining only the influence of gradient magnitude. Specifically, AbsGS computes the homodirectional view-space positional gradient ĝ𝑖 by taking the absolute value of each component before summing: ĝ𝑖 = ( ĝ𝑖,𝑥 , ĝ𝑖,𝑦 ) The absolute operation constrains the gradient directions of all pixels to be in the same direction along the x and y axes, thereby avoiding gradient collision. It uses the magnitudes of the gradient components along the x and y axes to jointly express the state of representation, and finally combines these two components through the 𝐿2 norm. This value directly reflects the expression state of all the pixels covered by the Gaussian, thus accurately identifying Gaussians with subpar expression, such as over-reconstructed Gaussians. Note that the ĝ𝑖 is not used in backpropagation of computation graph, it is an extra variable that only related to densification. Fig. 4 presents a real example to compare the selected Gaussians that need densification between 3D-GS and ours. We first optimize the scene 𝑡𝑟𝑒𝑒ℎ𝑖𝑙𝑙 of Mip-NeRF360 for 7000 iterations using 3D-GS. then we select over-reconstructed Gaussians by original 𝑔 𝑖 and ours homodirectional ĝ𝑖 . The selected Gaussians are highlight with white, as shown in Fig. 4 (d) and (e) . It shows that 3D-GS selects 69,269 Gaussians for densification but it missed most of the large Gaussians in over-reconstructed areas. In contrast, our proposed AbsGS only select 16,488 Gaussians while covering the majority of over-reconstructed areas. This experiment effectively demonstrates that our method is better suited as the criteria for densification. 
experiments 4.1 setup: 
 Datasets. Following 3D-GS [17] , we select scenes with highly diverse capture styles, spanning from enclosed indoor environments to expansive outdoor settings without clear boundaries. Specifically, we use all 9 unbounded indoor and outdoor scenes presented in Mip-NeRF360 [3] , two scenes from [18] and two scenes provided by [13] , totaling 13 distinct environments.Baselines. Across all dataset, we benchmark our proposed method against multi-level hash grid based Instant-NGP [21] , the current state-of-the-art NeRF-based method Mip-NeRF360, and widely-used 3D GS [17] . To ensure a fair comparison, all numerical data of these methods presented in tables are directly sourced from the original publication [17] unless otherwise specified. In addition, we observe that the scale of some scene details is smaller than the default radius threshold 𝜏 𝑆 that used to control split operation, and thus 3D-GS is hindered in sufficient split for these regions. To further improve enhance the ability of 3D-GS. we reduced the 𝜏 𝑆 from default 0.01 to 0.001 and retrain 3D-GS, which is denoted as 3D-GS* in following experiments. Our method AbsGS is also trained with 𝜏 𝑆 set as 0.001. Since 𝜏 𝑝 is a hyper-parameter that impact the behavior of AbsGS, we present the results when 𝜏 𝑝 is set to different values. Specifically, AbsGS-0004 and AbsGS-0008 represent the results when 𝜏 𝑝 is set to 0.0004 and 0.0008 respectively. Implementation Details. Our experiments is measured on a single NVIDIA V100 GPU with 32GB memory. Following common practice, we stop the Gaussian densification after 15k iterations and stop training at 30k iterations. We report the novel view synthesis metrics PSNR, SSIM, and LPIPS, and we also show the number of Gaussians and the memory used to store the parameters of optimized Gaussians to demonstrate the trade-off between efficiency and memory. To highlight the effectiveness of AbsGS in addressing over-reconstruction, homodirectional gradients are only used as guidance for split operation while clone operation follows the original strategy of 3D-GS directly. We select a larger gradient threshold 𝜏 𝑝 for split operation since AbsGS will increase the magnitude of ∇ 𝜇 𝑖 𝐿 than that in 3D-GS.Figure 5 : Qualitative comparisons of different methods on scenes from Mip-NeRF360 [3] and Tanks&Temples [18] datasets. The rendering result of 3D Gaussian Splatting is blurry at regions containing high-frequency details. Our AbsGS yields significantly better rendering quality with sharper details.Quantitative Results. We report the quantitative results in Table. 1. Both AbsGS-0004 and AbsGS-0008 outperform other baselines in most cases. It's noteworthy that our method consistently yields better result in terms of SSIM and LPIPS metrics, which capture more reliable human perception differences in images than PSNR metric. Additionally, it can be observed from the memory consumption that the effectiveness of AbsGS does not stem from an increased number of Gaussians. Compared to original 3D-GS, AbsGS-0008 only maintains roughly half the memory consumption and AbsGS-0004 also consistently maintains lower memory, while the quantitative result of them remain highly competitive.Qualitative Results. First, we show novel view synthesis results in Fig. 5 . The results indicate that our method significantly reduces the rendering blurring phenomenon and improves rendering quality across all scenes, such as weeds under the bench and uneven concrete ground. Second, to demonstrate the effectiveness of our method in eliminating large Gaussians in over-reconstructed regions, we select stump scene and visualize the point clouds and ellipsoids along with the number of Gaussians in Fig. 7 . We can observe that 3D-GS exhibits over-reconstruction in areas with similar colors but rich textures, such as the grassy area, where the point cloud is very sparse and the ellipsoids are excessively large. In contrast, our method effectively utilizes smaller Gaussians for representing those areas. It is worth mentioning that the number of Gaussians of AbsGS is less than that of 3D-GS, with Abs-0008 even being less than half of 3D-GS. This indicates that our method does not rely on more Gaussians to solve the problem of overreconstruction. Additionally, the scale threshold for Abs-0008* is set to 0.01 as same as 3D-GS, to demonstrate that the effectiveness of our method does not depend on adjusting the hyperparameter. We provide more qualitative comparisons in the supplementary material.Ablation Studies. In this section, we conduct ablation experiments to study the impact of scale threshold 𝜏 𝑆 and gradient threshold 𝜏 𝑝 on our method. We present quantitative results in in Tab.2. The experimental results demonstrate that lowering any of 𝜏 𝑝 and 𝜏 𝑆 improves rendering quality at the cost of memory. Additionally, to qualitatively illustrate the impact of 𝜏 𝑆 on rendering quality, we show rendering result in Fig. 8 . The visualization demonstrates that a smaller 𝜏 𝑆 helps to mitigate large Gaussians in over-reconstructed regions. The reason is that the scale of some scene details is smaller than 𝜏 𝑆 ; consequently, the Gaussions on those regions that have larger radius than 𝜏 𝑆 will not split and thus lead to blurry rendering.Why not solve the over-reconstruction problem by simply lowering the threshold 𝜏 𝑝 ? Since over-reconstruction issue is caused by gradient collision that make it difficult for the gradient of large Gaussions to exceed threshold 𝜏 𝑝 , a straightforward solution is lowering threshold 𝜏 𝑝 to identify more large Gaussians in over-reconstructed regions. However, we have observed that this solution doesn't perform well in practice and leads to significant memory consumption. Fig. 9 (a) shows the LPIPS and memory consumption of 3D-GS The gradually increased memory requirements make the simple solution impractical for real-world applications. Additionally, we show rendering result when 𝜏 𝑝 is set to 0.0001 in Fig. 9 (b). It's observed that the over-reconstruction issue is still evident even at the cost of 2.47GB memory. Further lowering the threshold below 0.0001 result in out of CUDA memory error when training a single V100 GPU. Above analysis reveals that simply lowering gradient threshold 𝜏 𝑝 is impractical to eliminate over-reconstruction.Why does addressing over reconstruction require a large amount of memory for 3D-GS while AbsGS does not? As illustrated above, directly reducing the threshold for 3D-GS 𝜏 𝑝 is highly inefficient and impractical while our method manages to solve over-reconstruction. Next, we uncover the reasons behind it by visualizing selected Gaussians for split. In Fig. 6 (a), we train the 𝑏𝑖𝑐𝑦𝑐𝑙𝑒 scene with 3D-GS using default parameters for 3000 steps. The rendering image and ellipsoid image reveal that there are numerous over-reconstructed areas in the scene, such as lawn and trees. Fig. 6 (b) and (c) respectively show the Gaussians selected under different gradient thresholds for 3D-GS and AbsGS, with the number of selected Gaussians inset. In the case of 3D-GS with the default threshold of 0.0002, although it selected 77969 Gaussians, it did not effectively encompass over-reconstructed region. In contrast, AbsGS can select most of the over-reconstructed region by selecting only 37390 Gaussians at the threshold of 0.001. Besides this, lowering the threshold indeed allows for the selection of more over-reconstructed areas for 3D-GS, but the number of selected Gaussians increases rapidly, and the rate of noise in all selected Gaussians is also apparent, resulting in large memory cost. In particular, see Fig. 6 (b) and Fig. 6 (c), it can be observed that when we lower the threshold, 3D-GS selects many Gaussians that do not need split but still can not select large Gaussians in over-reconstructed regions. Above analysis demonstrates that the homodirectional gradient used by AbsGS offers a much more accurate reflection of the representation quality, leading to fewer mistakenly selected Gaussians. This fundamental distinction proves why AbsGS is more efficient than 3D-GS. 
conclusion: 
 3D-GS has made significant strides in novel view synthesis tasks, but it frequently encounters issues such as blurriness and loss of detail stemming from over-reconstruction. This paper delves into the phenomenon of over-reconstruction and identifies gradient collision in 3D-GS's adaptive density control strategy as a primary cause. Specifically, the view-space positional gradient used in this strategy is the sum of sub-gradients of all pixels covered by a Gaussian, and these gradient directions cannot always remain consistent, leading to mutual cancellation. To tackle this challenge, our proposed AbsGS utilizes homodirectional view-space positional gradient by taking the absolute values of the x and y components of pixel-wise sub-gradients separately, eliminating the influence of direction. Extensive experiments conducted on multiple datasets against 3D-GS demonstrate the significant advantages of our method in eliminating over-reconstruction and restoring fine details. Moreover, our approach boasts lower memory consumption compared to 3D-GS, largely attributable to AbsGS's more accurate identification of over-reconstruction through the use of homodirectional gradients. 
 
 Appendix:  


Decision: This paper presents a comprehensive analysis of the cause of aforementioned artifacts called as gradient collision, which prevent large Gaussians that cover small-scale geometry from splitting. The proposed method is effective and new. All reviewers have positive final ratings, are satisfied with the response, and recommend accepting the paper. I agree with their recommendation. Thanks for the authors' effort and rebuttal.

Reviews: review: paper_summary: paper_summary: The paper propose an important densifycation strategy for 3DGS(3D Gaussian Splatting) to recover the details of the scene. The paper provide detail analysis of the comparision between the previous densification strategy and proposed densification strategy. The experiments demonstrated that the efficiency of the proposed method.
strength_weakness: strength_weakness: * The paper explored an important problem and propose a novel idea to solve the problem. The details of the reconstructed scene can be successfully recoverd using the proposed densification strategy. 

* The paper writing is clear and easy to follow.
limitations: limitations: There are some points in the paper are confusing and should be solved from the authors feedbacks.

* I wonder whether the proposed methods can not only limit to the 3D ellipsoid representation(3DGS)? Will the proposed densification strategy still be useful for 2D-surfel representation (eg. 2D Gaussian Splatting for Geometrically Accurate Radiance Fields, Huang et. al
)? I suggest the authors to provide some experiments and discuss about that. Since 2DGS is a new representation that have the trend to replace the 3DGS. Otherwise, the paper looks really like an incremental on 3DGS. If the proposed methods can still be used for different representation. I believe it will make the paper more fundamental.

* The memory usage of AbsGS. From the Figure 4(d,e). I notice that the points marked as densification points of the AbsGS is much less than original 3DGS(69269 vs 16448). However, I do not see significant decrease of the memory usage of the proposed methods (AbsGS-004 vs 3D-GS in table 1). I need some clarifications about that.

* Why the memory usage of AbsGS performs much different across 3 different dataset.  I notice that in DeepBlending dataset, the proposed methods decrease the memory usage most significantly while other two datasets do not reveal this. I wonder what is the reason about that.

* The paper miss some experiments on the NeRF-synthetic dataset,  does this mean that on synthetic dataset, absGS do not have significant improvement compared with 3DGS. I understand that the object in NeRF-synthetic dataset is much easier to reconstruct.
paper_summary: paper_summary: This paper introduces a novel densification scheme for 3D Gaussian Splatting, utilizing homodirectional gradients (taking absolute value) as guidance. The proposed method enhances rendering quality and achieves a more reasonable distribution of Gaussians, particularly in regions rich in details.
strength_weakness: strength_weakness: 1. This paper focuses on an important issue of blurriness in certain areas of 3D Gaussian Splatting (3DGS), which I also noticed when I first used 3DGS.
2. The solution presented in this paper is simple and effective (I happened to try it before reviewing the paper). I appreciate that the code of AbsGS is open source.
limitations: limitations: Some confusing points: 
1) Why do the authors describe the phenomenon where "regions containing high-frequency details are covered by only a small number of large Gaussians" as "suffering from over-reconstruction"? Typically, over-reconstruction implies the use of too many primitives. 
2) In Line 454, the authors mention that the "over-reconstruction" phenomenon is shown in Fig. 3(c). However, it is unclear how Fig. 3(c) was derived.
3) The number of Gaussians displayed in Fig. 1 is not consistent with the memory usage reported in Table 1. The proposed method in Fig.1 uses more gaussians while the average memory of each dataset in Table 1 is less than 3DGS. I understand that the results shown in Fig. 1 are calculated for a single scene, but I am curious whether this discrepancy is due to random variation or if there are other underlying reasons.
4) Fig.9 (a) shows the change of LPIPS as the threshold changes. What about the PSNR and SSIM?

Some typos, such as:
1) "Gaussion" in the caption of Fig.1.
paper_summary: paper_summary: This work proposes novel homodirectional gradient as the guidance for densification. The strategy efficiently identifies large Gaussians in over-reconstructed regions, and recovers fine details by splitting.
strength_weakness: strength_weakness: The work was novel and the experiment valid. The visual proof is sufficient to show that the method is effective for the problem.
limitations: limitations: 1.the relatively low improvement of this method is not enough to affect the contribution of this paper, and it is suggested that the motivation of gradient conflict should be further elaborated.
2. It is highly recommended to validate this generic approach against other Gs-based methods, such as Scaffold GS
paper_summary: paper_summary: This paper proposes a novel densification strategy for 3D Gaussian splatting. Specifically, this paper proves that 3DGS fails to identify over-reconstructed regions because of the gradient collision incurred by its densification strategy. Then, this paper proposes to erase the effect of gradient direction to address this issue. Experiments results demonstrate that the proposed densification strategy can effectively remove artifacts caused by wrong identification of over-reconstructed regions.
strength_weakness: strength_weakness: 1. The analysis of gradient collision is technically correct and easy to follow.
2. The proposed homodirectional gradient is simple yet effective.
3. The experimental results show that the proposed method can improve the effectiveness of densification, and thus, improve the rendering quality of over-reconstructed regions.
4. The paper is well-written and easy to follow.
limitations: limitations: 1. It would be better to add the rendered image of the proposed method in Fig. 6 for comparison.
2. It would be better to discuss whether the same issues exist in the densification of under-constructed regions.
3. “as illustrated in 3.” in line 450, page 4 should be corrected to “as illustrated in Fig. 3”.