3D Gaussian Rendering Can Be Sparser: Efficient Rendering via Learned Fragment Pruning
Authors: Zhifan Ye
Chenxi Wan
Chaojian Li
Jihoon Hong
Sixu Li
Leshu Li
Yongan Zhang
Yingyan Celine Lin
Venue: NeurIPS 2024 poster (2024)
Keywords: Computer Vision
Efficient Machine Learning
Neural Graphics
Primary area: machine_vision

Abstract: 3D Gaussian splatting has recently emerged as a promising technique for novel view synthesis from sparse image sets, yet comes at the cost of requiring millions of 3D Gaussian primitives to reconstruct each 3D scene. This largely limits its application to resource-constrained devices and applications.
Despite advances in Gaussian pruning techniques that aim to remove individual 3D Gaussian primitives, the significant reduction in primitives often fails to translate into commensurate increases in rendering speed, impeding efficiency and practical deployment. We identify that this discrepancy arises due to the overlooked impact of fragment count per Gaussian (i.e., the number of pixels each Gaussian is projected onto). To bridge this gap and meet the growing demands for efficient on-device 3D Gaussian rendering, we propose fragment pruning, an orthogonal enhancement to existing pruning methods that can significantly accelerate rendering by selectively pruning fragments within each Gaussian. Our pruning framework dynamically optimizes the pruning threshold for each Gaussian, markedly improving rendering speed and quality. Extensive experiments in both static and dynamic scenes validate the effectiveness of our approach. For instance, by integrating our fragment pruning technique with state-of-the-art Gaussian pruning methods, we achieve up to a 1.71$\times$ speedup on an edge GPU device, the Jetson Orin NX, and enhance rendering quality by an average of 0.16 PSNR on the Tanks\&Temples dataset. Our code is available at https://github.com/GATECH-EIC/Fragment-Pruning.

3D Gaussian Rendering Can Be Sparser: Efficient Rendering via Learned Fragment Pruning
Zhifan Ye, Chenxi Wan, Chaojian Li, Jihoon Hong, Sixu Li,
Leshu Li, Yongan Zhang, Yingyan (Celine) Lin Georgia Institute of Technology fzye327, celine. wiwl/inntechheeee
Abstract
3D Gaussian splatting has recently emerged as promising technique for novel view synthesis from sparse image sets, yet comes the cost of requiring millions of 3D Gaussian primitives to reconstruct each 3D scene. This largely limits its application to resource-constraines devices and applications. Despite advances Gaussian pruning techniques that aim to remove individual 3D Gaussian primitives, the significant reduction in primitives often fails to translate into commensurate increases in rendering speed, impeding efficiency and practical deployment. We identify that this discrepancy arises due to the overlooked impact fragment count per Gaussian (i.e., the number of pixels each Gaussian is projected onto). To bridge this gap and meet the growing demands for efficient on-device 3D Gaussian rendering, we propose fragment pruning, orthogonal enhancement existing pruning methods that can significantly accelerate rendering by selectively pruning fragments within each Gaussian. Our pruning framework dynamically optimizes the pruning threshold for each Gaussian, markedly improving rendering speed and quality. Extensive experiments in both static and dynamic scenes validate the effectiveness of our approach. For instance, by integrating our fragment pruning technique with state-of-the-an Gaussian pruning methods, we achieve up to a 1.71 x speedup on an edge GPU device, the Jetson Orin NX, and enhance rendering quality by an average of 0.16 PSNR on the Tanks& Temples dataset. Our code available at https: thub. WAw.///AATCEEIII//IAACA
Pruniig
Introduction
Novel view synthesis, which aims to generate photo-realistic images of a 3D scene from unseen viewpoints given set of posed multi-viww images as input, has been crucial for many virtual reality (VR) and augmented reality (AR) applications 1
In recent years, Neural Radiance Field 7 have shown promise in delivering high-quality rendering. However, the volume rendering approach [ adopted in NeRF requires expensive sampling (e.g., more than 100 times neural network inference per emitted ray (4) in 3D space, leading slow rendering speeds. To mitigate this problem, subsequent works have attempted to enhance rendering performance by: (1) storing radiance in explicit 3D representations such as voxel grids [9] 10 I octrees [2].
(NeRF) and its variants 55 6
and hash grids [3]; and (2) converting pre- pre-trained NeRFs into meshes 14 15 However, these methods still rely on the aforementioned costly volume rendering and often lead to increased storage consumption and decreased rendering fidelity. Recently, 3D Gaussian Splatting [16). which adopts a rasterization-batt rendering pipeline to avoid the costly point sampling required in volume rendering. has emerged as a promising solution for achieving a superior trade-off between rendering speed and quality. Nevertheless, 3D Gaussian Splatting [16] requires millions of 3D Gaussian primitives to reconstruct 3D scene (e.g., an average of 3 million per scene in the Mip-NeRF 360 dataset (6)), which still hinders real-time rendering on resource-constrained devices. For instance, illustrated in
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

25
FPS: 28, #F: 28,172,728
3DGS Ours
80
Image Quality (PSMRR
3DGS
Ours
Ours
23
Mini-Splat
40
Mip-NeRF 360
22-
Plenoxels
INGP-BiE
3DGS
10
FPS: #F: 56,788,756
20
#Fragments Per Pixel
Rendering Speed (FPS)
Figure 1: (a) The number of fragments per pixel and the total number of fragments in a bicycle scene from the Mip-NeRF 360 dataset. The bottom left figure corresponds to the vanilla 3D Gaussian Splatting approach (3DGS) [16] while the upper right figure illustrates that of approach top of 3DGS. (b) The rendering speed in FPS (Frames Per Second) and image quality in PSNR (Peak Signal-
Noise Ratio) of SOTA approaches (including Mip-NeRF 360 [5] INGP-Big
13
Plenoxels (9)
3DGS [6, and Mini-Splatting [8] and our proposed one.
Sec. 6.3 it achieves only 35 FPS on the Mip-NeRF 360 dataset [6] with a an edge GPU device 17
which is below the required speed for real-time rendering (c.g., 60 FPS).
To further enhance the rendering speed of 3D Gaussian Splatting on edge devices, prior works have proposed pruning individual Gaussian primitives. However, as shown in our profiling results in Sec. these prior works do not accelerate the rendering process proportionally to the reduction in the number of primitives, limiting their achievable efficiency. Our further analysis of the rendering process statistics reveals that these existing efforts overlook a critical opportunity, fragmen| pruning, which operates at the fragment, i.e., pixel, granularity. Specifically, we identify that although these methods significantly reduce the number of Gaussian primitives, they do not proportionally reduce the number of rendered fragments, as the number fragments per Gaussian increases. This increase directly impacts computational cost during the bottleneck rasterization stage of 3D Gaussian's 's rendering pipeline. To the best of our knowledge, we are the first to investigate and develop fragment pruning for 3D Gaussian Splatting.
Based on insights from the aforementioned profiling and analysis, we propose an adaptive fragment pruning framework that learns each Gaussian's 's fragment pruning threshold as a post-processing step for pre-trained 3D Gaussian Splatting models. Specifically, we use the sigmoid function to approximate the non-differentiable truncation function of a 3D Gaussian projected onto the 2D pixel grid (i.e., the image plane), enabling a differentiable fine-tuning pipeline. This approach enables each Gaussian primitive to learn an independent truncation threshold, thereby reducing the number of pixels it projects onto and, consequently, decreasing the fragment count per Gaussian. This process significantly enhances the rendering efficiency of the truncated Gaussian function. Fig. Iillustrates the fragment count before and after applying the proposed fragment pruning framework. along with its impact rendering efficiency and image quality. In summary, our proposed fragment pruning framework makes the following contributions:
We profile the 3D Gaussian rendering process before and after applying prior Gaussian pruning techniques and derive the following insights: (1) the rasterization stage of the entire rendering pipeline is the bottleneck in terms of runtime, regardless of whether the Gaussian pruning techniques are applied or not; (2) prior works do not reduce the runtime of the bottleneck rasterization stage proportion to the reduction of the number of Gaussian primitives; (3) this inefficiency arises because prior methods do not proportionally reduce the fragment count, which is key factor determining the computational cost in the bottleneck rasterization stage.
Motivated by the insights described above, we propose fragmeni pruning, which prunes 3D Gaussian primitives at the 2D fragment granularity instead of the raw 3D Gaussian primitive. Specifically, develop adaptive fragment pruning framework by making the truncation

---

function for each Gaussian primitive differentiable when projecting it onto the 2D pixel grid. We then fine-tune the pre-trained 3D Gaussian Splatting model using fully differentiable pipeline, allowing each Gaussian to automatically learn the optimal truncation threshold.
Extensive experiments and ablation studies on both static and dynamic scenes demon- strate the effectiveness of our proposed fragment pruning. For instance, it is orthogonal to prior Gaussian pruning techniques and can further boost the FPS of state-of-the -her (SOTA) Gaussian pruning works by 1.71 x without decreasing the rendering quality on the Tanks&Temples [9] dataset.
2 Related Works
Novel View Synthesis. Novel view synthesis aims to render photorealistic images from novel
viewpoints, given set of sparsely sampled images of a specific scene. Toward this goal, early efforts in this field employed classical computer vision techniques such as structure- ceetuurr-ffro-mmeee from-motionn [20] and multi-view stereo [21]. More recently, the adoption of deep learning methods has significantly
enhanced rendering quality [22 23 24 25). Specifically, NeRF 4 has emerged as a standout among
these deep learning-based novel view synthesis techniques. It models the radiance field of the target scene using implicit neural representations (i.e., multi-layer perceptrons) and achieves unprecedented rendering fidelity. However, NeRF relies on volume rendering B with costly sampling of its implicit neural representation along each emitted ray, resulting in slow rendering speeds. To enhance the rendering speeds of NeRF, subsequent research has incorporated explicit geometric structures or more efficient positional encodings to reduce the capacity of the multi-layer perceptron [12 I [3], or even eliminated neural networks entirely (]. Nevertheless, these approaches still adopt the volume rendering pipeline, and the corresponding rendering speeds are limited by the expensive sampling along each emitted ray.
3D Gaussian Splatting, Unlike the novel view synthesis works mentioned previously that rely on volume rendering, 3D Gaussian Splatting [66 parametrizes a scene using a set of 3D Gaussian primi- tives and utilizes GPU-friendly rasterization-basedd rendering pipeline. This approach circumvents the need for expensive sampling along each emitted ray. In addition to improvements in rendering speed, the use of continuous and anisotropic Gaussian primitives also enables faster convergence and higher reconstruction fidelity compared to the aforementioned NeRF-based methods. Consequently, this approach has inspired numerous follow-up works that extend its application to dynamic scene reconstruction 226 27 28 and 3D content generation 22 30 demonstrating superior performance across various domains.
3D Gaussian Splatting Pruning. Despite the impressive rendering quality and efficiency achieved by 3D Gaussian Splatting [16], high redundancy in the reconstructed 3D Gaussian models adversely affects rendering speeds, as identified by [31 18]. To further boost rendering speeds by compressing this identified redundancy, various techniques have been proposed to prune the redundant Gaussian primitives, drawing parallels to prior deep neural network pruning efforts [32 33 For example, Compact 3D Gaussian
(31 learns binary mask to remove unnecessary Gaussians. LightGaus- sian (34 introduces an importance measure to eliminate insignificant Gaussians below a certain threshold. Mini-Splatting (18) uses importance-basee sampling rather than strict pruning threshold to mitigate excessive culling in local regions. Similarly, (35] proposes redundancy-basse sampling method for Gaussian removal. However, all the aforementioned 3D Gaussian Splatting pruning techniques only address pruning at the granularity of individual Gaussian primitives, neglecting the impact of the number of fragments, produced after projecting 3D Gaussians onto the 2D image plane, on rendering speeds. In contrast, our work explores an orthogonal direction at the granularity of fragments, significantly enhancing the trade-offs between rendering speeds and quality in both static and dynamic scenes, as validated in Sec. 6
NeRF Mesh Convertion In addition to 3D Gaussians, polygon meshes also offer an efficient 3D scene representation with a rendering pipeline that is well-optimized on commercial devices. Previous works [1
15 36 have proposed converting pre-trained NeRFs into opaque polygon meshes to enhance rendering speed in novel view synthesis tasks. After conversion, a pixel's color depends solely on the closest projected polygon, which reduces per-pixel computational costs but makes challenging to faithfully reconstruct complex geometry and translucent objects. Consequently, the
3

---

converted mesh often struggles to achieve high rendering quality in real-world scenes. For instance, on the Mip-NeRF 360 dataset [6], the prior works on converting NeRF to mesh [4 [3 36 achieved
PSNR scores of 21.95 23.59, lower than the vanilla NeRF A], which reached a PSNR of 23.85 [6] In contrast, our approach, which adaptively reduces computational cost per pixel (i.e., fragment count per pixel, as shown in Fig. [ (a)) based on learned truncation thresholds, preserves the rendering quality pre-trained 3D Gaussian models.
3 Preliminary
3.1 Gaussian Primitives of 3D Gaussian Splatting
3D Gaussian Splatting [16) represents scene with collection of anisotropic Gaussian primitives. Each primitive parameterized by 3D Gaussian function G(x), with a covariance matrix Σ E R3x3 centered at a mean value (center point) μ E Rs:
G(x) = e- -(=--=) (x--)
(I)
The covariance matrix E can be further decomposed into a rotation matrix R and scaling matrix S: Σ RSS" R² Besides storing the aforementioned geometry information, each Gaussian is also assigned a learnable opacity factor o and a set of spherical harmonics (SH) coefficients for view-dependent colors, following previous practices [ 13
3.2 Rendering Pipeline of 3D Gaussian Splatting
Given the aforementioned 3D Gaussian primitives and a specific viewpoint (camera), the rendering pipeline for 3D Gaussian Splatting can be divided into three stages:
Projection First, each 3D Gaussian primitive is projected onto the 2D image plane to be rendered by taking the marginal distribution of the 3D Gaussian function. This stage results in set of 2D Gaussian primitives [37]. Specifically, using a viewing transformation W and a Jacobian matrix J, which approximates the local projective transformation with affine mapping, the covariance matrix E' of the 2D primitive can be derived as follows:
E' = JWEW"
(2)
Sorting Second, all Gaussian primitives are sorted by their depth values d, i.e., the distance from their centers to the camera origin. As result, the Gaussian primitives ranked in order from nearest to farthest relative to the viewpoint:
d < d2 < d3 <_. < daG,
where #G is the total number of Gaussian primitives.
Rasterization Finally, for each pixel on the 2D image plane be rendered, the color and opacity of the sorted 2D Gaussian primitives are rasterized and accumulated through a- e-blending:
AG
C(p)
Σ c.o.
(p)
II (p()))
(4)
j=l
where p is the pixel center and the opacity value as (p) is determined by sampling a weighted 21) Gaussian function at location p:
a, o,e fpp " 1(p-H:)
(5)
In practice, it is computationally infeasible to sample Eq, at all pixels for all Gaussian primitives, which involves multiple vector-matrix multiplications. Therefore, 3D Gaussian Splatting [16] truncates each 2D Gaussian function at predetermined a threshold T = 266 and ignores the impact of 2D Gaussian primitives on the pixels falling outside the truncation range. This approach essentially replaces opacity value ag with of in Eq.
df (p) (p() (p) > T}.
(6)
where is an indicator function that equals if and only if the condition is met, and equals 0 otherwise.

---

Table I: Per-scene rendering time breakdown for vanilla [16] and pruned 18 3D Gaussian Splat- ting. Rendering times are measured in milliseconds, based on the first camera pose in the test set. The latency at each stage is stable for this fixed pose, with fluctuations below 0.02 milliseconds. Specifically, #G and #F represent the number of Gaussians and the number of fragments for the corresponding scene.
Scene
Pruned
Projection
Sorting
Rasterization
#G
#F
#F #G
7.64
6.61
42.57
5,991,553
56,788,756
9.5
Bicycle
/
1.32
0.90
18.00
533,288
34,633,815
64.9
X
9.92
8.08
45.50
5,742,553
58,067,
10.1
Garden
1.58
1.45
20.69
575,510
33,342,349
57.9
2.37
2.43
47.20
1,535,973
83,986,935
54.7
Room
1.10
0.87
32.39
393,991
76,513,014
194.2
6.28
4.71
59.44
1,809,084
99.977,477
55.3
Kitchen
/
1.41
1.01
36.88
432,212
82,708,870
191.4
4
Profiling and Analysis of 3D Gaussian Rendering Pipeline
To understand the bottlenecks of the aforementioned 3D Gaussian Splatting rendering pipeline, we perform profiling on an OpenGl--accelerated Gaussian Splatting renderer (38) using the Jetson Orin NX edge GPU system [[7]. Specifically, in the profiling experiments, adopt two outdoor scenes, "Garden" and "Bicycle", and two indoor scenes, "Room" and "Kitchen", from the Mip-NeRF 360 dataset [6]. To quantify the impact of prior 3D Gaussian Splatting pruning rendering speeds, profile both the vanilla 3D Gaussian Splatting and Mini-Splatting [18), SOTA method that reduces Gaussian primitives by 85.4% while achieving higher rendering quality compared to the vanilla one.
4.1 Understanding the Efficiency Bottleneck the Rendering Pipeline
As summarized in Tab. I. the Rasterization stage is the major bottleneck in rendering speed for both the vanilla and pruned 3D Gaussian Splatting. Specifically, this stage accounts for 71.66% to 94.25% of the overall rendering runtime.
The runtime dominated by the Rasterization stage because: (1) for the Projection and Sorting stages, which perform per Gaussian projection and sort all the Gaussians based on depth order, respectively, the computational complexity is determined by the number of Gaussians (#G) to be rendered; (2) in contrast, the Rasterization stage's computational time is dominated by per fragment computation (i.e., Eq. D. whose complexity is mainly determined by the number of fragments (i.c., the sum of pixels covered by truncated 2D Gaussian primitives):
4G
#F = Σ Σ Ha,(p) > T).
Based on the summary in Tab. the number of fragments (#F) one orders magnitude larger than the number of Gaussians (#G) under all experimental settings, which results in higher computational complexity and longer rendering latency. This is why Rasterization is the primary bottleneck among the three stages.
4.2 Understanding the Effeet of Pruning 3D Gaussian Primitives
As summarized in Tab. [ we observe that pruning 3D Gaussian primitives with SOTA pruning techniques [18] results in varying degrees of speedup across the three stages the rendering pipeline. Specifically, when the number of Gaussians (#G) is reduced by 7.79x by [18]. the Projection and Sorting stages experience speedups of 4,85 x and 5.16x, respectively, while the Rasterization stage only gains speedup of 1.81 x. This implies that prior 3D Gaussian pruning efforts do not proportionally reduce the runtime the bottleneck stage relative to the reduction in the number of Gaussian primitives.
5

---

Original Gaussian with Fixed Truncation Threshold- T
Sigmold Function with Learnable Threshold/ Cutoff
Gaussian with Learned Truncation Threshold
Leareable
Sopa
Hagh Cale
Gaussian Renderer
Caloi
Gradient
Gradient
Cotoll
Images
Figure 2: The overall pipeline of our proposed fragment pruning framework. We apply a learnable truncation threshold for each Gaussian to reduce the number of pixels (i.e., fragments) covered by each Gaussian primitive.
Leveraging the aforementioned analysis on computational complexity across the three different stages, and considering the number Gaussian primitives (#G) and fragments (#F) before and after pruning as summarized in Tab. | we can conclude the following: The less-than-expected reduction in runtime primarily because the number of fragments (#F) only decreases by 1.13x after pruning, which caused by an increased average number of fragments per Gaussian primitive (#F HG). This leads to a limited reduction in the computational complexity at the bottleneck Rasterization stage.
The Proposed Fragment Pruning Framework
Motivated by the profiling and analysis Sec. which reveals that prior 3D Gaussian Splatting pruning efforts overlooked the impact of the number of fragments, we propose fragment pruning. This approach performs pruning at the granularity fragments rather than at the raw 3D Gaussian primitive level, as shown in Fig. 2 Specifically,
detail the pruning mechanism via adaptive Gaussian truncation in Sec. 5. and describe the differentiable learning pipeline for deternining the optimal truncation threshold in Sec. 5.2
5.1 Fragment Pruning through Adaptive Gaussian Truncation
Based the insights summarized in Sec. we propose pruning the number of fragments covered by i-th 3D Gaussian primitive, i.e.,
#F, = Σ (a)(p) > T},
(8)
pEP
by increasing the threshold T' compared to the default value used in the vanilla 3D Gaussian Splatting. Specifically, to accommodate the size variations of different 3D Gaussian primitives, we propose learning a per Gaussian truncation threshold T,, rather than adopting global threshold T. Thus, the opacity value for the i-th 3D Gaussian primitive, as shown in Eq. 6 can be formulated as follows:
(p (p) > Ti).
worth noting that to avoid the computational workload associated with calculating exponentials for Eq. 5] we truncate Gaussian primitives based on the exponent term in Eq. 5 (denoted as F, instead of directly on O;₃-
5.2 Differentiable Learning of Truncation Thresholds
To locate the optimal truncation threshold T; for each Gaussian primitive, proposed fragment pruning framework starts with a pre-trained 3D Gaussian model and fine-tunes it using fully differentiable pipeline.
'Truncating based on E; is mathematically equivalent to truncating based on a., The equivalent truncation threshold for is ".

---

During fine-tuning, the key barrier to building fully differentiable pipeline is the non non-differentiable indicator function II. Thus, we propose approximating I with the differentiable Sigmoid function:
I(E((x) T:} Sigmoid( Ex Ti
(10)
where t is positive constant. We set T; and as trainable during fine-tuning and freeze all other parameters, including R, S, and the SH coefficients. This is because we empirically find that fine-tuning the entire model can lead to overfitting issues.
After fine-tuning, we discard the approximation and directly adopt Fq. 9 for rendering with a reduced number of fragments. Therefore, there is no need compute the Sigmoid function during rendering, and no extra rendering time is incurred due to the approximation Eq. 10
Experiments
6.1 Experiment Setup
Datasets. We evaluate our proposed techniques on both static and dynamic scenes. For static scenes, adopt the five outdoor scenes and four indoor scenes from the Mip-NeRF 360 dataset [6], two scenes ("Train" and "Truck") from the Tanks& Temples dataset [19] and two scenes ("DrJohnson" and "Playroom") from the Deep Blending dataset [22]. For dynamic scenes, select the Plenoptic Video Dataset (39, which is composed of six real-world video sequences.
Devices. To validate the effectiveness of the proposed approach, benchmark the rendering speed of our method and the baselines on a consumer hardware device, Nvidia's 's edge GPU, the Jetson Orin NX (17]. The rendering resolution for all scenes is fixed 1080P (1920 x 1080) to match the minimal requirements of AR/VR applications
40
6.2 Implementation Details
Pre-training and Initialization. We the official implementations and the default hyperparame- ters 3D Gaussian Splatting (16), Mini-Splatting [16], and 4D Gaussian Splatting (28) to obtain pre-trained models. Before fine-tuning, initialize the truncation threshold T, for each Gaussian based on its opacity:
T: = lo 64
(11)
so that after initialization, the effective threshold on is 67
Fine-tuning on Static Scenes. To validate the robustness of the proposed techniques, we use the same set of hyperparameters to fine-tune all scenes across different datasets. Specifically, we fine-tune each scene for 5,000 epochs, utilizing batch size In particular, we adopt the Adam optimizer with learning rate of 0.01, 31 0.9, and 33 0.99 during the fine-tuning process. We adopt the same Loss and SSIM Loss as the pre-training process [6]
Fine-tuning on Dynamic Scenes. In dynamic scenes, we maintain the same hyperparameters, optimization strategies and loss functions as those used in static scenes. However, we adjust our training batch size to 4, adhering to the default batch size specified in the 4D Gaussian Splatting training (28).
To fully leverage the hardware- graphics pipeline of con- sumer GPUs, we conducted performance benchmarks for vanilla 3D Gaussian Splatting [16] and Mini-Splatting (18) using OpenGL-based renderer (38). Additionally, we have enhanced this renderer to incorporate our adaptive Gaussian truncation technique and to facilitate the rendering of dynamic scenes using 4D Gaussian Splatting 28
Renderer Implementation.
6.3 Quantative Results on Static Scenes
Tab. presents a quantitative comparison of the proposed approach and baseline methods on the static scenes. We report the average Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index
7

---

Table 2: Quantitative comparison of our method and previous works on the Plenoptic Video Dataset. Frame rates (FPS) of our method and 4DGS (28) are measured on Jetson Orin NX [17]. *: Frame rates capped 60 FPS. the Mip-NeRF 360 dataset, Mini Splating Ours achieved 60 FPS on out of scenes. In the Tanks&
Tanks&Temples Temples and Deep Blending dataset, all scenes achieved at 60 FPS.
Datasct
Mip-NeRF 360
Tanks& Temples SSIM
Deep Blending SSIM1
Method I Metric
PSNRT
SSIM
LPIPS.
FPS
PSNR
LPIPS]
FPST
PSNR1
LPIPS).
FPST
Plenoxels 9
23.08
0.626
0.436
21.08
0.719
0,379
23.06
0.795
0.510
INGP-Big [
25.59
0.699
0.331
21.92
0.745
0.305
24.96
0.817
0,390
Mip-NeRF 360 8
27.69
0.792
0.331
21.92
0.745
0.305
24.96
0.817
0.390
3D Gaussian
16
27.21
0.815
0.214
20
23.14
0.841
0.183
29.41
0.903
0.243
18
3D Gaussian
16
27.44
0,813
0.218
20
23.71
0.848
0.177
29.55
0.904
0.244
18
Ours
27.48
0.812
0.209
37
23.75
0.845
0.176
36
29.60
0.899
0,243
44
Mini-Splating
18
27.34
0.822
0.217
35
23.18
0.835
0.202
29.98
0,908
0.253
39
Ours
27.38
0.822
0.209
54"
23.34
0,835
0.200
60
30.04
0.905
0,252
60
Measure (SSIM), Learned Perceptual Image Patch Similarity (LPIPS), and Frames per Second (FPS) on the Mip-NeRF-360, Tanks& Temples, and Deep Blending datasets.
Our proposed fragment pruning consistently enhances rendering speed while maintaining or improv- ing rendering quality across various datasets. When applied to pre-trained 3D Gaussian Splatting models, this technique significantly increases FPS, with gains of +17, +20, and +26 across three different datasets, effectively more than doubling the rendering frame rates, Concurrently, it also boosts the PSNR by +0.04, +0.04, and +0.05, respectively, across the three datasets.
Furthermore, the data presented in Tab. 2 illustrate that the proposed fragment pruning approach outperforms the SOTA Gaussian primitive pruning technique, i.e., Mini-Splatting [18), in terms of the rendering speed vs. quality trade-offs. Across all three real-world datasets, the proposed fragment pruning consistently achieves higher FPS than Mini-Splatting, while maintaining superior or comparable rendering quality. For example, on the Tanks& Temples dataset [19], applying our fragment pruning technique to pre-trained 31) Gaussian Splatting model achieves a 0.57 PSNR
improvement and FPS increase over Mini-Splatting (18]
Most importantly, our experimental findings, as shown in Tab. 2 demonstrate that the proposed fragment pruning technique complements the SOTA Gaussian primitive pruning approach (18). By integrating our fragment pruning method with Mini-Splatting 18]. we achieved significant enhancements in rendering efficiency. Specifically, the application of fragment pruning conjunction with Mini-Splatee Splattipatting resulted in increases in rendering speeds by +19, +25, and +21 FPS, equating to improvements of +54%, +71%, and +54% on the Mip-NeRF-360. Tanks& Tanks&Temple.. and Deep Blending datasets, respectively, while maintaining or even enhancing rendering quality. Notably, on the T Temples and Deep Blending datasets, the combined use of fragment pruning and Gaussian primitive pruning achieved the maximum screen refresh rate of 60 FPS, underscoring the efficacy of these combined techniques in practical scenarios.
6.4 Quantative Results on Dynamic Scenes
Tab. 3 summarizes the quantitative improvements achieved by integrating fragment pruning into 4D Gaussian Splatting [27] In real-world dynamic scenes, the implementation of this pruning method
Table 3: Quantitative comparison of our method and previous works on the Plenoptic Video Dataset. I; numbers from 4DGS paper; 2: FPS only measured on four scenes, i.c. without Coffeer Martini and Flame Salmon, which run out of memory on Jetson NX.
Method
PSNRT
SSIM
LPIPS$
FPS
K- X-Planes-explici
43
30.88
0.020
0.23
K- K-Planes-hybrid 43
31.63
0.018
Mix VoxelssL
44
30.80
0.020
0.126
16.7
NeRFPlayer
45
30.69
0.035
0.111
0.045
HexPlane 46
31.70
0.014
0.075
0.56
HyperReel
47
31.10
0.037
0.096
2.00
4DGS (
32.01
0.030
0.055
14
Ours?
32.04
0.030
0.053
22
8

---

TruundTruth
3D Gauesian fps
3D Gauzzion +Oun
Mini Splatingg fpri
Mini Spictting Our
Guardd
JD Ganziin (1B.pe)
3D Gaussian Ourstéf fpst
Mini- Splatting (39 fps)
Mini- Spiattpttttingg + Ourst
713
713
713
713
713
GRxZR Truhh
Caussian (16fps)
3D Gazziim Ours(361pp (pa)
Min- Splattppptty (36-ffs
Mini Muil-Splatting OOrrst fps)
Figure 3: Qualitative comparison of 3D Gaussian Splatting and different Gaussian pruning methods. Zoom in for better view.
results in an average increase in rendering speed of +8 FPS, maintaining the same quality of rendering. This enhancement facilitated by the direct application of hyperparameters optimized for static scenes, demonstrating the robustness and versatility of the fragment pruning strategy.
6.5 Qualitative Results
Fig. qualitatively compares the rendering output from vanilla 3D Gaussian Splatting [6], 3D Gaussian Splatting with the proposed fragment pruning approach, Mini-Splatting [8] and Mini- Splatting with the proposed fragment pruning approach. Based on the first row of Fig. 3
we can observe that prior Gaussian primitive pruning approaches like Mini-Splatting result in severe artifacts in some distant regions (e.g., the leaves). In contrast, the proposed fragment pruning approach better preserves the rendering quality in such areas while achieving similar or better rendering efficiency. From the second row of Fig. 3 we can conclude that although Mini-Splatting removes the blurry reconstruction, it results in artifacts reconstructing the dotted line. Additionally, as shown in the last column the second row, learning per Gaussian truncation threshold in our proposed fragment pruning
partially recover the rendering quality. Similarly, Mini-Splatting can result in drop in rendering quality (a less clear number "1" as compared to the vanilla 3D Gaussian Splatting), while proposed fragment pruning can recover the reconstruction quality. More qualitative results are presented in the appendix.
the last row of Fig. 3
7 Discussion
7.1 Limitation
One limitation this work the additional fine-tuning time required, since we de- signed fragment pruning a post-training fine-tuning technique. As shown in Tab. 4 the fine-tuning stage incurs overhead of 16% 21% % relative to the pre- pre-training time. Integrating threshold learning into the original pre-training process is potential solution, which we leave for future work.
Table Comparison of pre-training and fine-tuning times on A5000 GPU (in minutes).
On 3D Gaussian [6
On Mini-Splatting [18
Dataset
Pre-training
Fine-tuning
Pre-training
Fine-tuning
Mip-NeRF 360
45.2
7.7
17.2
3.4
T 19
27.2
4.4
11.4
2.2
Deep Blending 22
37.4
7.1
16.2
3.4
7.2 Future Works
There plethora of downstream tasks that leverage the benefits of 3D Gaussian Splatting models including the generation of 3D contents and dynamic scene construction. In order to generate 3D objects from single image of text prompt, DreamGaussian [29 and GSGen [49] combine score
distillation sampling with 3D Gaussian primitives while LGM [50] utilize an asymmetric UNet

---

architecture to predict the parameters of Gaussian features. On the other hand, multiple works have extended 3D Gaussian Splatting models incorporate time by introducing time-variant opacity [5]] or directly including time as 4D Gaussian parameter 27 28 to generate dynamic scenes. Our work has focused on the efficiency our algorithm on dynamic scene generation using 4D Gaussian features, and we leave the application of fragment pruning to other tasks as future research.
8 Conclusion
In this work, we propose novel fragment pruning algorithm that effectively improves the rendering speed of 3D Gaussian Splatting models. We show through rigorous profiling that the main bottleneck of the rendering process is the rasterization stage whose computational complexity is determined by number of fragments. Building on this finding, we suggest an algorithm that learns the truncation threshold for each Gaussian instead of using predetermined heuristic threshold, which allows to truncate Gaussians in a more aggressive manner and thereby reduce the number of fragments. As result, our method has achieved significant rendering speed improvements over the original 3D) Gaussian model with negligible deterioration or even an improvement of quality. While our algorithm alone demonstrates superior performance gains over SOTA Gaussian primitive pruning techniques, it can be used in conjunction with SOTA primitive pruning algorithms to further boost the rendering speed of 3D Gaussian models.
Acknowledgments and Disclosure of Funding
This work is supported by the NSF Computing and Communication Foundations (CCF) program (Award ID: 2312758), and the CoCoSys, one of the seven centers in JUMP 2.0, a Semiconductor Research Corporation (SRC) program sponsored by DARPA. Furthermore, extend our gratitude towards the reviewers of this paper for their insightful comments and suggestions.
References
[1] Stephen Lombardi, Tomas Simon, Jason Saragih, Gabriel Schwartz, Andreas Lehrmann, and Yaser Sheikh. Neural volumes: learning dynamic renderable volumes from images. ACM Transactions Graphics, 38(4):1-14, July 2019.
[2] Ricardo Daniel Duckworth. NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections. In CVPR, 2021.
Noha Radwan, Mehdi S. M. Sajjadi, Jonathan T. Barron, Alexey Dosovitskiy, and
Nianchen Deng, Zhenyi He, Jiannan Ye, Budmonde Duinkharjav, Praneeth Chakravarthula, Xubo Yang, and Qi Sun. Fov-nerf: Foveated neural radiance fields for virtual reality. IEEE Transactions on Visualization and Computer Graphics, 28(11):338) (1)14))3854 3864, 2022.
[4] Ben Mildenhall, Pratul Srinivasan, Matthew Tancik, Jonathan Barron, Ravi Ramamoorthi, and Ren
Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. In ECCV, 2020.
Jonathan T. Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin Martin-Brualla, and Pratul P. Srinivasan. Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields, 2021.
[6] Jonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul Srinivasan, and Peter Hedman. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. CVPR, 2022.
[7] Jonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan, and Peter Hedman. Zip Zip-nerf: Anti- anti-aliased grid based neural radiance fields. ICCV, 2023.
[8] James T Kajiya and Brian P Von Herzen. Ray tracing volume densities. ACM SIGGRAPH computer graphics, 1853):165-174, 1984.
[9] Sara Fridovich-Kan Keil and Alex Yu, Matthew Tancik, Qinhong Chen, Benjamin Recht, and Angjoo Kanazawa. Plenoxels: Radiance fields without neural networks. In CVPR, 2022.
[10] Cheng Sun, Min Sun, and Hwann-Tzone Chen. Direct voxel grid optimization: Super-fast convergence for radiance fields reconstruction. In CVPR, 2022.
[11] Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, and Hao Su. Tensorf: Tensorial radiance fields. In European Conference Computer Vision (ECCV), 2022.
[12] Alex Yu, Ruilong Li, Matthew Tancik, Hao Li, Ren Ng. and Angjoo Kanazawa. PlenOctrees for real-time rendering of neural radiance fields. In ICCV, 2021.
10

---

[13] Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphies primitives with multiresolution hash encoding, ACM Trans. Graph., 1104)::02:11102:11, July 2022.
[14] Zhiqin Chen, Thomas Funkhouser, Peter Hedman, and Andrea Tagliasacchi. Mobilenerf: Exploiting the polygon rasterization pipeline for efficient neural field rendering on mobile architectures. In The Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
[15] Jiaxiang Tang, Hang Zhou, Xiaokang Chen, Tianshu Hu, Errui Ding, Jingdong Wang, and Gang Zeng. Del- icate textured mesh recovery from nerf via adaptive surface refinement. arXiv preprint arXiv:233..0209, 2022.
[16] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Transactions on Graphics, 42(4), July 2023.
[17] Jetson_ond orin__x_dee 100020001111111 1
_v0.5. httpss//devveloper www..ddowwlooo. oad. avidia. cww/ccoo//a//ao /asets// embedded/ embedded/seeurreeeeeee http://eeeeeeeeeeeee////s/sss/ee /tr///riinnnnnre/ree//e http://ttoooorrooonn//////////ee Orin _t_______SSS__ 10112-01100111 001_00.5.....000 5._.00..pppp..
(Ac-
cossed on 05/18/2024).
[18] Guangchi Fang and Bing Wang. Mini-splatiing Representing scenes with a constrained number of gaussians. arXiv preprint arXiv:2403. 14166, 2024.
[19] Amo Knapitsch, Jaesik Park, Qian-Yi Yi Zhou, and Vladlen Koltun. Tanks and temples: Benchmarking large-scale scene reconstruction. ACM Transactions on Graphics (ToG), 36(4):1-13, 2017.
[20] Noah Snavely, Steven M Seitz, and Richard Szeliski. Photo tourism: exploring photo collections in 3d. In ACM siggraph 2006 papers, pages 835 846, ACM, 2006.
[21] Michael Goesele, Noah Snavely, Brian Curless, Hugues Hoppe, and Steven M. Seitz. Multi-view steren for community photo collections. In 2007 IEEE/ I 1th International Conference on Computer Vision, pages 1-8, 2007.
[22] Peter Hedman, Julien Philip, True Price, Jan-Michael Frahm, George Drettakis, and Gabriel Brostow, Deep blending for vice-viewoiint image-basor based rendering. ACM Transactions on Graphics (ToG), 37(6):1-15. 15, 2018.
[23] Vincent Sitzmann, Justus Thies, Felix Heide, Matthias NieBner. Gordon Wetzstein, and Michael Zollhofer. Deepvoxels: Learning persistent 3d feature embeddings. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 2437-2446, 2019.
[24] Justus Thies, Michael Zollhöfer, and Matthias Nießner. Deferred neural rendering: Image synthesis using neural textures. Acm Transactions on Graphics (TOG), 38(4):1-12, 2019.
[25] Gernot Riegler and Vladlen Koltun, Free view synthesis, In Computer Esion-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XIX 16, pages 623-640. Springer, 2020.
[26] Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis. arXiv preprint arXiv:2308.00 09713, 2023.
[27] Guanjun Wu, Taoran Yi. Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Wang Xinggang. 4d gaussian splatting for real-time dynamic scene rendering. arXiv preprint
arXiv:2310.08526 2023.
[28] Zeyu Yang. Hongye Yang, Zijie Pan, and Li Zhang. Real-time photorealistic dynamic scene representation and rendering with 4d gaussian splatting. In International Conference on Learning Representations (ICLR), 2024.
[29] Jiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei I. Liu, and Gang Zeng. Dreamgaussian: Generative gaussian splatting for efficient 3d content creation. arXiv preprint arXiv:2309. 16653, 2023.
[30] Huan Ling, Scung Wook Kim, Antonio Torralba, Sanja Fidler, and Karsten Kreis. Align your gaussians: Text-to-4d with dynamic gaussians and composed diffusion models. arXiv preprint arX1v:2312.13763, 2023.
[31] Joo Chan Lee, Daniel Rho, Xiangyu Sun, Jong Hwan Ko, and Funbyung Park. Compact 3d gaussian representation for radiance field. arXiv preprint arXiv:2311.1366 2023.
[32] Zhuang Liu, Jianguo Li, Zhiqiang Shen, Gao Huang, Shoumeng Yan, and Changshui Zhang. Learning efficient convolutional networks through network slimming. In Proceedings of the IEEE international conference on computer vision, pages 2736-2744, 2017.
[33] Yann LeCun, John Denker, and Sara Solla. Optimal brain damage. Advances in neural information processing systems, 2, 1989.
[34] Zhiwen Fan, Kevin Wang, Kairun Wen, Zehao Zhu, Dejia Xu, and Zhangyang Wang. Lightgaussian: Unbounded 3d gaussian compression with 15x reduction and 200+ fps. 2023.
[35] Panagiotis Papantonakis, Georgios Kopanas, Bernhard Kerbl, Alexandre Lanvin, and George Drettakis. Reducing the memory footprint of 3d gaussian splatting. Proceedings of the ACM on Computer Graphics and Interactive Techniques, 7(1):117,, 2024.

---

[36] Marie-Julie Rakotosaona, Fabian Manhardt, Diego Martin Arroyo, Michael Niemeyer, Abhijit Kundu, and Federico Tombari. Nerfmeshing: Distilling neural radiance fields into geometrically-accuaate 3d meshes. In 2024 International Conference on 3D Vision (3DV), pages 1156-1165. IEEE, 2024.
[37] M. Zwicker, H. Pfister, J. van Baar, and M. Gross. Ewa volume splatting. In Proceedings Visualization, 2001. VIS '01., pages 29 538. 2001.
[38] Simon Niedermayr, Josef Stumpfegger, and Rüdiger Westermann. Compressed 3d gaussian splatting for accelerated novel view synthesis. arXiv preprint arXiv:2401 .02436, 2023.
[39] Tianye Li, Mira Slaveheva, Michael Zollhoefer, Simon Green, Christoph Lassner, Changil Kim, Tanner Schmidt, Steven Lovegrove, Michael Goesele, Richard Newcombe, et al. Neural vidco synthesis from multi-view video. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5521-5531, 2022.
[40] Mikko Pitkämen, Marko Viitane,, Alexandre Mercat, and Jamno Vanne. Remote vr gaming on mobile devices. In Proceedings of the 27th ACM International Conference on Multimedia, pages 2191-2193, 2019.
[41] Xueshi Hou, Yao Lu, and Sujit Dey. Wireless vr'ar with edge/cloud computing. 2017 26th International Conference on Computer Communication and Networks (ICCCN), pages 1-8. IEEE, 2017.
[42] Marko Viitane,, Jarno Vanne, Timo D Hämäläinen, and Ari Kulmala. Low latency edge rendering scheme interactive 360 degree virtual reality gaming. 2018 JEEE 38th international conference on distributed computing systems (ICDCS), pages 1557-1560. JEEE, 2018.
[43] Sara Fridovich-Kan Keil, Giacomo Mcanti, Frederik Rahbæk Warburg, Benjamin Recht, and Angjoo Kanazawa. K- K-planes: Explicit radiance fields in space, time, and appearance. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11779-1248,, 2023.
[44] Feng Wang. Sinan Tan, Xinghang Li, Zeyue Tian, Yafei Song, and Huaping Liu. Mixed neural voxels for fast multi-view video synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 19706-19716, 2023.
[45] Liangehen Song, Anpei Chen, Zhong Li, Zhang Chen, Lele Chen, Junsong Yuan, Yi Xu, and Andreas Geiger. Nerfplayer: A streamable dynamic scene representation with decomposed neural radiance fields. IEEE Transactions on Visualization and Computer Graphicx, 29(5):773--2722 2023.
[46] Ang Cao and Justin Johnson. Hexplane: A fast representation for dynamic scenes, In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 130- 330-141, 2023.
[47] Benjamin Attal, Jia-Bin Huang. Christian Richardt, Michael Zollhoefer, Johannes Kopf, Matthew O'Toole, and Changil Kim. Hyperreel: High-fidelity 6-dof video with ray ray-conditioned sampling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16610-16620, 2023.
[48] Zeyu Yang, Hongye Yang, Zijie Pan, Xiatian Zhu, and Li Zhang. Real-time photorcalistic dynamic scene representation and rendering with 4d gaussian splatting. arXiv preprint arXiv:23100 10642, 2023. arXiv preprint
[49] Zilong Chen, Feng Wang, and Huaping Liu. Text-to-3d using gaussian splatting. 2.v9 16585, 2023.
[50] Jaaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang, Gang Zeng, and Ziwei Liu. Lgm: Large multi-view gaussian model for high-resolution 3d content creation. arXiv preprint arXiv:240..05055 2024.
[51] Zhan Li, Zhang Chen, Zhong Li, and Yi Xu. Spacetime gaussian feature splatting for real-time dynamic view synthesis. arXiv preprint 2rXi:23312.16812 2023.
12

---

A More Rendering Output Visualization
Fig. provides more visualization on the rendering output from vanilla 3D Gaussian Splatingg [6].
3D Gaussian Splatting the proposed fragment pruning approach, Mini-Splatting and Mini-Splatting
+ the proposed fragment pruning approach.
Groun Truhh
3D Gaussinn (16 Ips)
Gaussian + Ousi
Mi Spllling (35
Mark Splelling + +Oust60 (je)
Ground Truh
3D Grusiam (20 fps
3D Gazzzan +Oust(37 f(x)
Mini Spletting (35 fpe
Mira Splatting + Ouant Spa)
Ground Truhh
3D Caussion (20
3D Geuzzia Ours(37 Eps)
Miri Spiatting (36
Min- Spletting +O Eps)
GmandTTnt
Garzion
3D Caumaion Ounet
Min- Splatting (35 fpe)
Miri Splatting Oamz35 fpe)
Ground Truth
3D Gausson
3D Gauzdan +umm337 fpe)
Miri Spiatting
Min Spianting Curmt144 Foc)
Ground Truth
3D Gausson (20 fpx)
3D Quasian +Oun(37
Misi Spiatting (36 fps)
Mint Strattigaattingg +OustBd Eps)
GeondTuuth
+unst (ps)
(3ri-S
Spii--ppiaiingg +unst
Figure 4: Qualitative comparison between vanilla 3D Gaussian Splatting, state-of-the-an (SOTA) Gaussian primitive pruning work, Mini-Splatting (18), and our proposed Fragment Pruning.
13

---

B More Fragment Density Visualization
Fig. 5 compares the rendering quality and per-pixel fragment density before and after applying the
proposed fragment pruning technique on pre-trained 3D Gaussian models [16]. As observed in column (c), the object borders exhibit the highest fragment density. By reducing the fragment density
at these borders, as shown in column (d), our method enhances rendering quality, especially along object edges (illustrated in column (b)).
20
0.59
0.25
280
0.25
0.90
0.25
0.00
0.25
0.00
(c)
Figure 5: Visualizing of rendering quality improvement and fragment density reduction. (a) Ground truth image. (b) Regions where fragment pruning improves rendering fidelity, measured by a reduction in per-pixel LI error. (c) Fragment density before fragment pruning. (d) Fragment density after fragment pruning.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The claims are justified by the following sections.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Having a separate section to discuss the limitations.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
15

---

Justification:
No theoretical results in the papers.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Code will be provided.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
16

---

Answer: [Yes]
Justification: Code will be provided.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ cppppi.ccppulli//uuuii wwwtc//ggue////eeeeceoocc CoodeSubeiiieeee ssi wswonnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Details provided in the paper. Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification:
No significant statistical instability, therefore error bards not provided,
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
17

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of exccution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Provided in the paper.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code Of Ethics
Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Question:
Answer: [Yes]
Justification:
The research conform with the NeurIPS Code Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification: This work mainly enhances the rendering efficiency, therefore is not directly tied to negative applications.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
18

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: This work mainly enhances the rendering efficiency, therefore is not directly tied to negative applications.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Cited in the paper.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the Datasets.papersstt paperswi thcode. cow/ datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
19

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: No new assets.
Guidelines:
The answer NA means that the paper does not release new assets,
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
20


Reviews:
[{"review_id": "2D3lUYgXA5", "reviewer": "Reviewer_ytrw", "summary": "The paper presents a pruning method to improve the time efficiency of 3D Gaussian Splatting. Different from existing methods that prune Gaussians, the proposed method prunes pixels per each Gaussian. Specifically, for each pixel ray, the proposed method reduces the number of Gaussians needed to compose the color. To this end, the paper proposes to learn the Gaussian-wise threshold to adaptively prune the unnecessary Gaussians for each pixel ray. Different from the fixed global threshold in existing 3DGS pipelines, their per-gaussian threshold effectively decreases the number of Gaussians without degrading the rendering quality. The paper validates their method in different datasets and two devices, showing the advantages over previous methods that try to improve the time efficiency.", "questions": "Please address the questions and concerns that I listed in the weaknesses section. \n\nI currently tend to vote for borderline rejection as I think the paper has something that needs to be fixed. But I'm personally very positive about this paper. So I encourage authors to address the concerns that I have. Especially, I would like to see authors show the proposed solution to reduce the number of Gaussian per pixel ray is fully explored -- e.g., insight into why it can work, comparison with the naive solution, and so on.\n\n-------\n\nFinal evaluation: \nThe rebuttal addressed most of the concerns. So I'm very happy to raise my score.", "limitations": "Yes, limitations and potential negative impacts have been discussed in the paper.", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The work has a certain impact on the applications of 3DGS. Because it’s simple and effective for improving the time efficiency of 3DGS.  To be more specific, the proposed method acts as an additional step that improves the inference efficiency of all trained 3D Gaussian Splats with degraded rendering quality.  This would be very useful for industrial applications. \n\nThe profiling results are useful to the community. This helps in understanding the latency distribution over steps of 3DGS.\n\nThe paper is also easy to follow.", "weaknesses": "The paper has several weaknesses which I’ll detail below\n\nWriting: \n- The title does not match what the paper achieves. In my understanding, the proposed method does not change the number of Gaussians. So it does not make sense to claim that the Gaussians can be sparser. \nThe concept of fragment is not easy to follow in the abstract and intro. From my personal viewpoint, the author could simply say the Gaussians per pixel ray. \n- In the abstract, it would be better to always mention the device/platform when the paper claims the improved speedup. At line 35 of the intro section, similarly, it would be better to be more specific. \n- The implementation details of fine-tuning have the loss description missing. Is that l1 loss and SSIM loss? \n- Also, what’s the connection between the proposed method and two staged frameworks to accelerate NeRF -- e.g., NeRF to mesh?  Intuitively, they share the same idea of reducing the number of samples per ray. In which case, I believe it makes sense to discuss the connection between these two methods. \n\nMethods: \n- I don’t quite understand the insight of fine-tuning to learn the Gaussian-wise threshold. Despite the improved results, I’m not sure why the method can automatically discover the perfect Gaussian-wise threshold that reduces the number of Gaussians per pixel ray. Theoretically, it’s also possible to learn the thresholds that increase the number of Gaussians per pixel ray. It would be better to elucidate the method with more insights.\n\nExperiments: \n- There are naive methods to encourage a smaller number of Gaussians per pixel. For example, we could encourage smaller scales of Gaussians.  It might also have the same effect as the proposed method.  Given the native nature of this simple idea, I would encourage the authors to simply discuss or compare it with this naive idea. \n- An edge-device demo could make this paper stronger, thus I strongly recommend it. The reason is that Gaussian Splatting is real-time in GPU already. While it’s necessary to further improve it as it’s useful in some other applications, it might be more interesting to the research community if the proposed method can enable real-time efficiency in edge devices.\n- From my point of view, there might be randomness in terms of profiling. So I disagree with the authors that the statistics are stable and error bars are not needed. \n- Instead of Vanilla 3DGS in 3D and 4D settings, I strongly recommend authors validate their methods in other variants of 3DGS including Mip3DGS, GS-MCMC, etc. I would expect the reduced benefits of using the proposed methods without this validation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We greatly appreciate your positive feedback and constructive suggestions for our work. Below, we address each of your comments in detail:\n\n---\n**W1: Title mismatch: The method changes the # of Gaussians per pixel ray.**\n\nThank you for the suggestion! We will revise the title accordingly and clarify that the projection of the 3D Gaussian onto the 2D image to be rendered is sparser, not the 3D Gaussian itself.\n\n---\n**W2 & W3: Device details, # of Gaussians, and losses.**\n\nThanks for pointing these out! To address your suggestions, the following details will be added to our final version:\n> + Line 18: The speedup is achieved on an edge GPU device Jetson Orin NX [18].\n> + Line 35: The concrete number of Gaussians is 3,161,131 on average in the Mip-NeRF 360 dataset [6] when trained using the original 3D Gaussian implementation [16].\n> + Sec. 6.2: We used the same loss function as the original 3D Gaussian Splatting paper [16] for static scenes (i.e., both SSIM Loss and L1 Loss) and the same loss function as the RT-4DGS paper [29] for dynamic scenes.\n\n\n---\n**W4: Connection with two staged frameworks.**\n\nThe connection between the prior two-stage works and our work can be summarized as follows:\n\n**Similarity**: Both prior works [14,a,b] and our work reduce computational workload by decreasing the computation per pixel ray.\n\n**Differences**: Our technique serves as an add-on to one of the most advanced scene representations, 3DGS, which allows for faster rendering without sacrificing quality. Different from prior two-stage works, our method does not alter the scene representation and thus can better maintain the rendering quality. As illustrated in Fig.2 of the global 1-page response, our method reduced the average number of fragments/Gaussians per pixel without strictly enforcing it to be 1. As a result, our approach can boost rendering speed while maintaining the rendering quality, achieving a +0.04 PSNR improvement on [6] over the vanilla 3DGS [16]. In contrast, the prior two-stage frameworks [14,a,b] convert NeRFs into meshes to enforce a single sample per pixel ray. This significant reduction, along with the differences between the scene representation used in training and rendering, leads to noticeable quality loss. For instance, on the MipNeRF-360 dataset [6], the works [14,a,b] achieved PSNR scores of 21.95/22.74/23.59, which are lower than even the vanilla NeRF [4] (23.85 PSNR, as reported by [6]). \n\nWe will incorporate the discussion above into our final version.\n\n\n[a] Delicate Textured Mesh Recovery from NeRF via Adaptive Surface Refinement, ICCV 2023 \n\n[b] Distilling Neural Radiance Fields into Geometrically-Accurate 3D Meshes, 3DV 2024\n\n\n\n---\n**W5: Why can it automatically discover the perfect Gaussian-wise threshold to reduce the number of Gaussians per pixel ray? It could theoretically learn thresholds that increase the number as well.**\n\nYour understanding is correct! Learning the threshold can either decrease or increase the number of Gaussians per pixel ray. **The key insight behind this is that the decrease or increase is determined by the types of signals we want to represent using the Gaussian function**. Specifically, as shown in Fig. 1(a) of our global 1-page response, enhancing each Gaussian function with a learnable threshold essentially gives each Gaussian a flexible cutoff range. Thus, when the signal we want to fit has sharp edges (e.g., the square signal in Fig. 1(d) of our global 1-page response), learning the Gaussian-specific threshold offers the highest fidelity in fitting the ground truth signal compared to the original Gaussian function. Since the real-world scenes captured in our experiments contain many such sharp edges, also the source of anti-aliasing artifacts that [6,7] target, the threshold is learned to make the Gaussian cutoff narrower. This results in a reduced number of Gaussians per pixel ray. Our visualization of the number of fragments per pixel, with and without our method, in Fig. 2 of the global 1-page response also indicates that the number of fragments decreases most significantly at the edges of scenes. \n\n---\n**W6: Naive methods: smaller scales of Gaussians.**\n\nFollowing your suggestions, we conducted experiments to encourage smaller scales of Gaussians by introducing an additional loss on scales. As summarized in Fig.3(a) of the global 1-page response, we found that while this approach can also improve rendering speed, it does not recover rendering quality to the same extent as our method. Specifically, the lower quality of the native method is due to the fact that smaller scales of Gaussians do not have the same effect as Gaussians with a cutoff, as illustrated in Fig. 1(a) and (c). This results in a failure to fit sharp signals, such as the square signal shown in Fig. 1(d) of the global 1-page response, as effectively as our method.\n\n\n---\n**W7: Edge-device demo.**\n\nWe have demonstrated the real-time edge-device demo setup on a 15-watt edge GPU device, the Nvidia Jetson Orin NX 16GB, as shown in Fig. 4(a) of the global 1-page response. We will include this setup as part of our code release in the final version.\n\n---\n**W8: Profiling with  error bars.**\n\nGreat suggestion! We have further enhanced our profiling in Fig. 4(b) of the global 1-page response by including the standard error.\n\n\n---\n**W9: Validate our methods in other variants of 3DGS.**\n\nThank you for the suggestion! Given the limited rebuttal time, we have extended our work to GS-MCMC [c]. As shown in the table below, our proposed method improves the rendering speed of GS-MCMC by 1.8× without compromising the rendering quality. We will conduct experiments on other 3D Gaussian variants, such as Mip3DGS, in the final version.\n\n|        | PSNR  | SSIM | LPIPS | FPS (On Orin NX)|\n|--------|-------|------|-------|-----|\n| GS-MCMC| 29.72 | 0.89 | 0.19  | 20  |\n| GS-MCMC + Ours  | 29.75 | 0.90 | 0.19  | 36  |\n\n[c] 3D Gaussian Splatting as Markov Chain Monte Carlo, Arxiv 2024", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xR7J70owhz", "reviewer": "Reviewer_RCBw", "summary": "This paper investeigates the relationship between primitive pruning and rendering speed. It identifies that pruning 3D primitives does not translate proportionally to higher rendering speed and shows the real measure affecting rendering speed is the number of fragments (projected splats partaking in pixel color rendering). Finally it proposes a simple idea for optimizing Gaussian opacity truncation threshold, rather than setting a fixed value that reduces the number of fragments and results in higher rendering speed while keeping compression factor and quality.", "questions": "please see above (weaknesses) for some questions.", "limitations": "The limitation is adequetly addressed.", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- I like the idea behind this paper as it does a thorough analysis on the effect of pruning methods for compression on the rendering speed. It identifies why compression is not proportionally resulting in higher rendering speed and then addresses the issue with a simple and effective idea.\n- The manually set threshold parameter in 3DGS is one of the many hyper-parameters of this method that need fine-tuning and probably is not optimal. Making this value trainable is an important contribution.\n- The method shows significant improvement in rendering speed while maintaining or slightly improving both the compression factor and the quality.", "weaknesses": "- I would love to see experiment showing rendering speed and PSNR for different fixed values of threshold vs the learnt threshold. This is to verify setting the threshold to a higher value manually (and training with that) has less improvement than the proposed method. This would effectively show that having per Gaussian optimized threshold is truly needed.\n- I think having a longer training time for this purpose is valid. However, I am curious as to why is the method posed as a post processing? Did you explore doing an alternating optimization between the usual 3DGs parameters and the thresholds during training? I am not sure if that would result in lower training time in total than the usual training + post-processing, but it is interesting to see.\n- Additional point cloud renderings to assess the density of points/fragments on the qualitative figures of the supplementary material is helpful to have.\n- The results for overfitting with training on all the 3DGS parameters is not shown in the supplemental material. It is better to have a visualization for that experiment in the supplementary for reference.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your recognition of our work and your suggestions to further enhance our experimental results.\n\n---\n**W1: I would love to see an experiment showing rendering speed and PSNR for different fixed values of threshold vs the learnt threshold. This would effectively show that having per Gaussian optimized threshold is truly needed.**\n\nWe conducted a grid search on fixed threshold values and followed the same fine-tuning protocol as described in Section 6.2 of our paper. The results summarized in Fig. 3 (b) of the global 1-page response indicate that our method achieves a better accuracy vs. efficiency trade-off than tuning fixed threshold values, highlighting that learning a per-Gaussian threshold, as in our method, is truly necessary.\n\n\n---\n**W2: Did you explore doing an alternating optimization between the usual 3DGs parameters and the thresholds during training? I am not sure if that would result in lower training time in total than the usual training + post-processing, but it is interesting to see.**\n\nThank you for this  insightful question! Following your suggestion, we conducted an alternating optimization experiment on the MipNeRF 360 dataset [6]. Specifically, we maintained the training process for the first 15,000 iterations as in the vanilla 3D Gaussian Splatting [16], then alternated between threshold learning and 3D Gaussian parameter learning every 5,000 iterations up to 40,000 iterations. We tested the PSNR every 5,000 iterations, and the results are shown in the table below:\n\n\n| Iteration  | PSNR (Bicycle) | PSNR (Garden) | PSNR (Kitchen) | PSNR (Room) | Avg   |\n|------------|----------------|---------------|----------------|-------------|-------|\n| 15000      | 24.47          | 26.43         | 30.27          | 30.23       | 27.85 |\n| 20000      | 25.06          | 27.07         | 30.96          | 30.83       | 28.48 |\n| 25000      | 25.07          | 27.05         | 30.99          | 30.86       | 28.49 |\n| 30000      | 25.07          | 27.06         | 31.00          | 30.87       | 28.50 |\n| 35000      | 25.07          | 27.05         | 31.00          | 30.88       | 28.50 |\n| 40000      | 25.07          | 27.05         | 31.00          | 30.88       | 28.50 |\n| 3DGS (30000)| 25.19         | 27.30         | 31.29          | 31.45       | 28.81 |\n| Ours (35000)| 25.21         | 27.33         | 31.49          | 31.68       | 28.93 |\n\n\nWe can observe that the PSNR quickly saturates at around 20,000 iterations and fails to match the PSNR of the vanilla 3D Gaussian and our method. We conjecture that the lower rendering quality in this setting, compared to our method, is due to reducing the covered pixels per Gaussian too early in the training stage, which limits the number of pixels from which it can receive gradients. In contrast, our method strikes a balance between ease of use and achieved performance compared to alternating optimization. However, we believe that the alternating optimization you proposed has the potential to achieve superior rendering quality with more thorough parameter tuning. We will continue to explore this approach and include a discussion on optimization methods in our final version.\n\n---\n**W3: Additional point cloud renderings to assess the density of points/fragments on the qualitative figures of the supplementary material are helpful to have**\n\nWe have added the per-pixel fragment density in Fig. 2 of the global 1-page response, demonstrating the effectiveness of our proposed method in reducing fragment density across different pixels (e.g., reducing the average fragments per pixel from 49.2 to 32.9 in the Bicycle scene [6]), thereby improving overall rendering speed. We will include this discussion and provide additional visualizations of the point cloud renderings in our final version.\n\n\n---\n**W4: The results for overfitting with training on all the 3DGS parameters are not shown in the supplemental material.**\n\nThank you for the suggestion! We conducted overfitting experiments by loading pre-trained checkpoints at 30,000 iterations from native 3DGS and continuing training for an additional 20,000 iterations following the native 3DGS settings. As shown in Fig. 3 (c) of the global 1-page response, while overfitting does improve the overall PSNR, it still results in a slightly lower PSNR (e.g., -0.03) compared to our method, even with 40% more iterations.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tuGqE0BDND", "reviewer": "Reviewer_TxBU", "summary": "This paper introduces a novel approach to accelerate rendering speed in 3D Gaussian Splatting (3DGS) by selectively pruning overlapping fragments. This technique serves as an orthogonal enhancement to existing pruning methods (focuses on reducing the number of primitives) by selectively pruning overlapping fragments, thereby significantly accelerating rendering. The proposed adaptive pruning framework dynamically optimizes pruning thresholds for each Gaussian fragment, leading to marked improvements in both rendering speed and quality.", "questions": "1. Why did you primarily compare with neural radiance field methods instead of rasterization-based radiance field rendering methods as your baseline comparison in the rendering pipeline?\n\n2. Why did you include only Mini-Splatting [1] without comparing with more state-of-the-art Gaussian primitive pruning techniques? Why was there no comparison with any state-of-the-art Gaussian primitive pruning techniques in dynamic scenes?\n\n[1] Guangchi Fang and Bing Wang. Mini-splatting: Representing scenes with a constrained number of gaussians. arXiv preprint arXiv:2403.14166, 2024.", "limitations": "The authors have already mentioned the limitations regarding training time. Additionally, this method may also have limitations when pruning other types of primitive representation.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Through a detailed analysis of the efficiency bottleneck of the rendering pipeline and the impact of the number of Gaussian primitives on efficiency, this paper introduces a novel concept of fragment pruning to enhance rendering efficiency in 3D Gaussian Splatting (3DGS).\n\n2. Dynamically optimizes pruning thresholds for each Gaussian fragment, leading to significant improvements in rendering speed and quality.\n\n3. The experimental results in both static and dynamic scenes demonstrate the effectiveness of the proposed fragment pruning method in enhancing rendering efficiency without compromising visual fidelity.", "weaknesses": "1. The experimental section of the paper includes only Mini-Splatting [1] as a state-of-the-art Gaussian primitive pruning method among the baselines in static scenes' evaluations, while other comparisons are based on raw rendering pipelines. In dynamic scenes' evaluations, there no comparison with any state-of-the-art Gaussian primitive pruning techniques. While this comparison demonstrates the effectiveness of the proposed method, it lacks a demonstration of its advancement over other state-of-the-art Gaussian primitive pruning methods. Including more state-of-the-art Gaussian primitive pruning methods could enhance the persuasiveness of the evaluation.\n\n2. The baseline selection of the static scenes is unreasonable. In the experimental section (Table 2), among the four rendering pipelines, only one is rasterization-based radiance field rendering method. Since the focus of this paper is on pruning based on 3D Gaussian Splatting (3DGS) representation, the comparison should primarily involve rasterization-based radiance field rendering methods similar to 3DGS.\n\n[1] Guangchi Fang and Bing Wang. Mini-splatting: Representing scenes with a constrained number of gaussians. arXiv preprint arXiv:2403.14166, 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for raising questions regarding our baseline choices! We provide the following clarifications:\n\n---\n**W1 & Q2: No comparison with other state-of-the-art Gaussian pruning pipelines on static scenes.**\n\nFollowing your suggestion, we have added the comparison with additional Gaussian primitive pruning methods on the Mip-NeRF-360 dataset in the table below.\n\n| Method                      | PSNR | SSIM  | LPIPS | FPS (On Orin NX) |\n|-----------------------------|------|-------|-------|------------------|\n| LightGaussian [35]           | 27.28| 0.805 | 0.243 | 28               |\n| Compact 3D Gaussian [32]     | 27.08| 0.798 | 0.247 | 24               |\n| LP-3DGS [a] (RadSplat Score)| **27.47**| 0.812 | 0.227 | 24               |\n| Mini-Splatting [19]          | 27.34| **0.822** | 0.217 | 35          |\n| Mini-Splatting [19] + Ours       | 27.38| **0.822** | **0.209** | **54**      |\n\n\n\nWe observe that (1) among all prior Gaussian primitive pruning methods, Mini-Splatting [19] achieved the best trade-off between rendering quality and speed, and (2) our method, which performs pruning at the fragment level, further improves both the rendering quality and speed of Mini-Splatting [19]: +0.04 PSNR and 1.54× faster rendering speeds.\n\n[a] LP-3DGS: Learning to Prune 3D Gaussian Splatting, Arxiv 2024\n\n\n---\n**W2 & Q2: Lack of comparison with other Gaussian primitive pruning works on dynamic scenes.**\n \nWe would like to clarify that, to the best of our knowledge, existing Gaussian primitive pruning works focus exclusively on *static* 3D Gaussians. We are not aware of any work that performs pruning on dynamic 3D Gaussians. Our work is the first post-processing pruning framework designed to improve rendering speed on dynamic scenes. Consequently, our experiments on dynamic scenes were conducted on top of RT-4DGS [29], the state-of-the-art open-source 3DGS for dynamic scenes in terms of rendering quality and speed trade-off. As shown in Table 3 of our paper, our method achieved 1.57× faster rendering speeds with a 0.03 PSNR improvement compared to RT-4DGS [29].\n\n\n\n---\n**Q1: Why did you primarily compare with neural radiance field methods instead of rasterization-based radiance field rendering as your baseline?**\n\n\nWe did not primarily compare our work with rasterization-based methods due to their significantly lower rendering quality (e.g., more than 3 PSNR lower, as shown in the table below) compared to NeRF-based methods and native 3DGS. This baseline selection strategy aligns with that used in native 3DGS. Meanwhile, following your suggestion, we have included a comparison with additional rasterization-based methods in the table below.\n\n| Method                        | PSNR  | SSIM  | LPIPS | FPS (On Orin NX)           |\n|-------------------------------|-------|-------|-------|----------------------------|\n| MobileNeRF [14]               | 21.95 | 0.470 | 0.470 | 60                         |\n| NeRF2Mesh [b] (render w/ mesh) | 22.36 | 0.493 | 0.478 | 60                         |\n| NeRFMeshing [c]               | 23.59 | -     | -     | Code not released          |\n| Mini-Splatting [19]           | 27.34 | **0.822** | 0.217 | 35                         |\n| Mini-Splatting [19] + Ours    | 27.38 | **0.822** | **0.209** | 54                   |\n\n\nWe can see that despite the faster rendering speeds (> 60 FPS), the rasterization-based method fails to achieve the same rendering quality as the native 3DGS. In contrast, our method not only achieves approximately 60 FPS rendering speeds but also slightly improves the rendering quality (+0.04 PSNR).\n\n[b] Delicate Textured Mesh Recovery from NeRF via Adaptive Surface Refinement, ICCV 2023 \n\n[c] Distilling Neural Radiance Fields into Geometrically-Accurate 3D Meshes, 3DV 2024", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bVAMzGpxrz", "reviewer": "Reviewer_7Aqo", "summary": "This manuscript proposes a differentiable pruning strategy on 3D Gaussian splatting by performing the pruning stage after full optimization and continuing the optimization of the Gaussians with additional parameters that determine when to prune the Gaussians when they splat to 2D ( 2D pruning instead of the normal 3D pruning [19]). The proposed method gains marginal rendering speed compared to 3DGS on standard novel view synthesis benchmarks.", "questions": "inital Rating week reject\n\nReason: The work provides an alternative Gaussian Splatting pruning based on 2D fragments instead of 3D pruning [19], the proposed alternative is a major engineering feature that lacks technical insight, with marginal rendering speed improvement compared to the slow optimization ( twice ). The scope of the work is limited to the current 3DGS and does not extend beyond that hence I am not convinced the paper is ready for publication at NeurIPS\n\nfinal rating: borderline accept \nThe authors have addressed many of my concerns ( see rebuttal and answers below )", "limitations": "N/A", "rating": 5, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- Pruning 3D Gaussians splatting is a relevant topic of research with several works trying to address it.  \n- The code is provided", "weaknesses": "- The work suffers limited novelty. The proposed profiling, adding fragment pruning compares to the naive Gaussian pruning [19], and the resulting marginal improvement seems like a great engineering work that is not most suitable for a Neurips paper. The scope is limited such that there is no technical insight that can be used elsewhere in other 3D computer vision research areas based on the discussions provided in this paper. Next year when people stop using 3D Gaussian Splatting and start using something else ( as they did with NeRFs), how will this research be relevant? \n- The pipeline requires **TWO** full scene optimizations, one with the native 3DGS and the other with adding the differentibe fragment pruning. This added cost is not properly reflected in the results in Table 1 and Table 2. , which is not very scientific. The gained rendering speed is also not substantial enough to justify the cost in my opinion. \n- Why is there a gain in PSNR after pruning? Does this seem counterintuitive ?! The naive optimization can just easily reduce the opacity to 0 and get the same results instead of pruning it.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the issues you raised!  We address your concerns below:\n\n---\n**W1: Limited novelty and scope.**\n\nFirst, we humbly clarify that this work does deliver new technical insight, which is “**reducing fragment-level redundancy, in addition to commonly used primitive-level redundancy, enables a better trade-off between the rendering speed and quality**.” In addition, compared to existing 3DGS pruning works, which prune entire Gaussian primitives, we are the first to explore redundancy at the fragment level. Hence, our approach is complementary to existing works and serves as an effective plug-in module of easy use.\nAs such, we believe our insight and technique could extend beyond 3D Gaussian Splatting to other works that utilize a rasterization-based rendering pipeline, such as point clouds [a] and surface splatting [b,c].\n\nSecond, 3D Gaussian-based rendering techniques have not only achieved superior performance in scene reconstruction and rendering but have also been successfully applied to various other 3D vision-related applications, such as animatable human avatars [d], scene editing [e], open vocabulary querying of dynamic scenes [f], and even physical simulation [g]. Given their success in these downstream tasks, we believe that fully optimizing the efficiency of the 3D Gaussian-based rendering pipeline is a crucial and timely research question, which can largely facilitate the deployment of these 3D Gaussian-powered applications on edge devices like AR/VR headsets and smartphones. \n\nGiven the promise of 3DGS, our technique can serve as a plug-and-play add-on, allowing faster rendering speed without sacrificing rendering quality, thus promising to benefit the entire 3D community. We demonstrate this by extending our work to 4D Gaussian Splatting [29] (see Tab. 3 of our paper) and 3DGS-MCMC [h] (attached at the end of our response).\n\n\n[a] TRIPS: Trilinear Point Splatting for Real-Time Radiance Field Rendering, Eurographics 2024\n\n[b] Surface Splatting, SIGGRAPH 2001\n\n[c] Differentiable surface splatting for point-based geometry processing, ToG 2019\n\n[d] D3GA - Drivable 3D Gaussian Avatars, Arxiv 2023\n\n[e] GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splatting, CVPR 2024\n\n[f] LangSplat: 3D Language Gaussian Splatting, cvpr 2024\n\n[g] PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamics, CVPR 2024\n\n[h] 3D Gaussian Splatting as Markov Chain Monte Carlo, Arxiv 2024\n\n---\n**W2: Extra training cost not reflected in Tab. 1 and 2.**\n\nThank you for the suggestion! For Tab. 1 in our paper, we would like to clarify that it presents the profiling experiments conducted on top of existing works, thus incurring no additional training cost. For Tab. 2, we have summarized the corresponding additional training cost (measured on an RTX A5000 GPU) of our method compared to the baselines, as shown in the table below.\n\n| Dataset   | On top of Unpruned 3D Gaussians [16] | On top of Pruned 3D Gaussians [19]       |\n|-|-|-|\n| Mip-NeRF 360 [6]| 7.68 min (17% of [16]’s training time)| 3.43 min (20% of [19]’s training time) |\n| Tanks & Temples [20]| 4.35 min (16% of [16]’s training time)| 2.17 min (19% of [19]’s training time) |\n| Deep Blending [23]| 7.11 min (19% of [16]’s training time)| 3.41 min (21% of [19]’s training time) |\n\n\nWe can see that the additional training cost adds less than 25% overhead compared to the baselines, as we optimize opacity and truncation thresholds for less than 30% of the iterations in the second training stage. Furthermore, if the goal is to achieve rendering quality on par with the baselines, we can further reduce the number of iterations in the second stage of our method (e.g., reducing from 5,000 to 1,000 iterations), achieving 1.7× higher FPS while incurring only 3.5% additional training cost and maintaining the same rendering quality (i.e., PSNR/SSIM) on the Mip-NeRF 360 dataset compared to native 3DGS. We will include this discussion in the final version of our paper.\n\n---\n**W3: Why increased PSNR after pruning? Naive optimization: easily reduce the opacity to 0 instead of pruning it.**\n\nGood question! In a nutshell, the PSNR gain achieved by our work stems from the ability of a Gaussian function with a cutoff to better fit sharp signals in real-world scenes compared to the native Gaussian function, as shown in Fig. 1 of the global 1-page response. Specifically, we would like to clarify the following two points:\n\n**P1: Enhancing the Gaussian function with learnable thresholds increases its ability to fit signals.**\nAs illustrated in Fig. 1(a), enhancing each Gaussian function with a learnable threshold essentially gives each Gaussian a flexible cutoff range. In Fig. 1(c), we show that the truncated Gaussian function better fits various signals as compared to the native Gaussian functions, thanks to the enhanced capability of the learnable threshold/cutoff. Since sharp signals widely exist in real-world scenes and are the primary motivation behind existing anti-aliasing works [6,7], better fitting these signals using a Gaussian with a cutoff, as in our method, improves the PSNR by an average of 0.16 on the Tanks & Temples dataset, as shown in Tab. 2 of our paper.\n\n**P2: Mathematically, reducing the opacity of Gaussians is NOT equivalent to fragment pruning.**\nAs illustrated in Fig. 1(a) and Fig. 1(b), we demonstrate that fragment pruning (i.e., a learnable cutoff of the Gaussian function) is not mathematically equivalent to lowering the opacity (i.e., a global magnitude factor) of a Gaussian function. The former impacts only the Gaussian value around the cutoff threshold, while the latter reduces the magnitude of the Gaussian function globally. Therefore, the Gaussian with cutoffs adopted in our method can better fit sharp signals, such as square signals, compared to “easily reduce the opacity to 0”.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear Area Chairs and Reviewers,

We would like to express our gratitude to all the reviewers for their time and effort in providing valuable feedback. Your positive and constructive comments on our paper, particularly regarding its novelty and practical applications, are greatly appreciated. It is gald to see that the concept of fragment pruning and its potential impact has been well-received.

In response to the inquiries requesting additional experiments and further clarifications, we have supplied the requested experiments and provided detailed clarifications, summarized below.

**The following experiments have been provided:**
1. **Error Bars in Latency Profiling:** Included in Fig. 4(b) of the attached PDF.
2. **Extending Fragment Pruning Approach to GS-MCMC:** Results provided in our response to Reviewer ytrw’s W9.
3. **Regularizing the Scale of Gaussian Primitives:** Results included in our response to Reviewer ytrw’s W6 and Fig. 3(a) of the attached PDF.
4. **Benchmarking with Fixed Truncation Threshold:** Results provided in our response to Reviewer RCBw’s W1.
5. **Alternating Optimization Between Truncation Threshold and Other Parameters:** Results included in our response to Reviewer RCBw’s W2.
6. **Additional Visualization of Fragment Density:** Visualization included in Fig. 2 of the attached PDF.
7. **Overfitting Training on All Parameters of 3D Gaussian:** Results included in Fig. 3(c) of the attached PDF.
8. **Quantifying the Training Overhead of Our Proposed Approach:** Results included in the attached PDF.

**Questions Clarified:**
1. **Connection with Two-Stage NeRF to Mesh Framework:** Discussed in our response to Reviewer ytrw’s W4.
2. **Automatic Discovery of the Optimal Threshold and Fragment Density Behavior:** Explained in our response to Reviewer ytrw’s W5.
3. **Choice of Baselines for Benchmark on Static Scenes:** Clarified in our response to Reviewer TxBU’s W1 and Q1.
4. **Choice of Baselines for Benchmark on Dynamic Scenes:** Clarified in our response to Reviewer TxBU’s Q2.
5. **Novelty and Insights of Our Work and Extension to Other Research Areas:** Discussed in our response to Reviewer 7Aqo’s W1.
6. **Justification of Additional Training Costs in Terms of Performance Gains:** Explained in our response to Reviewer 7Aqo’s W2.
7. **Comparison of Reducing Opacity to Zero with Our Proposed Approach:** Clarified in our response to Reviewer 7Aqo’s W3.

We are open to providing further details if any points are still unclear. We would appreciate it if you could review our rebuttal and hope the new experiments and clarifications address your concerns. Please let us know if our responses do not resolve any of your concerns so that we can further clarify.

Best Regards,
Authors of Paper 13442

Meta-review: Following the rebuttal, this paper received all-accept leaning scores of borderline accept, borderline accept, weak accept, weak accept.

The AC concurs with the reviewers and suggests acceptance of this paper as a poster.
Decision: Accept (poster)