LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPS
Authors: Zhiwen Fan
Kevin Wang
Kairun Wen
Zehao Zhu
Dejia Xu
Zhangyang Wang
Venue: NeurIPS 2024 spotlight (2024)
Keywords: Gaussian Splatting
Efficient Rendering
Primary area: machine_vision
TL;DR: LightGaussian effectively compresses 3D Gaussians while enhancing rendering speeds

Abstract: Recent advances in real-time neural rendering using point-based techniques have enabled broader adoption of 3D representations. However, foundational approaches like 3D Gaussian Splatting impose substantial storage overhead, as Structure-from-Motion (SfM) points can grow to millions, often requiring gigabyte-level disk space for a single unbounded scene. This growth presents scalability challenges and hinders splatting efficiency. To address this, we introduce LightGaussian, a method for transforming 3D Gaussians into a more compact format. Inspired by Network Pruning, LightGaussian identifies Gaussians with minimal global significance on scene reconstruction, and applies a pruning and recovery process to reduce redundancy while preserving visual quality. Knowledge distillation and pseudo-view augmentation then transfer spherical harmonic coefficients to a lower degree, yielding compact representations. Gaussian Vector Quantization, based on each Gaussian’s global significance, further lowers bitwidth with minimal accuracy loss. LightGaussian achieves an average 15 times compression rate while boosting FPS from 144 to 237 within the 3D-GS framework, enabling efficient complex scene representation on the Mip-NeRF 360 and Tank & Temple datasets. The proposed Gaussian pruning approach is also adaptable to other 3D representations (e.g., Scaffold-GS), demonstrating strong generalization capabilities.

LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPS
Zhiwen Fan ; Kevin Wang',, Kairun Wen', Zehao Zhu', Dejia Xu', Zhangyang Wang'
'The University of Texas at Austin XXMU
(zhiwenfan, kevinwang 1839, atlaswang www.:/awwwwawa.au.saaaaaaa...... .dd
Project Website: https://ighttaussimn www.gggassiiiniiihuubbb. i0
Ours (6.5M points, 0.916)
Ours (45MB)
mip-NeRF360 (8. (8.6MB)
(782MB)
INGP (48MB)
0.70
SSIM
VO NeRF (63MB)
Plenoeel (2.1G8)
0.40
3D-GS (1.4M poin1s8.917)
50
100
150
200
crame-per-second (Render Speed)
Figure 1: Compressibility and Speed: LightGaussian compacts 3D Gaussians, reducing storage from 782MB to 45MB and boosting FPS from 144 to 237, while maintaining visual quality.
Abstract
Recent advances in real-time neural rendering using point-based techniques have enabled broader adoption of 3D representations. However, foundational approaches like 3D Gaussian Splatting impose substantial storage overhead. Structure-from Motion (SfM) points can grow to millions, often requiring gigabyte-level disk space for single unbounded scene. This growth presents scalability challenges and hinders splatting efficiency. To address this, we introduce LightGaussian, method for transforming 3D Gaussians into
compact format. Inspired by Network Pruning, LightGaussian identifies Gaussians with minimal global significance on scene reconstruction, and applies a pruning and recovery process to reduce redundancy while preserving visual quality. Knowledge distillation and pseudo-view augmentation then transfer spherical harmonic coefficients lower degree. yielding compact representations. Gaussian Vector Quantization, based on each Gaussian's global significance, further lowers bitwidth with minimal accuracy loss. LightGaussian achieves an average 15x compression rate while boosting FPS from 144 237 within the 3D-GS framework, enabling efficient complex scene representation on the Mip-NeRF 360 and Tank & Temple datasets. The proposed Gaussian pruning approach is also adaptable other 3D representations (e.g., Scaffold-GS), demonstrating strong generalization capabilities.
1 Introduction
Novel view synthesis (NVS) aims to generate realo-re images of 3D scene from unobserved viewpoints, given set of calibrated multi-view images. This capability has widespread applications
ZZ. Fan and K. Wang contributed cqually; Z. Fan is the Project Lead
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

in virtual reality M. augmented reality 2), digital twins [3], and autonomous driving 4 Neural Radiance Fields (NeRFs) 75 have shown promise in 3D modeling from multi-view images by mapping 3D locations and view directions to view-dependent color and volumetric density, with pixel intensity rendered through volume rendering []. However, NeRF and its variants face limitations in rendering speed, limiting their deployment in real-world scenarios. To address this, voxel- representations [9H13,, hash grids [14], and neural light fields [15] have been developed. Despite improvements, these methods often compromise between quality and speed.
Recent progress in point-baser based 3D Gaussian Splatting (3D-GS) [T6] has enabled real-time rendering with realiste quality for complex scenes. By representing the scene with explicit 3D Gaussians and using splatting technique [17]. 3D-GS balances speed and quality, making it suitable for large-scale scenarios like digital twins and autonomous driving. However, point-based methods incur high storage costs, as each point and its attributes require independent storage. Additionally, heuristic densification of sparse SfM points into dense Gaussians often results in overparameterizztion, leading to excessive storage and slower rendering speeds. For instance, typical unbounded 360-degree scene in 3D-GS may require over IGB of storage (e.g.. 1.4GB for the Bicycle scene).
In this paper, we address storage and rendering speed issues by developing a compact representation that retains the original rendering quality. The heuristic densification process in 3D-GS results in significant redundancy. Our method, LightGaussian, reduces redundancy by targeting both Gaussian count (N) and feature dimension (F) through a comprehensive pipeline:
To reduce Gaussian count (N), we propose a Gaussian Pruning and Recovery step, identi- fying and removing Gaussians with minimal impact visual quality, followed by recovery to ensure smooth adaptation.
For compressing features (F), we introduce an SH Distillation process to compact higher- degree spherical harmonic (SH) coefficients, supported by pseudo-view augmentation. Additionally, we employ Vector Quantization (VQ) to adaptively select codebook of Gaus- sian attributes (e.g, positions, scales, and rotations), reducing precision for less significant features and applying quantization-aware fine-tuning to maintain quality.
In summary, LightGaussian achieves substantial compression (e.g.. from 782MB to 45MB) while maintaining visual fidelity (SSIM decrease of only 0.007 on Mip-NeRF 360). Rendering speed also improves, reaching over 200 FPS on complex scenes, Our Gaussian Pruning and Recovery method generalizes well, enhancing performance across different 3D Gaussian formats, such as Scaffold-GS.
2 Related Works
Efficient 3D Scene Representations. Neural radiance fields (NeRF) S use multi-layer perceptrons (MLPs) to represent scenes, setting new standards for view synthesis quality. However, NeRF models face challenges with slow inference speeds, limiting practical use. Efforts to address this
have explored ray re-parameterizations 16 77. explicit spatial data structures 21 9 14 [3)
caching and distillation techniques [55] 24 and ray-based representations 25 26]. Nevertheless,
achieving real-time rendering remains challenging for NeRF methods, especially for large-scale scenes where multiple queries per pixel hinder performance.
Point-based representations, like Gaussians, have been explored in various applications, such as shape reconstruction (27), molecular modeling (28,, and point-cloud replacement (29, as well as in
shadow 30 and cloud rendering [31] Pulsar 32 demonstrated efficient sphere rasterization, while
3D Gaussian Splatting (3D-GS) 16 applies anisotropic Gaussians [3] with tile-based sorting to
achieve real-time speed and quality comparable to MLP-based methods like Mip-NeRF 360 (7).
Despite its strengths, 3D-GS has high storage demands due to the extensive attributes stored with each Gaussian, often requiring gigabytes per scene and hindering rendering efficiency. Recent concurrent works address these issues by using region-basee based vector quantization [34], K-means codebooks (35). view-direction exclusion [36), or learned binary masks and grid-based neural networks instead of spherical harmonics (SHs) [37] to reduce model size.
Pruning and Quantization. Model pruning reduces neural network complexity by removing non- significant parameters, balancing performance and resource use. Unstructured 38 and structured

---

High degreeii cive
Calcabation
Raterber
Sipilf ane Seore
Vier graddeee
Fondrcomtio
Raturbor
Te Reconio Hipaati
Law-degrce ene
Attributss
Multi-view Images
Gaussian Quantization
3D Gaussians
Gaussian Prune & Recovery
SH Distillation
Figure Pipeline of LightGaussian. 3D Gaussians are optimized from multi-view images and SfM points. LighiGaussian calculates each Gaussian's global significance on training data, pruning those with the least significance. Next, SH coefficients are distilled into a compact format using synthesized pseudo-views, Finally, vector quantization, including codebook initialization and assignment, reduces model bandwidth.
pruning 39 40] eliminate parameters at the weight level and neuron/channel levels, resulting in smaller, more efficient architecture. Iterative magnitude pruning (IMP), where low-magnitude weights are progressively removed, has proven effective in methods like lottery ticket rewinding [41 42
compresses data by representing it with discrete codebook entries (tokens), using mean square error (MSE) to select the closest match in the codebook for each data vector. Prior studies 444 have shown that learning discrete, compact representations enhances visual understanding and model robustness. VQ has thus been widely applied in image synthesis [47)
Vector quantization (VQ) 43
text-to-image generation
48 and novel view synthesis [88
49
50
51 54 15] trains smaller student model by transferring knowledge from larger teacher model 555] In 3D vision, KD has been applied to neural scene representations, leveraging view renderings to incorporate 2D priors. For example, DreamFusion (56) and NeuralLift-360 [57] use pre-trained diffusion models for 3D generation, while models like DFF [58], NeRF-SOS [59]. INS [60], and SA3D [61] distill 2D image feature extractors 3D tasks. KD has also been central to compressing scene representation models [15 24
Knowledge Distillation. Knowledge Distillation (KD)
3 Methods
Overview. The LightGaussian framework is illustrated in Fig. The 3D-GS model trained on multi-view images, initialized from SIM point clouds, and expanded to millions of Gaussians to represent the scene comprehensively. Our pipeline then processes the 3D-GS model into a compact format using Gaussian Prune and Recovery to reduce the number of Gaussians, SH Distillation to climinate redundant SHs while retaining key lighting information, and Vector Quantization to store Gaussians with lower bit-width.
3.1 Background: 3D Gaussian Splatting
3D Gaussian Splatting (3D-GS) [16] is explicit point-based 3D scene representation, utilizing Gaussians with various attributes to model the scene. When representing a complex real-world scene, 3D-GS is initialized from a sparse point cloud generated by SfM, and Gaussian Densification is applied to increase the Gaussian counts that are used for handling small-scale geometry insufficiently covered and over reconstruction. Formally, each Gaussian is characterized by a covariance matrix Σ and center point X, which is referred to as the mean value of the Gaussian:
G(X) = e -x-'x='''
RSSTR! RT
where Σ can be decomposed into a scaling matrix and rotation matrix R.
The complex directional appearance is modeled by an additional property, Spherical Harmonics (SH), with coefficients, fc & 1, 2......, where D represents the number of coefficients of SH with degree D. A higher degree equips 3D-GS with better capacity to model the view- dependent effect but causes significantly heavier attribute load.
When rendering 2D images from the 3D Gaussians, the technique of splatting 62
is employed
for the Gaussians within the camera planes. With viewing transform denoted as W and the Jacobian
3

---

PDF
1.0
15.0
CDF
12.5
0.8
PD
10.0
0.6
PSNR: 27.2 (0-651
0.4
5.0
0.2
0.0
0.0
0.2
0.6
1.0
PSNR: 10.3 POInt
opacity value
Figure 3: Zero-shot Opacity-based Pruning. A significant number of Gaussians exhibit small opacity values (top). Simply utilizing Gaussian opacity as an indicator for pruning the least important Gaussians results in the rendered image losing intricate details (bottom), with the PSNR dropping from 27.2 to 25.3. This has inspired us to find better criteria to measure global significance in terms of rendering quality. The accumulated Probability Density Function(PDF) is equal 1.
of the affine approximation of the projective transformation represented by J, the covariance matrix E' in camera coordinates can be computed E' = J JWEWT J Specifically, for each pixel, the color and opacity of all the Gaussians are computed using the Gaussian's 's representation Eq. The blending of N ordered points that overlap the pixel is given by the formula:
C= Σ C;O0, (1 IEN
Here. c, ag represents the view-dependent color and opacity, calculated from 2D Gaussian with covariance Σ multiplied by an optimizable per-3D Gaussian's opacity. In summary, each Gaussian point is characterized by attributes including: position X R3, color defined by spherical harmonics coefficients SH E g(++1>² 3 (where & represents the degrees of freedom), opacity a E IR, rotation factor R R4, and scaling factor S R3
3.2 Gaussian Pruning & Recovery
Gaussian densification [16. which clones and refines the initial SfM point cloud, enhances small-scale geometry and detailed scene appearance by improving coverage. While this approach significantly boosts reconstruction quality, it increases the number of Gaussians from thousands millions after optimization, resulting in substantial storage demands Inspired by neural network pruning techniques (63) that remove less impactful neurons while preserving overall performance, we propose a tailored pruning strategy for 3D Gaussian Splatting to reduce pee-pparameeerized points while maintaining accuracy. Identifying redundant yet recoverable Gaussians is essential to our approach. However, pruning Gaussians based on simple criteria (e.g., point opacity) risks degrading modeling performance, may eliminate essential scene details, as shown in Fig. 3
Global Significance Calculation. Inspired by Eq. 2. we go beyond using Gaussian opacity alone to assess significance by evaluating 3D Gaussians within the view frustum, projecting them onto the camera viewpoint for rendering. The initial significance score of each Gaussian (denoted as G(X, )) can then be quantified based on its contribution to each pixel (ray, r;) across all training views using the criteria (G(()) ;), r:) whether they intersect or not. Consequently, we iterate over all training pixels calculate the hit count of each Gaussian. The score is further refined by the adjusted 3D Gaussian's 's volume 7 ?EE) and 3D Gaussian's 's opacity J. as they all contribute for the rendering formula. The volume calculation equation of the j-th 3D Guassian is V(E,) 1 rabe, where abc are the 3 dimensions of Scale (S). We also consider the transmittance T calculated with one subtract all the 3D Gaussian's opacity the ray hit before the j-th 3D Gaussian, T [ ] (1- as). Finally, the global significance score can be summarized as:
MHW
GS,
Σ
(GG(X)).rd) .0. T TE)(E)
(3)

---

where is the Gaussian index, denotes a pixel, and M, H. and W represent the number of training views, image height, and width, respectively. is an indicator function that determines whether Gaussian intersects given ray. However, using Gaussian volume alone tends to overemphasize background Gaussians, leading to excessive pruning of Gaussians modeling fine geometry. Thus, we propose more adaptive approach to measuring volume dimensions:
7 (2)
WVorm) )
V(E)
V horm
min
maxx90
Here, the calculated Gaussian volume is firstly normalized by the 90% largest of all sorted Gaussians, clipping the range between 0 and I, to avoid excessive floating Gaussians derived from vanilla 3D-GS. The / is introduced to provide additional flexibilityy.
Gaussian Co-adaptation. We rank all Gaussians by their global significance scores to quantitatively guide pruning of the lower-ranked Gaussians. The remaining Gaussians are then jointly adapted by fine-tuning their atributes-witho additional densífication- offset the minor loss from pruning. This adaptation is performed using photometric loss on the original training views, aligned with 3D-GS training for 5,000 iterations.
3.3 Distilling into Compact SHs
In the uncompressed Gaussian Splat representation, substantial 81.3 percent of the feature dimension is occupied by Spherical Harmonics (SH) coefficients, requiring (45+3) floating-pon values per splat. Directly reducing the SH degree can save disk space but also diminishes surface "shininess" and affects specular reflections.
To balance model size with scene quality, we introduce knowledge distillation approach. Knowledge distilled from a teacher model with full-degree SHs to student model with truncated, lower-degree SHs. Supervision is based on the difference in predicted pixel intensities between the two models, with images synthesized at camera positions by sampling around each training view according to a Gaussian distribution:
NW
Edistill
Σ
C teachher (r:; [R]t]) - C student (re; [R(t)]
(5)
HW
Eseuddo tuain + +N(,00
(6)
where R and f denote rendering camera rotation and position, "pecdoo and Euain represent the newly synthesized and training camera positions, respectively. / denotes Gaussian distribution with mean O and variance o², which is added to the original position to generate the new position.
3.4 Gaussian Attributes Vector Quantization
Vector quantization (VQ) clusters voxels into compact codebooks, enabling high compression rates. However, quantizing Gaussian attributes in a point-based, inherently non-Euclidean representation poses notable challenges. Our empirical findings indicate that applying quantization across all elements Neements-especiallyy for attributes like position, rotation, and results in significant accuracy losses and a marked reduction in precision when represented discretely.
We propose applying VQ to the Spherical Harmonics (SH) coefficients, based on the assumption that subgroup of 3d Gaussians typically exhibits similar appearance. Fundamentally, VQ segments the Gaussians g = {g1. ......, gN (here we apply it on SH) to the K codes in the codebook = (c1: C....... Ck ). where each g,, C. E Rd and K & N. means SH dimension. We aim to strike a balance between rendering quality loss and compression rate by leveraging the pre-computed significance score from Eq.3) Based on this, we apply VQ seleetively on the least significant elements in the Spherical Harmonies (SHs). Specifically, we initialize C via K-means, iteratively sample batch of G. associates them to the closest codes by euclidean distance, and update each e₁ via moving average rule: с₂ Aa C₁ + (1 Aa) 1/TT, EggER(e,) GS,- gj, where Tz Egg ER(CL) GS; the significance score (Eq. 3 which is assigned to the code vector C&. R(cx) the set of Gaussians associated to the k-th code. da = 0.8 represents the decay value, which is utilized to update the
5

---

Mip NeRI 360 Datasets
Tank and Temple Datasets
Methods
FPST
Size;
PSNK:
SSIMT
LPPPSL
FEST
Size.
PSNRT
SSIMT
LPPPS.
Plenoxels 12
6.79
2.1 GB
23,08
0.626
0.463
11.2
2.3
21,08
0.719
0.379
INGP-Big
Mip-NoRF 360
0.06
8.6MB
27.69
0.792
0.237
0.14
8.6MB
22.22
0.759
0.257
VQ VQ-DVGO [9
4.65
63MB
24.23
0.636
0.393
Compressed 3D-GS
34
152
28MB
27.03
0.802
0.238
202
17MB
23.54
0.838
0.189
Compuct 3D 37
128
48MB
27.08
0.798
0.247
185
39MB
23.32
0.831
0.201
41
3D-GS- 16
144
782MB
27.40
0.813
0.217
106
435MB
23.66
0.845
0.178
237
45MB
0.806
357
23.44
0.832
0.202
Table 1: Quantitative Comparisons in Real-world Large-scale Scenes: Voxel-based methods 12
14] exhibit insufficient capacity for representing large-scale scenes and are unable to achieve real-time performance. Mip-NeRF360 [7] produces the highest visual quality, but requires over 16 seconds to render a single image. Our method strikes a balance among FPS, model size, and rendering quality, achieving the best balance among efficient representations. For fair metric and visual comparison, re-trained 3D-GS and Compressed 3D-GS (34) using their original code and configurations on our platform (NVIDIA A6000 GPU), with results marked by *
Ground-ruth
RGB (3D-GS)
Residual Map (3D-GS)
RGB (Ouns)
Residual Map (Ours)
Figure 4: Visual Comparisons. We compare LightGaussian with the vanilla 3D-GS [6], displaying residual map between predictions and ground truth scaled from 0 127 emphasize differences. LightGaussian retains most specular reflections (yellow boxes) in its compact format, with slight change in lightness visible in the bottom white box. The residual maps illustrate discrepancies between rendered images and ground-truth RGB values, where darker areas indicate closer alignment. For full dynamic viewpoint comparison, please see our supplementary video.
code vector using moving average. We fine-tune the codebook for 5,000 iterations, while fixing the moussian-to-codebon mapping. We disable additional clone/split operations and leverage photometric loss on the training views. To preserve essential attributes, including Spherical Harmonics with higher global significance scores, along with Gaussian position, shape, rotation, and opacity, we skip VQ for these elements and store them directly in float16 format.
4 Experiments
4.1 Experimental Settings
Datasets and Metrics. We conduct comparisons using the scene-scale view synthesis dataset provided by Mip-NeRF360 (64), which comprises nine real-world large-scale scenes, including five unbounded outdoor and four indoor settings with complex backgrounds. In addition, we utilize the Tanks and Temples dataset [65]. comprehensive unbounded dataset, and select the same scenes as used in [16]. Performance metries on synthetic object-level datasets will be detailed in the supplementary materials. We report metrics including the peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and perceptual similarity as measured by LPIPS (66).
Compared Baselines. We compare our approach with methods suited for large-scale scene mod- [2], Mip-NeRF360 77 and 3D-GS [16]. Additionally, evaluate
cling, including Plenoxel

---

Expl
Model
FPST
Size
PSNRT
SSIMT
LPIPS.
[
Baseline (3D.GS [TG]
156.21
365MB
31.34
0.917
0.221
[2]
Gaussion Pruning
308.10
123MB
30.67
0.910
0.234
[3]
+ Co-adaptation
304.64
123MB
31.64
0.918
0.228
[4]
Compactness
311.58
81MB
30.32
0.904
0.238
[5]
photometric loss
302.20
81MB
31.42
0.916
0.234
[6]
+ Distillation Pscudo-views
302.89
81MB
31.48
0.917
0.231
17]
Codebook Quant.
301
31.09
0.918
0.236
18 VQ finetune
302.01
21MB
31.40
0.916
0.232
9
LightGaussian (Ours)
302.01
21MB
31.40
0.916
0.232
Table 2: Ablation studies on the Gaussian Pruning & Recovery, SH Compactness, and the Vector Quantization. Scene: Room. Zero-shot Gaussian pruning leads degraded rendering quality (#2), but Co-adaptation can recover most of the scene details (#3).. Directly eliminating high-order SH negatively affects the quality (#4), while distillation with pseudo-view helps to mitigate the gap (#5, #6). Codebook quantization further reduces the required model size and bandwidth (#7, #8).
Model
FPSt
Sizel
PSNRT
SSIM
LPIPS.
Baseline
156.21
365MB
31.34
0.917
0.221
Hit Count Only
301.52
123MB
28.16
0.886
0.261
+ Co-adaptation
303.43
123MB
30.13
0.912
0.238
x Opacity
310.29
123MB
30.27
0.909
0.239
+ Co adaptation
304.84
123MB
31.60
0.916
0.231
Opacity x >Voolume).
312.30
123MB
30.67
0.910
0.234
+ Co-adaptation
304.64
123MB
31.64
0.918
0.228
Table 3: Ablation study of the Gaussian Pruning & Recovery, by using different Gaussian attributes for computing its global significance score. By considering only the hit count of each Gaussian from training rays, the zero-shot pruning leads to inferior performance. Incorporating the opacity and volume drives us to a better criterion. The subsequent Gaussian Co-adaptation is used to recover most of the information loss from the pruning of redundant Gaussians.
Model
Sizel
PSNR1
SSIM
LPIPS
Baseline
80.99MB
31.48
0.917
0.231
+FP16
MR
31.35
0.914
0.232
+ VQ All att.
18.74MB
23.11
0.731
0.378
+ VQ All att.
GS
19.74MB
26.23
0.826
0.323
VQ SH.
21.10MB
30.68
0.907
0.244
VQ SH x
10MB
31.16
0.911
0.235
LightGaussian (w/ VQ Finetune)
20MB
31.40
0.916
0.232
Table 4: Ablation Study: Gaussian Attribute Vector Quantization (VQ). Quantizing all attributes to
FP16 achieves a smaller model, but applying VQ to all attributes degrades modeling accuracy. Using Global Significance (GS) to target less crucial Gaussians mitigates this loss. Some attributes (e.g.. scale) are sensitive to VQ. so we limit VQ to the SH coefficients. By combining VQ with significance scores, LightGaussian achieves an effective balance between model size and quality.
against efficient techniques like Instant-NGP [14), which uses a hash grid for storage efficiency, and VQ-DVGO [49], which extends DVGO D with voxel pruning and VQ for optimized representation.
Implementation Details. Our framework is implemented in PyTorch and integrates the differen- tiable Gaussian rasterization technique from 3D-GS [16] All performance evaluations are conducted on an A6000 GPU. In the Global Significance Calculation phase, we assign power value of 0.1 in Eq. 4 and proceed fine-tune the model for 5,000 steps during the Gaussian Co-adaptation process. For SH distillation, we downscale the 3-degree SHs to 2-degree, thereby reducing 21 elements for each Gaussian. This is further optimized by setting a to 0.1 in the pseudo view synthesis stage. In the Gaussian VQ step, the codebook size is configured to 8192, selecting SHs with the least 60% significance score for the vector quantization (VQ ratio), to balance the trade-off between compression efficiency and fidelity.
7

---

Size +
FPS
PSNR
SSIM 1
LPIPS
Scaffold-GS
173.60
152
27.96
0.8240
0.2075
Scaffold-GS + LighhGGausian
112.56
178
27.78
0.8187
0.2197
Table 5: LightGaussian removes redundant neural Gaussians in Scaffold-GS, reducing model size and improving rendering speed. All experiments were rerun our platform for fair comparison, with results averaged on the MipNeRP-360 dataset.
4.2 Experimental Results
Quantitative Results. We assess the performance of various methods for novel view synthesis, with quantitative metrics summarized in Tab. [ This includes efficient voxel-based NeRFs like Plenoxel [12] and Instant-NGP (14), the compact MLP-based Mip-NeRF360 [7]. vector-quantized
NeRF 49 and 3D Gaussian Splatting [6].
On the MeR-Ne dataset, MLP-based NeRF methods achieve competitive accuracy with compact representations but suffer from slow inference speeds (0.06 FPS), limiting their practicality. Voxel- based NeRFs improve rendering efficiency but still fall short of real-time performance; for example, Plenoxel requires 2. IB for single large-scale scene. In contrast, 3D-GS offers good balance between quality and speed but demands substantial storage per scene.
Our method, LightGaussian, exceeds existing techniques with rendering speeds over 200 enabled by efficient rasterization that prunes insignificant Gaussians. This approach reduces 3D Gaussian model redundancy, cutting storage from 782MB 45MB on Mip-NeRF360- 15x reduction. LighiGaussian also outperforms 3D-GS on the Tank & Temple datasets, nearly doubling rendering speed and reducing storage from 380MB to 22MB.
We conducted a comparative analysis of rendering results between 3D-GS and LightGaussian, focusing intricate details and background regions, shown in Fig. 4 Both 3D-GS and LightGaussian exhibit comparable visual quality, even in challenging scenes with thin structures, showing that LightGaussian effectively removes redundancy while preserving reconstruction fidelity.
Qualitative Results.
Generalization to Other Point-based Representations We further apply our Gaussian Pruning approach to Scaffold-SS 677, an advanced neural Gaussian-based 3D representation. In this method, we prune Gaussians that contribute minimally to scene reconstruction, reducing redundancy while preserving visual fidelity. Experiments the MipNeRF360 dataset demonstrate that pruning 80% of neural Gaussians increases the rendering speed from 152 to 178 FPS, as shown in Tab, S These results underscore LightGaussian's 's potential as effective optimization tool for other Gaussian- bassd representations.
4.3 Ablation Studies
We performed ablation studies on each component of our method to evaluate their individual impacts. Results show that Gaussian Pruning and Recovery effectively removes redundant Gaussians, retaining only those crucial for scene representation. SH Distillation reduces the Spherical Harmonics degree, simplifying the lighting model with minimal quality loss. Furthermore, Vector Quantization efficiently compresses the feature space, creating more compact representation. Combined, these modules substantially improve the overall efficiency and performance our framework.
Significance Criteria To identify the optimal criteria for measuring each Gaussian's 's global sig- nificance, we evaluated key characteristies: Gaussian-ray interaction count, Gaussian opacity, and functional volume. Tab. 3 Fillustraaes that integrating these three factors - by weighting the hit count with opacity and Gaussian volume- - yields the highest rendering quality post zero-shot pruning. A quick Gaussian Co-adaptation (the last row) which optimizes the remaining Gaussians, can effectively recover the rendering accuracy to its pre-prue pre-pruning level. The visual comparison rasterized images, before and after pruning, alongside the depiction of pruned Gaussians, is presented in Fig. 5
SH Distillation & Vector Quantization. Directly removing high-degree components in Spherical Harmonics (SHs) causes significant performance degradation compared to the full model (Exp #3 -

---

5.6M pper 0.856 (SSIM))
1.8M (tts 0.847 (SSM)
2.3M ppeap 0.863
0.7M gpsi 0.866
Before Pruning
Rasterized Residual
After Pruning
Figure 5: Visualization of Pruned Gaussians. We show the prumed Gaussians (middle) obtained by applying the proposed Gaussian Prune and Recovery. The residual is visualized by rasterizing the pruned Gaussians.
4), particularly with the loss of specular reflections across varying viewpoints. However, introducing knowledge distillation from the full model (Exp #4 5) allows for size reduction while preserving essential viewing effects. Additionally, incorporating pseudo-views during training (Exp #5 + 6) demonstrates the effectiveness of simulated views in enhancing the learning of specular reflections.
While applying VQ to all attributes yields poorer results (see Tab. 4p. this impact is mitigated by leveraging Gaussian Global Significance.
Exp ID
Madel setting
Model Size
FPS 1
PSNR 1
SSIM 1
LPIPS
[1]
GS
31.687
0.200
[2]
[1] + 2-degree SH compactness
140.16 MB
238
30.625
0.917
0.208
[3] + proposed distillation
0.201
[4] [1] + I-degree SH compactness
88.89 MB
249
29.173
0.900
0.223
[5] + proposed distillation
31.440
Table 6: Ablation with varying SH compactness: With a Gaussian pruning ratio of 66% and SH
reduced to degrees, LightGaussian nearly maintains rendering quality (SSIM drops slightly from 0.927 to 0.926). Reducing SH further to degree lowers SSIM 0.923.
450
0.80
0.94
SSIM Values
0.75
Quanti taaetize Ratio
0.92
0.92
350
0.70
0.90
0.90
300
SSIM
0.65
0.88
0.88
250
0.60
0.86
200
0.86
0.55
0.0
0.2
0.4
0.6
0.B
0.2
0.4
0.6
0.8
Gaussian Prune Ratio
Gaussian Ratio
(a). SSIM, FPS Prune Ratio
SSIM, Quantize Ratio VQ Ratio
Figure 6: Rendering performance comparison across different pruning levels and VQ ratios. Note that in subfigure (b), we use the VQ ratio to indicate the proportion SH parameters quantized to an 8192-codebook, while the (overall) quantization ratio reflects combined compression from VQ and FP32-to-FP16 bitwise quantization.
Degradation & Speed vs. Compression Ratio We investigate the interplay between rendering quality and speed across various model compression ratios. Specifically, adjust parameters such

---

as Gaussian Pruning & Recovery, SH Distilling, and VQ. As depicted in Fig 6 we note a marked decline in rendering quality when the pruning ratio reaches 70%, with a more rapid deterioration as the VQ ratio approaches 65%. In Tab 6 we observe that 2-degree SH compactness reduces the SSIM from 0.927 to 0.926, while 1-degree SH compactness further reduces it to 0.923.
Conclusion, Limitations, and Broad Impact
We present LightGaussian, novel framework that transforms heavy point-based representations into a compact format for efficient novel view synthesis. Designed for practical use, LightGaussian leverages 3D Gaussians to model large-scale scenes, effectively identifying and pruning the least significant Gaussians generated through densification. It achieves over 15x data reduction, boosts FPS to over 200, and minimally impacts rendering quality. Exploring zero-shot compression across various 3D-GS-based frameworks remains a promising direction for future research.
Broadly, LightGaussian's 's compact representation has the potential to democratize high-quality 3D content for applications in VR, AR, and autonomous driving by reducing resource demands, enabling more accessible and scalable deployment across industries. While we do not foresee any significant risk from this specific technique, 3D reconstruction may infringe on personal privacy when applied in public spaces or with drone footage, contradicting our ethical intentions.
References
[1] Jianmei Dai, Zhilong Zhang, Shiwen Mao, and Danpu Liu. A view synthesis-based 360 vr caching system over mec-enabled c-ran. IEEE Transactions Circuits and Systems for Video Technology, (0(10)33843-3855 2019.
[2] Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, and Noah Snavely. Stereo magnifi- cation: Learning view synthesis using multiplane images. arXiv preprint 1805.01700.0917, 2018.
[3] Luca De Luigi, Damiano Bolognini, Federico Domeniconi, Daniele De Gregorio, Matteo Poggi, and Luigi Di Stefano. Scannerf: scalable benchmark for neural radiance fields. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 816-825, 2023.
[4] Zirui Wu, Tianyu Liu, Liyi Luo, Zhide Zhong, Jianteng Chen, Hongmin Xiao, Chao Hou, Haozhe Lou, Yuantao Chen, Runyi Yang, et al. Mars: An instance-aware, modular and realistic simulator for autonomous driving. arXiv preprint 23v7 2023.
[5] Ben Mildenhall, Pratul Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing Scenes As Neural Radiance Fields for View Synthesis. Communications of the ACM, 65(1)999-166 2021.
[6] Jonathan Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin- Bruuala, and Pratul P Srinivasan. Mip-Nerf: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5855-5864 2021.
[7] Jonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan, and Peter Hedman. Mip-NeRF 360: Unbounded Anti AniisAliased Neural Radiance Fields. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5460- 5469. 2022.
[8] Robert A Drebin, Loren Carpenter, and Pat Hanrahan. Volume rendering. ACM Siggraph Computer Graphics, 22(4):65-74, 1988.
[9] Cheng Sun, Min Sun, and Hwann- Tzong Chen. Direct Voxel Grid Optimization: Super-Fast Convergence for Radiance Fields Reconstruction. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 5459-5469, 2022.
[10] Liao Wang, Jiakai Zhang, Xinhang Liu, Fuqiang Zhao, Yanshun Zhang, Yingliang Zhang, Minye Wu, Jingyi Yu, and Lan Xu. Fourier PlenOctrees for Dynamic Radiance Field Rendering in Real-Time. In Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pages 13524-13533 2022.
10

---

[11] Katja Schwarz, Axel Sauer, Michael Niemeyer, Yiyi Liao, and Andreas Geiger. Voxgraf: Fast 3d-Aware Image Synthesis With Sparse Voxel Grids. ArXiv Preprint 2XXiv:2006.06955, 2022.
[12] Alex Yu, Sara Fridovich- Keil, Matthew Tancik, Qinhong Chen, Benjamin Recht, and Angjoo Kanazawa. Plenoxels: Radiance Fields Without Neural Networks. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5491-5500, 2022.
[13] Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, and Hao Su. Tensorf: Tensorial radiance fields. In European Conference on Computer Vision, pages 333-350. Springer, 2022.
[14] Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant Neural Graphics Primitives With a Multiresolution Hash Encoding. ACM Transactions on Graphics (TOG), 41:1 15, 2022.
[15] Huan Wang, Jian Ren, Zeng Huang, Kyle Olszewski, Menglei Chai, Yun Fu, and Sergey Tulyakov. R21: Distilling neural radiance field to neural light field for efficient novel view synthesis. In European Conference on Computer Vision, pages 612 629. Springer, 2022.
[16] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering, ACM Transactions on Graphics (ToG), 42(4):1- 14, 2023.
[17] Georgios Kopanas, Julien Philip, Thomas Leimkühler, and George Drettakis. Point-based neural rendering with per-view optimization. In Computer Graphics Forum, volume 40, pages 29-43, Wiley Online Library, 2021.
[18] Alex Yu, Ruilong Li, Matthew Tancik, Hao Li, Ren Ng, and Angjoo Kanazawa. Plenoctrees for real-time rendering of neural radiance fields. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 5752-5761. 2021.
[19] Sara Fridovich- Keil, Alex Yu, Matthew Tancik, Qinhong Chen, Benjamin Recht, and Angjoo Kanazawa. Plenoxels: Radiance Fields Without Neural Networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5501-5510, 2022.
[20] Lingjie Liu, Jiatao Gu, Kyaw Zaw Lin, Tat-Seng Chua, and Christian Theobalt. Neural Sparse Voxel Fields. Advances in Neural Information Processing Systems, 33:1565 1-15663, 2020.
[21] Tao Hu, Shu Liu, Yilun Chen, Tiancheng Shen, and Jiaya Jia. EfficientNeRF Efficient Neural Radiance Fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12902-12911, 2022.
[22] Stephan J Garbin, Marek Kowalski, Matthew Johnson, Jamie Shotton, and Julien Valentin. Fast- nerf: High-fidelity neural rendering at 200fps. Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 14346-14355 2021.
[23] Peter Hedman, Pratul P Srinivasan, Ben Mildenhall, Jonathan T Barron, and Paul Debevec. Baking neural radiance fields for real-time view synthesis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5875-5884, 2021.
[24] Christian Reiser, Songyou Peng, Yiyi Liao, and Andreas Geiger. Kilonerf: Speeding up neural radiance fields with thousands of tiny mlps. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 14335-14345, 2021.
[25] Benjamin Attal, Jia-Bin Bin Huang, Michael Zollhoefer, Johannes Kopf. and Changil Kim. Learning Neural Light Fields With Ray-Space Embedding Networks. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19787-19797, 2022.
[26] Vincent Sitzmann, Semon Rezchikov, William T. Freeman, Joshua B. Tenenbaum, and Frédo Du- rand. Light Field Networks: Neural Scene Representations With Single-Evaluation Rendering. Advances in Neural Information Processing Systems, 34, 2021.
[27] Leonid Keselman and Martial Hebert. Flexible techniques for differentiable rendering with 3d gaussians, arXiv preprint 37X1v:2308.14737, 2023.

---

[28] James F Blimn. A generalization of algebraic surface drawing, ACM transactions on graphics (TOG), 133):235-25,, 1982.
[29] Benjamin Eckart, Kihwan Kim, Alejandro Troccoli, Alonzo Kelly, and Jan Kautz. Accelerated generative models for 3d point cloud data. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5497-5505, 2016.
[30] Manjushree Nulkar and Klaus Mueller. Splatting with shadows. In Volume Graphics 2001: Proceedings of the Joint IEEE TCVG and Eurographics Workshop in Stony Brook, New York, USA, June 21-22, 2001, pages 35-49. Springer, 2001.
[31] Petr Man. Generating and real-time rendering of clouds. In Central European seminar computer graphics, volume 1. Citeseer Castá-Papiernicke Slovakia, 2006.
[32] Christoph Lassner and Michael Zollhofer. Pulsar: Efficient sphere-based neural rendering. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 1440-1449, 2021.
[33] Matthias Zwicker, Hanspeter Pfister, Jeroen Van Baar, and Markus Gross. Ewa volume splatting. In Proceedings Visualization, 2001. VIS"OI., pages 29-538. IEEE, 2001.
[34] Simon Niedermayr, Josef Stumpfegger, and Rüdiger Westermann. Compressed 3d gaussian splatting for accelerated novel view synthesis. arXiv preprint 0xXi1:24001.223366 2023.
[35] KL Navaneet, Kossar Pourahmadi Meibodi, Soroush Abbasi Koohpayegani, and Hamed Pirsi- avash. Compact3d: Compressing gaussian splat radiance field models with vector quantization. arxiv preprint arXiv:2311.18159 2023.
[36] Yuanhao Cai, Yixun Liang, Jiahao Wang, Angtian Wang, Yulun Zhang, Xiaokang Yang, Zong- wei Zhou, and Alan Yuille. Radiative gaussian splatting for efficient x-ray novel view synthesis. In European Conference on Computer Vision, pages 283-299. Springer, 2025.
[37] Joo Chan Lee, Daniel Rho, Xiangyu Sun, Jong Hwan Ko, and Eunbyung Park. Compact 3d gaussian representation for radiance field. arXiv preprint arXiv:2311. 1368,,,,,,,, 2023.
[38] Yann LeCun, John Denker, and Sara Solla. Optimal brain damage. Advances neural information processing systems, 2, 1989.
[39] Yang He, Guoliang Kang, Xuanyi Dong, Yanwei Fu, and Yi Yang. Soft filter pruning for accelerating deep convolutional neural networks. arXiv preprint arXiv:1808. 100.666666. 2018.
[40] Sajid Anwar, Kyuyeon Hwang, and Wonyong Sung. Structured pruning of deep convolutional neural networks, ACM Journal on Emerging Technologies in Computing Systems (JETC), 13(3):1-18, 2017.
[41] Jonathan Frankle and Michael Carbin. The lottery ticket hypothesis: Finding sparse, trainable neural networks. arXiv preprint arXiv:1802: 03635, 2018
[42] Jonathan Frankle, Gintare Karolina Dziugaite, Daniel Roy, and Michael Carbin. Linear mode connectivity and the lottery ticket hypothesis. In International Conference on Machine Learning, pages 3259-3269. PMLR, 2020.
[43] Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017.
[44] Gabriella Csurka, Christopher Dance, Lixin Fan, Jutta Willamowski, and Cédric Bray. Visual categorization with bags of keypoints. In Workshop statistical learning in computer vision, ECCV, volume 1, pages 1-2. Prague, 2004.
[45] Robert Gray. Vector quantization. IEEE Assp Magazine, 1(2):4-99, 1984.
[46] Chengzhi Mao, Lu Jiang, Mostafa Dehghani, Carl Vondrick, Rahul Sukthankar, and Irfan Essa. Discrete representations strengthen vision transformer robustness. arXiv preprint arX1v:2111.10493, 2021.
12

---

[47] Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12873- 12883, 2021.
[48] Shuyang Gu, Dong Chen, Jianmin Bao, Fang Wen, Bo Zhang, Dongdong Chen, Lu Yuan, and Baining Guo. Vector quantized diffusion model for text-to-image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patfern Recognition, pages 10696-10706, 2022.
[49] Lingzhi Li, Zhen Shen, Zhongshu Wang. Li Shen, and Liefeng Bo. Compressing volumetric radiance fields to 1 mb. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattem Recognition, pages 4222-4231, 2023.
[50] Haoyu Guo, Sida Peng, Yunzhi Yan, Linzhan Mou, Yujun Shen, Hujun Bao, and Xiaowei Zhou. Compact neural volumetric video representations with dynamic codebooks. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
[51] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprini arXiv: 150.00253,, 2015.
[52] Zhiqiang Shen, Zechun Liu, Dejia Xu, Zitian Chen, Kwang-Ting Ting Cheng, and Marios Savvides. Is label smoothing truly incompatible with knowledge distillation: An empirical study, arXiv preprint arXiv:2104.00676. 2021.
[53] Victor Sanh, Lysandre Debut, Julien Chaumond. and Thomas Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108. 2019.
[54] Lin Wang and Kuk-Jin Yoon. Knowledge distillation and student-teacher learning for visual intelligence: A review and new outlooks. IEEE transactions on pattern analysis and machine intelligence, 44(6):3048-300 3068,, 2021.
[55] Cristian Buciluă, Rich Caruana, and Alexandru Niculescu-Mizil. Model compression. In Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 535-541, 2006.
[56] Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209. 14988, 2022.
[57] Dejia Xu, Yifan Jiang, Peihao Wang, Zhiwen Fan, Yi Wang, and Zhangyang Wang. Neurallift- 360: Lifting an in-the-wild 2d photo to a 3d object with 360 ('deg] views. arXiv preprint 1rX1v22211.16431, 2022.
[58] Sosuke Kobayashi, Eiichi Matsumoto, and Vincent Sitzmann. Decomposing nerf for editing via feature field distillation. Advances in Neural Information Processing Systems, 03:23311-233344 2022.
[59] Zhiwen Fan, Peihao Wang, Yifan Jiang, Xinyu Gong, Dejia Xu, and Zhangyang Wang. Nerf-sos: Any-view self-supervised object segmentation on complex scenes. arXiv preprint arXiv:2209.08776, 2022.
[60] Zhiwen Fan, Yifan Jiang, Peihao Wang, Xinyu Gong, Dejia Xu, and Zhangyang Wang. Unified implicit neural stylization. In European Conference on Computer Vision, pages 636-654. Springer, 2022.
[61] Jiazhong Cen, Zanwei Zhou, Jiemin Fang, Wei Shen, Lingxi Xie, Xiaopeng Zhang, and Qi Tian. Segment anything in 3d with nerfs. arXiv preprint arXiv:2304. M.Xi:22304.12008, 2023.
[62] Wang Yifan, Felice Serena, Shihao Wu, Cengiz Öztireli, and Olga Sorkine- Differen- tiable surface splatting for point-based geometry processing. ACM Transactions on Graphics (TOG), 3-14 2019.
[63] Song Han, Jeff Pool, John Tran, and William Dally. Learning both weights and connections for efficient neural network. Advances in neural information processing systems, 28, 2015.
13

---

[64] Jonathan T Barron, Ben Mildenhall, Dor Verbin, Pratul P Srinivasan, and Peter Hedman. Mip- nerf 360: Unbounded anti-aliased neural radiance fields. In Proceedings of the IEEE/CVP Conference Computer Vision and Pattern Recognition, pages 5470-5479, 2022.
[65] Arno Knapitsch, Jaesik Park, Qian-Yi Yi Zhou, and Vladlen Koltun. Tanks and temples: Bench- marking large-scale scene reconstruction. ACM Transactions on Graphics (ToG), 36(4):1-13, 2017.
[66] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unrea- sonable effectiveness of deep features as perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586-595, 2018.
[67] Tao Lu, Mulin Yu, Linning Xu, Yuanbo Xiangli, Limin Wang, Dahua Lin, and Bo Dai. Scaffold- gs: Structured 3d gaussians for view-adaptive rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20654 2066444 2024.
14

---

Appendix
Algorithm Explanation. We detail the procedures of LightGaussian in Algorithm | The trained 3D-GS [T6] features Gaussian location with a dimension of 3, Spherical Harmonics coefficients with a dimension of 48, and opacity, rotation, and scale, whose dimensions are 1, 4, and 3, respectively.
Algorithm The overall pipeline of LightGaussian
Initialize: Training view images I (I. R and their associated camera poses
{
Rxx4] Ti=1-
1: # Pre-Training 3D-GS 16
2: # G. with attributes XYZ, S11-3deg, Opacity, Rotation, Scale.
3: G {G, E [2 (+++++++++++))) ر < 3D-GS(I,
Initial 3D-GS: g
4: #Gaussian Pruning and Recovery; G g'
5: GS (G)
Calculate Global Significance Score GS
6: Ç" + PRUNE(S, GS)
P Prune Least Significant Ones of G. based on GS
7: g' < RECOVERY PP)
D Gaussian Recovery (Finetune g' on P)
8: #Distilling into Compact SHs: Reduce from 3-degree to 2-degree
9: g" < REDUCESH((")
> Reduce the SH degree SH Distillation
while Few Steps do
11: P. Sample View
Synthesize Pscudo Views
12:
I TEACHER (P.T g
Teacher render P Student render
13:
L. + STUDENT (P, g")
14:
VL + Loss(L..
15: g" <- ADAM(VL)
Backprop & Step
16: end while
17: #Vector Quantization.
18: 9" + VQ(g", GS)
VQ based on Significance Score VQ Finetuning
19: gT < VQFINETUNE("")
20: #Save Model.
21: Save optimized model g"d disk.
Specifically, we detail how we calculate each Gaussian's Global Significance Score in Algorithm
Algorithm 2 Global significance calculation for gaussians.
1: # G contains all gaussians with attributes XYZ, SH-3deg, Opacity, Rotation, Scale.
2: # P Sample cameras from training views.
3: function CaIGS(G, PP)
ScoreList list()
5: for all Gaussian G; in G do
6: score <-
b. Init Global Significance Init Hit Count Value
7: count f- 0
for all Pixel/Ray f. in RenderFunc(P)) do
9:
count GETHITCOUNT(G, r.)
D. Gaussian Hit count
10:
Score weighted by opacity/volume
11:
score +- score + count opacity. VOLUMEFUUN((sale)
b Eq.3 in draft.
12:
end for ssppessistappeend(scooeee
14:
end for
15:
return ScoreList
16:
end function
Post- Dost-Finetuning of Vector Quantization. Applying vector quantization (VQ) based on the sig- nificance score facilitates a reduction in the required codebook size compared to the full model. Nonetheless, this method results in decrease in accuracy as it groups the attributes into several clusters (codes). We observe that the SSIM decreases from 0.923 (#6) to 0.915 (#7), as illustrated in Table of the main draft. This observation motivates us to joint optimize all other attributes and the codes post the VQ. Specifically, by assigning a unique code to each Gaussian, we proceed to implement differentiable finetuning using photometric loss, which is similar to the methodology
15

---

RGB
RC
Residual
Exp
Exp #2
Exp #3
Exp #4
Residual RGB
Exp #6
Exp #7
Exp #8
Exp #9
Figure 7: Visual Comparisons for Ablation Study. We visualize the rendered RGB images and the residual map between the ground-truth image, aligned with the experiment ID shown in Tab. The final model (Exp #9) demonstrates close results to 3D-GS (Exp #1), while the Gaussian Co-adaptation, along with SH distillation, almost completely mitigates the information loss. To highlight the difference, we visualize the SSIM between the rendered images and the GT, where whiter output the residual signifies closer alignment with the GT, denoting better quality, and the area with greater color intensity indicates lower quality.
employed in 3D-GS, and optimize all the attributes in conjunction with VQ. This approach permits precise adjustments to the codebook and the rest attributes, improving its alignment with the training images (#8, SSIM improves from 0.915 0.923).
More Experiment Results
In addition to realistic indoor and outdoor scenes in the Mip-NeRF360 7 and Tank and Temple datasets [65]. we further evaluate our method on the synthetic Blender dataset [5], and provide scene-wise evaluation on all datasets, accompanied by detailed visualizations.
Results on NeRF-Synthetic (Blenderr Dataset. The synthetic Blender dataset includes eight photo-realistic synthetic objects with ground-truth truth controlled camera poses and rendered viewpoints (100 for training and 200 for testing). Similar 3D-GS (16), we start training the model using random initialization. Consequently, we calculate the Global Significance of each Gaussian, work to reduce the SH redundancy, and apply the Vector Quantization (codebook size set 8192) to the learned representation. Overall comparisons with previous methods are listed in Table where we observe our LightGaussiar markedly reduces the average storage size from 52.38MB 7.89MB, while improving the FPS from 310 to 411 with only a slight rendering quality decrease.
Additional Qualitative Results on Mip-NeRF360. We provide extra visualizations for 3D-GS 16, LightGaussian (ours), and VQ-DVGO 49). accompanied by the corresponding residual maps from the
16

---

Table 7: Per-scene results SyntheticNNeRE.
Method
Chair
Drums
Ficus
Hotdog
Lego
Materials
Mic
Ship
Avg.
Size(MB)
3D-GS
94.612
64.163
35.839 5,473
39.607
64,910
29.335
34.185
56,400
52.381
Ours
13.785
9.596
5.994
9.600
4.542
5.252
8,464
7.838
PSNR(dB)
3D-GGS
35,436
26,294
35.614
37.848
35.45
30.533
36,585
31.642
33.716
Ours
34.769
26.022
34.484
36.461
34.944
29.341
35.370
30.405
32.725
SSIM
3D-GS
0,987
0,954
0,987
0.985
0.981
0.961
0.992
0.904
0.969
Ours
0,986
0.952
0.985
0.982
0.979
0.954
0.990
0.896
0.965
LPIPS
3D-GS
0.0133
0.0405
0.0121
0.0232
0.0195
0.0404
0.0073
0.111
0.0334
Ours
0.0142
0.0431
0.0137
0.0275
0.0222
0.0461
0.0087
0.121
0.0370
Table 8: Quantitative Comparison (PSNR) on Neo-Ne and Tank & Temple Scenes, We re-train 3D-GS, Compressed 3D-GS, Compact 3D-GS using their original code and 3D-GS configu- rations on our platform, to perform a fair comparison.
Methods
bicycle
fowerss
garden
stump
100m]
PSNR
trechill
counter
kitchen
bonsai
(rick(&&))
Train (RIBTTTT
Pienoxels
21.912
20.097
23.1947
20.661
22.248
27.594
23.624
23.420
24.669
23.221
18.927
INGP-Big
30.685
23.383
mip- NeR-NN
24.305
21.649
26.875
26 175
22.979
11.467
29:447
31.989
13.397
24.912
19.523
VQ-DVGO
22.092
18.934
24.127
23.428
28.502
21.440
26.035
25.459
27.990
ID-GS
25.191
21.114
21.29
26. 26.598
31.336
22.176
28. 989
31.128
31.918
25.372
21,957
Compresseed 3D-GS
25,040
21,126
26.817
26 357
31,072
22 298
28. 653
30.755
31.161
25.194
21,882
Compact 3D&GS
24.795
20.974
26.703
26.200
30.645
22 22.570
28.553
30.332
31.84
25,041
21.544
25.196
21,538
26.77
30,869
31,414
25,399
ground truth, As evidenced in Figure 8 and Figure 9 LightGaussian outperforms VQ-DVGO, which utilizes NeRF as basic representation. Furthermore, LightGaussian achieves comparable rendering quality 3D-GS [16]. demonstrating the effectiveness of our proposed compact representation.
Additional Quantitative Results on Mip-NeRF360. Tables and 10 present the comprehensive еπrо metrics compiled for our evaluation across all real-world scenes MMRFNN and Tank and Temple datasets). Our method not only compresses the average model size from 782MB 45MB, but also consistently demonstrates comparable metrics with 3D-GS on all scenes. LightGaussian additionally shows better rendering quality than Plenoxel, INGP, mip mip-NeRF360, and VQ-DVGO.
Implementation Details of VQ-DVGO. In the implementation of VQ-DVGO 49 we initially obtain a non-compressed grid model following the default training configuration of DVGO [9. The pruning quantile 3p is set to 0.001, the keeping quantile 8g set to 0.9999, and the codebook size is configured to 4096. We save the volume density and the non-VQ. voxels in the fp16 format without additional quantization. For the joint finetuning process, we have increased the iteration count to 25,000 25,000, surpassing the default setting of 10,000 iterations, to maximize the model's capabilities. All other parameters are aligned with those specified in the original VQ-DVGO paper [49, ensuring a faithful replication of established methodologies.
Table 9; Quantitative Comparison (SSIM) on Mip-NeRF360 and Tank & Temple Scenes. We re-train 3D-GS, Compressed 3D-GS, Compact 3D-GS using their original code and 3D-GS configu- rations our platform, to perform a fair comparison.
Methods
flowers
SSIM
trechill
kitchen
Track(&TT
Train/&&T)
0.606
0.509
0.759
INGP-Big
0.512
0.486
0.701
0.504
0.542
0.871
0.817
0.858
0.906
0,800
0.689
mip NeRI3360
0.685
0.584
0.809
0.631
0.745
0.910
0.892
0.917
0.938
0.857
0.660
VQ-DVGO
0.473
0.355
0.614
0.563
0.861
0.476
0.777
0.760
0.842
3D-GS
0,862
0.917
0.906
0.811
Compressed 3D-GS
0.752
0.584
0.845
0.757
0.912
0.622
0.896
0.915
0.932
0.873
0.803
Compact 3D-GS
0.737
0.568
0.837
0.753
0.909
0.632
0.892
0.913
0.931
0.869
0.788
Ours
0.765
0.598
0.855
0.776
0.916
0.635
0.898
0.919
0.931
0.877
0.801
17

---

Ground-truth
Ground-truhh
Ground-truth
RGB (3D-GS)
RGB (3D-GSS
RGB (3D-GS)
Residual Map (3D-GS)
Residual Map (3D-GS)
Residual Map (3D-GS)
RGB (Ours)
RGB (Ours)
RGB (Ours)
Residual Map (Ours)
Residual Map (Ours)
Residual Map (Ours)
RGB VO-DYGO)
RGB (VO-DVGO)
RGB (VO-DVGO)
Residual Map (VQ-DVGO)
Residual Map (VQ-DVGO)
Residual Map (VQ-DVGO)
Figure Additional Visual Comparisons on the Mip-NeRF360 Datasets. We present the rendering results from 3D-GS [6), LightGaussian, and VQ-DVGO (4), The residual maps highlight the differences between the rendered images and ground truth (GT) images.
Overall Analysis. We report the performance of the investigation on the proposed modules in Tab. and Fig. 7 We verify that our design of Gaussian Pruning & Recovery effective in removing redundant Gaussians (Exp #1 11+) with negligible quality degradation while preserving rendering accuracy. This proves that the proposed global significance, based on principle of splatting, accurately represents the critical aspects of each Gaussian. By removing the high-degree SHs and transferring the knowledge to a compact representation (Exp #3 -5), our method successfully demonstrates the benefits from using soft targets and extra data from view augmentation, results in negligible changes
18

---

Ground-truth
Ground-trulh
Ground-trutl
RGB (3D-G8)
RGB (3D-GS)
RGB (3D-GS)
RGB (3D+GS)
Residual Map (3D-SS)
Residual Map (3D+GS)
Reaiuall Map EDDGSS
Residual Map (3DGSS)
RGB (Ours)
RGB (Ours)
RGB (Ours)
RGB (Ours)
Residual Map (Our)
Residual Man (Ours)
Residual Map (Ours)
Residual Map (Ours)
RGB DVGO)
RGB VVO-VVG))
RGB (YO.VVG))
RGB DVGO
Residual Map (VQ-DVGO)
Residual Map (VQ-DVGO)
Residual Map (VQ-DVGO)
Residual Map (VQ-DVGO)
Figure Additional Visual Comparisons on Mip-NeRF360 Datasets. We present the rendering results from 3D-GS [16), LightGaussian, and VQ-DVGO 449. The residual maps highlight the differences between the rendered images and ground truth (GT) images.
in specular reflection. In practical post-processing, the vector quantization on the least important Gaussians (Exp #7), showeases the advantage of adopting VQ to further reduce model size.
Table 10: Quantitative Comparison (LPIPS) on Mip-NeRF360 and Tank & Temple Scenes. We re-train 3D-GS, Compressed 3D-GS, Compact 3D-GS using their original code and 3D-GS configurations our platform, to perform fair comparison.
Methods
Blwers
[PPISS
Trechill
kitchen
frack(&&T
Train/&&T)
INGP-Big
0.446
0.441
0.257
0.421
0.450
0.261
0.306
0.195
0.205
0.249
0.360
mip NeRF360
0.305
0.346
0.171
0.265
0.347
0.213
0.207
0.128
0.179
0.159
0.354
VQ-DVGO
0.402
0.216
0.244
3D-GS
0.108
Compressed 3D-GS
0.237
0.358
0.139
0.248
0.234
0.215
0.140
0.219
0.157
0.220
Compact 3D-GS
0.256
0.378
0.144
0.256
0.233
0.347
0.223
0.140
0.218
0.163
0.240
Ours
0.218
0,352
0.122
0.222
0.232
0.338
0.220
0.141
0.321
0.155
0.239
19

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
We have clearly stated the contributions and scope in the Abstract and Intro-
duction.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: We describe the limitations in the Section S
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: The paper does not include theoretical results.
4. Experimental Result Reproducibility
Question: Does the paper fully disclose all the information needed reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We have presented the information in Section 4.1
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [No]
Justification: We will release our code after our paper gets accepted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e-g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The training and test details are specified in Section 4.1 Implementation Details.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: The paper does not involved statistical test.
8. Experiments Compute Resources
Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [Yes]
Justification: We report the type of resources used in the experiments in Section 4.1
9. Code Of Ethics
20

---

Question:
Does the research conducted in the paper conform, in every respect. with the
NeurIPS Code of Ethics
Answer: [Yes]
Justification: The research conducted in the paper conform the NeurlPS Code of Ethics.
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
We have discusses the broader impact in Section
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The released model does not have high risk for misuse.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We cite corresponding papers for the asserts we use in Section 2
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: This work does not involve crowdsourcing nor research with human subjects. 15. Institutional Review Board (IRB) Approvals Equivalent for Research with Human
Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This work does not involve crowdsourcing nor research with human subjects.
21


Reviews:
[{"review_id": "DzbMDfGgRH", "reviewer": "Reviewer_3NHK", "summary": "This manuscript presents a pipeline to drastically reduce the size of pretrained 3D Gaussian splatting models in a way that preserves novel view image fidelity and increases rendering speed. This pipeline consists of three parts: 1) pruning based on an introduced global significance score followed by fine-tuning, 2) reducing the spherical harmonic dimension of the Gaussians and using knowledge distillation to recover the lost specular information, 3) vector quantization of the spherical harmonics of gaussians below a certain global significance score threshold. The global significance score per Gaussian is accumulated over all pixel rays that that intersect that Gaussian and is the sum of the opacity of the Gaussian multiplied by a normalized volume of the Gaussian. Experimental results show a marked decrease in the model size and increase in rendering speeds while image quality is only slightly degraded.", "questions": "I’m rating this manuscript as “Accept”. I encourage the authors to engage with the weaknesses I’ve listed and will consider raising my score if they are addressed in the rebuttal.", "limitations": "This manuscript has adequately covered the limitations of the proposed pipeline.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "S1: This manuscript is well written and easy to follow. Details are provided that should allow for the ability to replicate experiments.\n\nS2: The current size of 3D-GS models makes them unusable in low resource settings — such as VR/AR — and compressing these models will open up many new advancements in downstream applications. The proposed pipeline demonstrates great performance on real world scenes and will be a benefit to researchers working in resource constrained settings.", "weaknesses": "**Main weaknesses:**\n\nW1: I’m surprised that contribution per pixel ray is sufficient for an effective the global significance score. It seems like a per Gaussian score would need to account for both the Gaussian’s contribution to the input image pixels AND the location of those pixels in 3D space. A Gaussian observed by multiple cameras should hold more importance as it will be more likely to faithfully reconstruct the 3D scene than a Gaussian only observed by a single camera.\n\nW2: I’m also surprised that the global significance score doesn’t need to account for the exponential drop in contribution to the pixel color due to alpha blending from Gaussians in front of the Gaussian whose score is being computed.\n\nW3: Does the global significance score get updated for all Gaussians along a pixel’s ray or is it updated for only those Gaussians that contribute to the pixel color? I suspect it is the latter, but this should be clearly stated in the manuscript.\n\nW4: It’s surprising to me that recovering the higher degree spherical harmonic representation in the lower spherical harmonic degrees via knowledge distillation and sampled views is needed to maintain high image quality after spherical harmonic reduction.  I’d expect that recovering the color via fine-tuning in the just the training views should work quite well and knowledge distillation and sampled views should be ablated against this.\n\n\n**Cosmetic issues:**\n\nEq. 4: Volume should always be positive, so there shouldn’t be a need to take the max of normalized volume and 0.\n\nLine 200: “network” should be “model”, 3D-GS models aren’t networks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**[W1 & W2] Does the score need to account for the exponential drop in contribution to the pixel color? Is the score for each Gaussian computed using multiple cameras or a single camera?**  \nWe thank you for the insightful suggestions. With considering the “exponential drop” in Eq.3, we found the Gaussian Pruning metrics slightly improve on both indoor scene and outdoor unbounded scene. In the table below, we replace the Gaussian opacity $\\sigma$ with $\\sigma \\cdot T$, $\\alpha$, and $\\alpha \\cdot T$, where $T$ is transmittaance and $\\alpha$ is parameterized by opacity, scale and covariance. We found the use of opacity perform similar with alpha, while considering the transmittance all slightly improve the pruning effectiveness. We will include a more detailed discussion in the revision. Eq. 3 considers the impact from all training cameras (denoted as M) to formulate the score for each Gaussian.\n\n\n| Scene Bicycle                  | PSNR ↑ | SSIM ↑ | LPIPS ↓ |\n|--------------------------|------------|-----------|------------| \n| opacity  |25.09 |\t0.7593|\t0.2261|\n| opacity*transmittance | 25.17 |\t0.7647 |\t0.2196|\n| alpha  |25.08 |\t0.7591|\t0.2256|\n| alpha*transmittance | 25.15 |\t0.7630 |\t0.2206|\n\n| Scene Room                  | PSNR ↑ | SSIM ↑ | LPIPS ↓ |\n|--------------------------|------------|-----------|------------| \n| opacity  |31.38 |\t0.9144 |0.2364|\n| opacity*transmittance | 31.39 |\t0.9158 |0.2320|\n| alpha  |31.35 |0.9135|0.2374|\n| alpha*transmittance |31.38 |\t0.9158 |0.2317|\n\n\n\n**[W3] Does the score get updated for all Gaussians along a pixel’s ray or for only those Gaussians that contribute to the pixel color?**  \nYou are right, its’ the latter and we will clarify more about this in Eq.3 about the criteria 1(G(X_j), r_i) in the Sec.3.2. The intersection will stop when the accumulated opacity reaches 0.9999, same as the way in 3D-GS.\n\n\n**[W4] Why distillation is needed in converting high-order SH to low-order SH?**  \nTo validate the effectiveness of distillation with augmentation, we ablate the distillation design choices in rows [5] and [6] of Table 2. Additionally, we conducted another experiment on Scene Room (MipNeRF360) that only utilizes photometric loss from the training view to recover the model with low-degree Spherical Harmonics coefficients. The results indicate that simply fine-tuning with photometric error can recover most of the accuracy, while the use of teacher-student distillation with data augmentation can further slightly improve the rendering quality. The table below are based on new configuration mentioned in the general response.\n\n\n| Methods/Metrics                       | PSNR ↑ | SSIM ↑ | LPIPS ↓ |\n|--------------------------------------|--------|--------|---------|\n| Photometric Loss                               | 31.42 | 0.9157 |\t0.2338\n| Distillation +  Pseudo-views                       | 31.48 |\t0.9167|\t0.2313\n\n\n\nIn the general machine learning domain, we also noticed literature [1] that studies the use of teacher-student distillation to recover full accuracy for the student. By utilizing \"mixup\" data augmentation to construct virtual training examples, the distillation process can \"generate support points outside the original image manifold,\" which is beneficial for accuracy recovery.\n\n\n\n\n\n**[W5] Cosmetic issues?**  \nWe will revise the draft accordingly.\n\n\n**Reference**  \n[1]. Knowledge distillation: A good teacher is patient and consistent, CVPR 2022", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NJe95wsomm", "reviewer": "Reviewer_bxja", "summary": "In this paper, the authors delivered a compact 3D Gaussian representation, i.e., LightGaussian for novel view synthesis. There are three technical contributions. Firstly, the authors present Gaussian Pruning and Recovery that measure the significance of each Gaussian to the view quality and then prune 3D Gaussian that has minimal impact on the visual quality. Secondly, the authors propose Spherical Harmonics (SH) distillation that condenses the information from higher-order coefficients to a more compact form. Thirdly, a Vector Quantization step is presented to reduce the redundancy in spatial and lighting representations.", "questions": "I have a question for the presented SH distillation step\n\nSome existing works just use an view-independent component (also low-order SH coefficients) to represent the color of 3D Gaussian [1]. Did you ever try this method? It is interesting to have a discussion between the SH distillation with this option.\n\n[1] Radiative Gaussian Splatting for Efficient X-ray Novel View Synthesis. In ECCV 2024.", "limitations": "Yes, the authors have analyzed the limitations and broader impact of the method in section 4.4 of the main paper.", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "i) The technology in this paper is very comprehensive, including the three most mainstream methods of compressing deep learning models, i.e., pruning, distillation, and quantization. Thus, this paper has great engineering significance and can inspire subsequent works of compressing 3DGS from three points of view.\n\nii) The idea of Gaussian Pruning & Recovery and SH distillation is novel. Computing the score for each 3D Gaussian to represent its significance is fancy and reasonable. The insight of designing Eq. (3) as the score function is interesting. As for the distillation, few people pay attention to the learnable numbers, especially the SH coefficients. Distilling the high-order coefficients with pseudo views also provides a new idea of how to reduce the attributes / learnable parameters of 3D Gaussian.\n\niii) The proposed LightGaussian achieves a better balance between the reconstruction quality, storage usage, and inference speed. To be specific, in table 1, LightGaussian yields improvements of 0.2 dB in PSNR, 81 fps in inference speed, and 6 MB in storage memory than the state-of-the-art method compact 3DGS on the Mip-NeRF 360 datasets. Meanwhile, LightGaussian achieves faster speed and small model size on the Tank and Template datasets with only a 0.2 dB drop.\n\niv) The experiments are sufficient. The results in Table 2 clearly demonstrates the effect of each component. The experiments in Table 3, 4, and 5 study the variants of the three compressing techniques. The authors consider very comprehensive situations in these ablation study experiments.\n\nv) The static webpage and video demo in the supplementary are very attractive and Exquisite, which save a lot of time for readers to understand what the authors did in this work.", "weaknesses": "i) The story line is not coherent and the motivation may need more explanation. In particular, the three technical contributions, i.e., Gaussian Pruning and Recovery, SH Distillation, and Vector Quantization seem like three independent works. Combining them together without clear motivation make this paper like a technical report. Although I appreciate the work the authors have done, it would be better to improve the writing by establishing a coherent story line and adding clear motivation for the proposed methods.\n\nii) It is better to add discussion and fair comparison with previous 3DGS compressing works such as compressed 3D-GS [34] and compact 3D-GS [36]. For example, what is the difference between the compress techniques? And what about the performance with the same baseline model?\n\niii) Some technical details may require more explanation, for example, why designing Eq. (3) like this as the contribution score of each 3DGS? It is interesting to know the process and key insight of the designing process. How you think about this question?\n\niv) For the main results in table 1, why the Compressed 3D-GS does not have the inference speed? And why LightGaussian is better than SOTA on the Mip-NeRF 360 dataset and performs worse on the Tank and Template datasets? It is better to add analysis and discussion here to explain this performance gap.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**[W1] Rephrase the motivation.**  \nWe are motivated by the observation that the efficient point-based representation, 3D-GS, and its many follow-ups perform poorly in model size because they have to store each of the N (usually millions) points. A large number of points typically results in slow rendering speed. Additionally, we found a heavy attribute size for each primitive, especially due to the high-degree Spherical Harmonics used for preserving the view-dependent effect.  \nThis motivates us to design a holistic pipeline that effectively reduces redundancy in the Gaussians (NxC) for both the primitive count (N) and feature dimension (C). We start by identifying the Gaussians that contribute the least to the training observations (images) and prune them away. Gaussian pruning not only reduces model redundancy but also significantly increases rendering efficiency, as there are fewer Gaussians in the viewing frustum. Further redundancy in the feature dimension is addressed through distillation to reduce the degree of Spherical Harmonics with pseudo-view augmentation to preserve the view-dependent effect. Vector Quantization (VQ) is a general technique that we utilize with joint fine-tuning to further remove primitive redundancy.  \nWe hope our pipeline is general and can be used as a plug-in tool for many researchers in this field. We will highlight the motivation in the revised introduction.  \n\n**[W2] Discussion and fair comparison with Compressed 3DGS and Compact 3DGS.**  \n**Compact 3DGS** proposes a learnable mask strategy to reduce the number of Gaussians and utilizes a grid-based neural field to replace the spherical harmonics for a compact representation.  \n**Compressed 3DGS** proposes a post-processing method that uses sensitivity-aware vector clustering with quantization-aware training to compress the 3D Gaussians.  \nIn contrast, **LightGaussian** proposes a visibility-based Gaussian Significance Score to identify the least important Gaussians and introduce a distillation strategy with virtual view augmentation to transfer knowledge from teacher to student with low-degree Spherical Harmonics coefficients.  \nPlease refer to the general response for a fair comparison.\n\n\n**[W3] Explanation over Eq.3.**  \nThe motivation behind Eq. 3 is that the densification in 3DGS is based on spatial gradients and can be noisy, as it recovers the full 3D representation from limited training views. This results in an overparameterized scene representation to match the training pixels, which can affect the rendering speed if too many Gaussians are in the viewing frustum. Our insight is to find a general formulation to consider the contribution of each 3D Gaussian to the training views, determined by whether they intersect or not (denoted by $1(G(X), r)$), the opacity, and the volume of a Gaussian and how it impacts the training pixels.\n\n\n**[W4.1] Why FPS for Compressed 3D-GS is missing?**  \nIt is because we reiterated the metrics from their original paper, which does not provide FPS metrics. We reran the fair comparisons in the tables in General Response.\n\n**[W4.2] Discrepancy of the performance between MipNeRF360 and Tank and Temple?**  \nIn the main draft (Tab. 1), we use the re-trained 3D-GS, which performs slightly differently from the reported results in 3D-GS. We also reiterate the results from the compared methods in our main draft, as they do not disclose the training details. We conduct fair comparisons to align with the methods, mentioned in general response. It can be observed that LightGaussian has significantly higher FPS while preserving similar visual quantitative metrics compared to the other methods.\n\n\n**[Q1] Applying view-independent RGB similar to X-Gaussian?**  \nThanks for the suggestion! X-Gaussian redesigns a radiative Gaussian point cloud model inspired by the isotropic nature of X-ray imaging, incorporating Differentiable Radiative Rasterization and Angle-pose Cuboid Uniform Initialization for X-ray scanners. However, unlike X-ray imaging, simply utilizing view-independent methods significantly decreases the PSNR from 27.40dB to 24.89dB (MipNeRF360 datasets), which is significantly lower than our method (27.13dB). We will include this discussion in the revision.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9sOq2aogam", "reviewer": "Reviewer_yR9d", "summary": "The paper focuses on compressing 3D Gaussian Splatting (3D-GS) models by mainly focusing on reducing the number of Gaussians and compressing the feature size of Gaussians. With three key steps: gaussian pruning and recovery, spherical harmonics distillation, and vector quantization, the paper achieves ~15x reduction in disk usage and ~2x faster rendering speeds with minimal loss in rendering quality. The effectiveness is verified on Mip-NeRF 360, Tank and Temple, and NeRF synthetic datasets.", "questions": "Please address the issues I pointed out in the 'weakness' section. Furthermore, I would appreciate clarification on the following questions in the authors' rebuttal.\n* Can authors provide the rendering speeds of [34] which are missing in Table 1?\n* In Table 2 are the reported FPSs for training (finetuning) or testing? If for testing why the FPS changes after some steps that should only affect the training procedure, for example from step [4] to [5]. Are they just noises?\n* In the experiments do you use a universal pruning ratio in the Gaussian pruning step, or is it scene-specific? \n* (Minor) In Algorithm 1 line 12 why use G' rather than G as the teacher? As the rendering quality of G' is slightly worse than G. Is it because G' is faster to render?", "limitations": "The authors provide a discussion of limitations in the paper.", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "* Compacting the storage cost and rendering speed of 3DGS with minimal loss of rendering quality is an interesting task and has a lot of applications such as rendering on VR devices.\n* The paper proposed a heuristic metric to evaluate the importance of each Gaussian, to help identify the less important Gaussian which are then pruned out to reduce the storage cost and accelerate the rendering speed.\n* The paper used knowledge distillation after reducing the dimension of the spherical harmonic features to recover the rendering quality.\n* The paper did detailed ablation studies on the three key steps and the sub-steps, plus different design choices and hyperparameters, which clearly shows the effectiveness of each components.", "weaknesses": "* The proposed vector quantization step lacks novelty, for example both [34] and [36] used vector quantization to compress Gaussian attributes, with minor differences in the method. \n* In comparison to the prior 3DGS compression works, the improvement is limited. As shown in Table 1, the improvement on the Mip-NeRF 360 dataset is not clear compared to [34], and the results are worse than [34] on the Tank and Temple dataset in terms of both rendering quality and model size. Besides, why the rendering speeds of [34] are not available?\n* The method requires at least two finetuning steps (Gaussian co-adaptation and VQ finetuning) for the compression, beyond the pre-trained 3DGS model, and each finetuning step requires the same iterations as pre-training. Prior work for example [36] can compress 3DGS in an end-to-end manner, and achieve similar performance. How long does the method take to compress one 3DGS model, and in comparison with the 3DGS compression baselines?\n\nIn conclusion, the paper is not significantly better than the 3DGS compression baselines. The proposed two novel compression methods such as the Gaussian pruning by a heuristic metric and the knowledge distillation do not show considerable improvement over the baselines in terms of rendering quality, rendering speed, and storage cost.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**[W1] Vector quantization step lacks novelty**  \nLightGaussian is motivated to design a holistic pipeline that effectively reduces the redundancy in the optimized Gaussians (NxC) for both the primitive count (N) and feature dimension (C). A large number of points will additionally result in slow rendering speed. The heavy attribute size for each primitive is mainly caused by the high-degree Spherical Harmonics used for preserving the view-dependent effect.\nThus, LightGaussian proposes to identify the Gaussians that contribute the least to the training observations (images) to prune them away. This decreases the model size from 353MB to 116MB while increasing FPS from 192 to 303 (Tab.4). Further redundancy in the feature dimension is addressed through distillation to reduce the degree of Spherical Harmonics with pseudo-view augmentation to preserve the view-dependent effect. The model size is further reduced from 116MB to 77MB (Tab.4).  \nVector Quantization (VQ) is a general technique in image synthesis [1] and neural volumetric video representations [2]. We also utilize VQ with joint fine-tuning to further remove primitive redundancy. Overall, we achieve an average compression rate of over 15x while significantly boosting the FPS from 119 to 209, thanks to the holistic framework. We will moderate the tone of the contribution of Vector Quantization within our overall framework.\n\n\n\n**[W2] Improvement is limited? Improvement is not clear in Table 1. Why are the rendering speeds of [34] not available?**  \nPlease refer to the general response for the metrics under a fair experimental setting. Our method significantly boosts the FPS from 144 (3D-GS) to 237, a 64% improvement. We also surpass Compact 3DGS by 53% and Compressed 3DGS by 55% in FPS.\n\n\n**[W3] Generalization of LightGaussian? Time Comparison?**  \n\nPlease refer to the general response.\n\n**[W4] Improvement of Gaussian Pruning and SH Distillation?**  \nWe kindly argue that all three components contribute to both compactness and rendering efficiency. As shown in our ablation study (Table 2): pruning the least important Gaussians improves the FPS from 192 to 303 (+57%) while reducing the model size from 353MB to 116MB (-67%); SH compactness further reduces the model size to 77MB (-33%). We are hopeful that this new paradigm will disseminate valuable insights for efficient 3D learning.  \nWe respectfully reiterate another reviewer's comment regarding this point: \"The idea of Gaussian Pruning & Recovery and SH distillation is novel, and achieves a better balance between the reconstruction quality, storage usage, and inference speed. \" **(Review #bxja)**; “demonstrates great performance on real world scenes.” **(Review #3NHK)**; “LightGaussian introduces a novel pruning strategy based on global significance” **(Review #TPCr)**\n\n\n**[Q1] Rendering speeds of [34] in Table 1?**  \nPlease refer to the general response.\n\n**[Q2] Why FPS changes from step [4] to [5] in table 2**  \nThe evaluation is automatically performed after each compression step, rendering 1000 images and reporting the average FPS. Thus, slight discrepancies in each row are caused by the independent evaluation. For rows [7] and [8], where the 3D Gaussians are not further updated, we repeat the numbers from row [6].\n\n**[Q3]: Universal pruning ratio, or scene-specific?**  \nIt is a universal ratio determined by experiments, as shown in Figure 5.\n\n\n**[Q4] Why use G' rather than G as the teacher?**  \nThey perform very similarly, with G’ being slightly faster.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "h8ebrsMwIV", "reviewer": "Reviewer_TPCr", "summary": "LightGaussian introduces a three-stage technique to efficiently reduce the number of Gaussian primitives. In the first stage, redundant Gaussian primitives are pruned based on global significance, rather than opacity. The second stage involves SH distillation, which utilizes data augmentation from synthesized pseudo views. Finally, the third stage employs Vector Quantization of the SH coefficients. By incorporating these three approaches, LightGaussian achieves a remarkable 15x compression while maintaining competitive rendering quality.", "questions": "1. Please provide a more detailed analysis of compression GS methods, such as Compressed 3D-GS and Compact 3D-GS, in the experimental section. In addition to comparing FPS, it would be valuable to include a comparison of training times as well.\n2. It would greatly enhance the paper if the authors compare LightGaussian with Scaffold-GS and HAC in the experimental section for a more comprehensive evaluation.", "limitations": "The authors discuss the limitation in Section 5.", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. LightGaussian introduces a novel pruning strategy based on global significance, along with SH Distillation and Vector Quantization techniques, to effectively reduce both primitive redundancy and feature redundancy.\n2. LightGaussian functions as a plugin and can be utilized in any GS representation model.\n3. The authors conducted a comprehensive ablation study to demonstrate the effectiveness of the proposed method.\n4. This method efficiently reduces the number of Gaussian primitives, leading to a significant improvement in rendering speed.\n5. The paper is well-written and easy to follow.", "weaknesses": "1. **Lack of necessary experimental analysis.** Although the authors compare LightGaussian with Compressed 3D-GS and Compact 3D-GS in Table 1, they fail to provide any analysis in the experimental results section, which is quite peculiar. Moreover, most of the experimental analyses focus on NeRF, which seems unnecessary. Furthermore, in Table 1, it is difficult to distinguish the visual quality performance and compressed capacity between LightGaussian and Compressed 3D-GS, and, strangely, there is no qualitative number provided for the FPS of Compressed 3D-GS.\n2. **Lack of novelty.** This method appears to be a post-processing approach of 3D-GS and lacks essential characteristics, unlike Scaffold-GS [1], which proposes a hierarchical 3D Gaussian representation and only stores the information of anchors. It would be beneficial for the authors to compare LightGaussian with Scaffold-GS and its subsequent work, HAC [2], in terms of both experimental results and analysis.\n3. **Poor layout formatting.** In Figure 7, the author incorrectly typeset the paper, leading to a compromised reading experience. By the way, the comparison in Figure 6 is not clearly distinguishable, as the eight images appear to be nearly identical.\n\n[1] Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering\n\n[2] HAC: Hash-grid Assisted Context for 3D Gaussian Splatting Compression", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**[W1]: Analysis over other methods? Why is FPS missing for Compressed 3D-GS?**  \nUnder a fair experimental setting, we observed that LightGaussian outperforms Compact 3DGS and Compressed 3DGS in 4 out of all 5 metrics on the MipNeRF360 dataset while also running fastest on Tank and Temples datasets. Notably, our method significantly boosts the FPS from 144 (3D-GS) to 237, a 64% improvement. We also surpass Compact 3DGS by 53% and Compressed 3DGS by 55% in FPS.  LightGaussian performs similarly in overall visual quality while preserving thin structures slightly better than other methods, shown in the attached PDF. Please refer to the general response for more details.\n\n**[W2]: LightGaussian and Scaffold-GS**  \nWe integrated the Gaussian Pruning & Recovery into Scaffold-GS to prune up to 80% redundant neural Gaussians and improve the rendering efficiency. See general response for details.\n\n\n**[W4] Update Figure 6 and Figure 7.**  \nWe will revise the layout of Figure 7 and update Figure 6 with zoomed-in regions to improve the reading experience for both figures.\n\n**[Q1] FPS and training time comparison with Compressed 3D-GS and Compact 3D-GS**  \nPlease refer to the general response for training and inference efficiency.\n\n\n**[Q2] Comparison with Scaffold-GS and HAC.**  \nScaffold-GS proposes to initialize a sparse grid of anchor points from SfM points, and tethers a set of neural Gaussians with learnable offsets. This method constrained the distribution of 3D while achieving better reconstruction accuracy. HAC further explores  a structured hash grid to exploit the inherent consistencies among unorganized 3D Gaussians that achieves remarkable compression ratio over 3D-GS.\nWe demonstrate the application of LightGaussian upon Scaffold-GS in general response.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
**General Response: Rendering speed of Compressed 3D-GS is missing, and a fair comparison?**  
The reason for omitting the FPS of Compressed 3D-GS [34] in the main draft is because we reiterated the metrics from their original paper, which does not provide FPS metrics.   
We also respectfully point out that the original 3D-GS utilizes different downsampling ratios for different scenes (see full_eval.py in their GitHub repository) while the compared methods do not clarify their settings. Thus, for a fair comparison, we reran experiments for LightGaussian, Compact 3DGS [36], and Compressed 3DGS [34] on the same platform (NVIDIA A6000) using the default 3D-GS resolution configuration. We observed that our method outperforms Compact 3DGS and Compressed 3DGS in 4 out of all 5 metrics on the MipNeRF360 dataset while also running fastest on Tank and Temples datasets. Notably, our method significantly boosts the FPS from 144 (3D-GS) to 237, a 64% improvement. We also surpass Compact 3DGS by 53% and Compressed 3DGS by 55% in FPS.  
We provide visual comparisons among the adopted methods in the attached PDF.  


| Data: MipNeRF360                 | FPS ↑  | Size ↓ | PSNR ↑ | SSIM ↑ | LPIPS ↓ |
|--------------------------|-----------|----------|------------|-----------|------------| 
| 3D-GS (baseline)  | 144     | 782.11 | 27.40    | 0.8192  | 0.217 | 
| Compact 3D-GS   | 154       | 49.40 | 27.01   | 0.7986  | 0.243   |
| Compressed 3D-GS   | 152     | 28.63     | 27.03   | 0.8018  | 0.238     |
| LightGaussian            | 237    | 45.21     | 27.13   | 0.8066  | 0.237       |

| Data: Tank and Temples                  | FPS ↑  | Size ↓ | PSNR ↑ | SSIM ↑ | LPIPS ↓ |
|---------------------------|----------|----------|------------|-----------|--------------|
| 3D-GS (baseline)     | 182    | 431.0 | 23.66     | 0.8445 | 0.178      |
| Compact 3D-GS      | 238   | 39.43   | 23.29    | 0.8285  | 0.202     |
| Compressed 3D-GS| 202   | 17.68   | 23.54    | 0.8380  | 0.189     |
| LightGaussian         | 357    | 25.30  | 23.44     | 0.8318  | 0.202    |


**General Response: Training vs Rendering Speed?**   
We report the training and inference efficiency based on the fair experimental setting on MipNeRF360 datasets.

| Method                      | Rendering (FPS) | Train Time (minutes)  |
|-----------------------------|-----------------|-----------------------|
| 3D-GS                       | 144             | 23.8 mins             |
| Compact 3D-GS               | 154             | 33.7 mins             |
| 3D-GS + Compressed 3D-GS    | 152             | 23.8 + 3.8 mins       |
| 3D-GS + LightGaussian       | 237             | 23.8 + 9.0 mins       |


**General Response: Generalization to different methods?**   
LightGaussian can function as a plugin and is more general for various point-based representations. Specifically, we also validate LightGaussian on Scaffold-GS [1] to prune redundant neural Gaussians using our Visibility-aware Gaussian Pruning & Recovery. We observe that by pruning 80% of neural Gaussians, we can accelerate the rendering speed of Scaffold-GS from 152 to 173 FPS. Results are averaged on MipNeRF360 datasets. We reran the experiments on our platform for fair comparisons.

| Methods/Metrics                      | FPS | PSNR ↑ | SSIM ↑ | LPIPS ↓ |
|--------------------------------------|-----|--------|--------|---------|
| 3D-GS                                | 144 | 27.40  | 0.8192 | 0.217   |
| Scaffold-GS                          | 152 | 27.96  | 0.8240 | 0.2075  |
| HAC                                  | 167 | 27.76  | 0.8191 | 0.2198  |
| Scaffold-GS + LightGaussian          | 173 | 27.78  | 0.8187 | 0.2197  |



**Reference**  
[1] Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering, CVPR 2024.

Meta-review: Pre-rebuttal, the paper received mixed scores that required a lot of attention from the authors. Recurring issues were related to missing comparisons to existing 3DGS compression methods (in terms of the speed and memory consumption), and to lack of novelty.

Authors did a good job clearing all concerns related to additional comparisons, and they are strongly encouraged to include this additional analysis in the final version. Regarding the novety concerns, the AC agrees that each individual component is not novel on its own, but the significant effectivity of the proposed component marriage is novel. 

Post-rebuttal, all reviewers raised their scores above the acceptance threshold which sends the paper well above the acceptance bar.
Authors are encouraged to include the additional performance analysis (requested by all reviewers) in the final version, together with the promised improvements to the exposition (reviewer bxja).
Decision: Accept (spotlight)