LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPS

Authors: zhiwen  fan
kevin  wang
kairun  wen
zehao  zhu
dejia  xu
zhangyang  wang
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-10 01:11:35

Abstract: Recent advances in real-time neural rendering using point-based techniques have enabled broader adoption of 3D representations. However, foundational approaches like 3D Gaussian Splatting impose substantial storage overhead, as Structure-from-Motion (SfM) points can grow to millions, often requiring gigabyte-level disk space for a single unbounded scene. This growth presents scalability challenges and hinders splatting efficiency. To address this, we introduce LightGaussian, a method for transforming 3D Gaussians into a more compact format. Inspired by Network Pruning, LightGaussian identifies Gaussians with minimal global significance on scene reconstruction, and applies a pruning and recovery process to reduce redundancy while preserving visual quality. Knowledge distillation and pseudo-view augmentation then transfer spherical harmonic coefficients to a lower degree, yielding compact representations. Gaussian Vector Quantization, based on each Gaussian’s global significance, further lowers bitwidth with minimal accuracy loss. LightGaussian achieves an average 15 times compression rate while boosting FPS from 144 to 237 within the 3D-GS framework, enabling efficient complex scene representation on the Mip-NeRF 360 and Tank & Temple datasets. The proposed Gaussian pruning approach is also adaptable to other 3D representations (e.g., Scaffold-GS), demonstrating strong generalization capabilities.

Main paper: 
 introduction: 
 Novel view synthesis (NVS) aims to generate photo-realistic images of a 3D scene from unobserved viewpoints, given a set of calibrated multi-view images. This capability has widespread applications 2 Related Works Efficient 3D Scene Representations. Neural radiance fields (NeRF) [5] use multi-layer perceptrons (MLPs) to represent scenes, setting new standards for view synthesis quality. However, NeRF models face challenges with slow inference speeds, limiting practical use. Efforts to address this have explored ray re-parameterizations [6, 7] , explicit spatial data structures [18-21, 9, 14, 13] , caching and distillation techniques [15, [22] [23] [24] , and ray-based representations [25, 26] . Nevertheless, achieving real-time rendering remains challenging for NeRF methods, especially for large-scale scenes where multiple queries per pixel hinder performance.Point-based representations, like Gaussians, have been explored in various applications, such as shape reconstruction [27] , molecular modeling [28] , and point-cloud replacement [29] , as well as in shadow [30] and cloud rendering [31] . Pulsar [32] demonstrated efficient sphere rasterization, while 3D Gaussian Splatting (3D-GS) [16] applies anisotropic Gaussians [33] with tile-based sorting to achieve real-time speed and quality comparable to MLP-based methods like Mip-NeRF 360 [7] .Despite its strengths, 3D-GS has high storage demands due to the extensive attributes stored with each Gaussian, often requiring gigabytes per scene and hindering rendering efficiency. Recent concurrent works address these issues by using region-based vector quantization [34] , K-means codebooks [35] , view-direction exclusion [36] , or learned binary masks and grid-based neural networks instead of spherical harmonics (SHs) [37] to reduce model size.Pruning and Quantization. Model pruning reduces neural network complexity by removing nonsignificant parameters, balancing performance and resource use. Unstructured [38] and structured pruning [39, 40] eliminate parameters at the weight level and neuron/channel levels, resulting in a smaller, more efficient architecture. Iterative magnitude pruning (IMP), where low-magnitude weights are progressively removed, has proven effective in methods like lottery ticket rewinding [41, 42] .Vector quantization (VQ) [43] compresses data by representing it with discrete codebook entries (tokens), using mean square error (MSE) to select the closest match in the codebook for each data vector. Prior studies [44] [45] [46] have shown that learning discrete, compact representations enhances visual understanding and model robustness. VQ has thus been widely applied in image synthesis [47] , text-to-image generation [48] , and novel view synthesis [18, 49, 50] .Knowledge Distillation. Knowledge Distillation (KD) [51] [52] [53] [54] 15] trains a smaller student model by transferring knowledge from a larger teacher model [55] . In 3D vision, KD has been applied to neural scene representations, leveraging view renderings to incorporate 2D priors. For example, DreamFusion [56] and NeuralLift-360 [57] use pre-trained diffusion models for 3D generation, while models like DFF [58] , NeRF-SOS [59] , INS [60] , and SA3D [61] distill 2D image feature extractors to 3D tasks. KD has also been central to compressing scene representation models [15, 24] . 
methods: 
 Overview. The LightGaussian framework is illustrated in Fig. 2 . The 3D-GS model is trained on multi-view images, initialized from SfM point clouds, and expanded to millions of Gaussians to represent the scene comprehensively. Our pipeline then processes the 3D-GS model into a compact format using Gaussian Prune and Recovery to reduce the number of Gaussians, SH Distillation to eliminate redundant SHs while retaining key lighting information, and Vector Quantization to store Gaussians with lower bit-width.3D Gaussian Splatting (3D-GS) [16] is an explicit point-based 3D scene representation, utilizing Gaussians with various attributes to model the scene. When representing a complex real-world scene, 3D-GS is initialized from a sparse point cloud generated by SfM, and Gaussian Densification is applied to increase the Gaussian counts that are used for handling small-scale geometry insufficiently covered and over reconstruction. Formally, each Gaussian is characterized by a covariance matrix Σ and a center point X, which is referred to as the mean value of the Gaussian:EQUATIONwhere Σ can be decomposed into a scaling matrix S and a rotation matrix R.The complex directional appearance is modeled by an additional property, Spherical Harmonics (SH), with n coefficients, c i ∈ R 3 |i = 1, 2, . . . , n where n = D 2 represents the number of coefficients of SH with degree D. A higher degree D equips 3D-GS with a better capacity to model the viewdependent effect but causes a significantly heavier attribute load.When rendering 2D images from the 3D Gaussians, the technique of splatting [62, 17] is employed for the Gaussians within the camera planes. With a viewing transform denoted as W and the Jacobian of the affine approximation of the projective transformation represented by J , the covariance matrix Σ ′ in camera coordinates can be computed as Σ ′ = J W ΣW T J T . Specifically, for each pixel, the color and opacity of all the Gaussians are computed using the Gaussian's representation Eq. 1. The blending of N ordered points that overlap the pixel is given by the formula:EQUATIONHere, c i , α i represents the view-dependent color and opacity, calculated from a 2D Gaussian with covariance Σ multiplied by an optimizable per-3D Gaussian's opacity. In summary, each Gaussian point is characterized by attributes including: position X ∈ R 3 , color defined by spherical harmonics coefficients SH ∈ R (k+1) 2 × 3 (where k represents the degrees of freedom), opacity α ∈ R, rotation factor R ∈ R 4 , and scaling factor S ∈ R 3 .Gaussian densification [16] , which clones and refines the initial SfM point cloud, enhances small-scale geometry and detailed scene appearance by improving coverage. While this approach significantly boosts reconstruction quality, it increases the number of Gaussians from thousands to millions after optimization, resulting in substantial storage demands.Inspired by neural network pruning techniques [63] that remove less impactful neurons while preserving overall performance, we propose a tailored pruning strategy for 3D Gaussian Splatting to reduce over-parameterized points while maintaining accuracy. Identifying redundant yet recoverable Gaussians is essential to our approach. However, pruning Gaussians based on simple criteria (e.g., point opacity) risks degrading modeling performance, as it may eliminate essential scene details, as shown in Fig. 3 .Global Significance Calculation. Inspired by Eq. 2, we go beyond using Gaussian opacity alone to assess significance by evaluating 3D Gaussians within the view frustum, projecting them onto the camera viewpoint for rendering. The initial significance score of each Gaussian (denoted as G(X j )) can then be quantified based on its contribution to each pixel (ray, r i ) across all training views using the criteria 1(G(X j ), r i ) whether they intersect or not. Consequently, we iterate over all training pixels to calculate the hit count of each Gaussian. The score is further refined by the adjusted 3D Gaussian's volume γ (Σ j ) and 3D Gaussian's opacity σ j , as they all contribute for the rendering formula. The volume calculation equation of the j-th 3D Guassian is V(Σ j ) = 4 3 πabc, where abc are the 3 dimensions of Scale (S). We also consider the transmittance T is calculated with one subtract all the 3D Gaussian's opacity the ray hit before the j-th 3D Gaussian, T = j-1 i=1 (1-σ i ). Finally, the global significance score can be summarized as:EQUATIONwhere j is the Gaussian index, i denotes a pixel, and M , H, and W represent the number of training views, image height, and width, respectively. 1 is an indicator function that determines whether a Gaussian intersects a given ray. However, using Gaussian volume alone tends to overemphasize background Gaussians, leading to excessive pruning of Gaussians modeling fine geometry. Thus, we propose a more adaptive approach to measuring volume dimensions:EQUATIONV norm = min V(Σ) V max90 , 1 .Here, the calculated Gaussian volume is firstly normalized by the 90% largest of all sorted Gaussians, clipping the range between 0 and 1, to avoid excessive floating Gaussians derived from vanilla 3D-GS. The β is introduced to provide additional flexibility.Gaussian Co-adaptation. We rank all Gaussians by their global significance scores to quantitatively guide pruning of the lower-ranked Gaussians. The remaining Gaussians are then jointly adapted by fine-tuning their attributes-without additional densification-to offset the minor loss from pruning. This adaptation is performed using photometric loss on the original training views, aligned with 3D-GS training for 5,000 iterations.In the uncompressed Gaussian Splat representation, a substantial 81.3 percent of the feature dimension is occupied by Spherical Harmonics (SH) coefficients, requiring (45+3) floating-point values per splat. Directly reducing the SH degree can save disk space but also diminishes surface "shininess" and affects specular reflections.To balance model size with scene quality, we introduce a knowledge distillation approach. Knowledge is distilled from a teacher model with full-degree SHs to a student model with truncated, lower-degree SHs. Supervision is based on the difference in predicted pixel intensities between the two models, with images synthesized at camera positions by sampling around each training view according to a Gaussian distribution:L distill = 1 HW HW i=1 ∥C teacher (r i ; [R|t]) -C student (r i ; [R|t])∥ 2 2 .(5)EQUATIONwhere R and t denote rendering camera rotation and position, t pseudo and t train represent the newly synthesized and training camera positions, respectively. N denotes a Gaussian distribution with mean 0 and variance σ 2 , which is added to the original position to generate the new position.Vector quantization (VQ) clusters voxels into compact codebooks, enabling high compression rates. However, quantizing Gaussian attributes in a point-based, inherently non-Euclidean representation poses notable challenges. Our empirical findings indicate that applying quantization across all elements-especially for attributes like position, rotation, and scale-results in significant accuracy losses and a marked reduction in precision when represented discretely.We propose applying VQ to the Spherical Harmonics (SH) coefficients, based on the assumption that a subgroup of 3d Gaussians typically exhibits a similar appearance. Fundamentally, VQ segments the Gaussians G = {g 1 , g 2 , . . . , g N } (here we apply it on SH) to the K codes in the codebook C = {c 1 , c 2 , . . . , c K }, where each g j , c k ∈ R d and K ≪ N. d means SH dimension. We aim to strike a balance between rendering quality loss and compression rate by leveraging the pre-computed significance score from Eq. 3. Based on this, we apply VQ selectively on the least significant elements in the Spherical Harmonics (SHs). Specifically, we initialize C via K-means, iteratively sample a batch of G, associates them to the closest codes by euclidean distance, and update each c k via moving average rule:c k = λ d • c k + (1 -λ d ) • 1/T k • gj ∈R(c k ) GS j • g j , where T k = gj ∈R(c k ) GS jis the significance score (Eq. 3) which is assigned to the code vector c k , R(c k ) is the set of Gaussians associated to the k-th code. λ d = 0.8 represents the decay value, which is utilized to update the [12, 14] exhibit insufficient capacity for representing large-scale scenes and are unable to achieve real-time performance. Mip-NeRF360 [7] produces the highest visual quality, but requires over 16 seconds to render a single image. Our method strikes a balance among FPS, model size, and rendering quality, achieving the best balance among efficient representations. For fair metric and visual comparison, we re-trained 3D-GS and Compressed 3D-GS [34] using their original code and configurations on our platform (NVIDIA A6000 GPU), with results marked by * . For a full dynamic viewpoint comparison, please see our supplementary video.code vector using a moving average. We fine-tune the codebook for 5,000 iterations, while fixing the gaussian-to-codebook mapping. We disable additional clone/split operations and leverage photometric loss on the training views. To preserve essential attributes, including Spherical Harmonics with higher global significance scores, along with Gaussian position, shape, rotation, and opacity, we skip VQ for these elements and store them directly in float16 format. 
experimental settings: 
 Datasets and Metrics. We conduct comparisons using the scene-scale view synthesis dataset provided by Mip-NeRF360 [64] , which comprises nine real-world large-scale scenes, including five unbounded outdoor and four indoor settings with complex backgrounds. In addition, we utilize the Tanks and Temples dataset [65] , a comprehensive unbounded dataset, and select the same scenes as used in [16] . Performance metrics on synthetic object-level datasets will be detailed in the supplementary materials. We report metrics including the peak signal-to-noise ratio (PSNR), structural similarity (SSIM), and perceptual similarity as measured by LPIPS [66] .Compared Baselines. We compare our approach with methods suited for large-scale scene modeling, including Plenoxel [12] , Mip-NeRF360 [7] , and 3D-GS [16] . Additionally, we evaluate against efficient techniques like Instant-NGP [14] , which uses a hash grid for storage efficiency, and VQ-DVGO [49] , which extends DVGO [9] with voxel pruning and VQ for optimized representation.Implementation Details. Our framework is implemented in PyTorch and integrates the differentiable Gaussian rasterization technique from 3D-GS [16] . All performance evaluations are conducted on an A6000 GPU. In the Global Significance Calculation phase, we assign a power value of 0.1 in Eq. 4 and proceed to fine-tune the model for 5,000 steps during the Gaussian Co-adaptation process.For SH distillation, we downscale the 3-degree SHs to 2-degree, thereby reducing 21 elements for each Gaussian. This is further optimized by setting σ to 0.1 in the pseudo view synthesis stage.In the Gaussian VQ step, the codebook size is configured to 8192, selecting SHs with the least 60% significance score for the vector quantization (VQ ratio), to balance the trade-off between compression efficiency and fidelity. Quantitative Results. We assess the performance of various methods for novel view synthesis, with quantitative metrics summarized in Tab. 1. This includes efficient voxel-based NeRFs like Plenoxel [12] and Instant-NGP [14] , the compact MLP-based Mip-NeRF360 [7] , vector-quantized NeRF [49] , and 3D Gaussian Splatting [16] .On the Mip-NeRF360 dataset, MLP-based NeRF methods achieve competitive accuracy with compact representations but suffer from slow inference speeds (0.06 FPS), limiting their practicality. Voxelbased NeRFs improve rendering efficiency but still fall short of real-time performance; for example, Plenoxel requires 2.1GB for a single large-scale scene. In contrast, 3D-GS offers a good balance between quality and speed but demands substantial storage per scene.Our method, LightGaussian, exceeds existing techniques with rendering speeds over 200 FPS, enabled by efficient rasterization that prunes insignificant Gaussians. This approach reduces 3D Gaussian model redundancy, cutting storage from 782MB to 45MB on Mip-NeRF360-a 15× reduction.LightGaussian also outperforms 3D-GS on the Tank & Temple datasets, nearly doubling rendering speed and reducing storage from 380MB to 22MB.Qualitative Results. We conducted a comparative analysis of rendering results between 3D-GS and LightGaussian, focusing on intricate details and background regions, as shown in Fig. 4 . Both 3D-GS and LightGaussian exhibit comparable visual quality, even in challenging scenes with thin structures, showing that LightGaussian effectively removes redundancy while preserving reconstruction fidelity.We further apply our Gaussian Pruning approach to Scaffold-GS [67] , an advanced neural Gaussian-based 3D representation. In this method, we prune Gaussians that contribute minimally to scene reconstruction, reducing redundancy while preserving visual fidelity. Experiments on the MipNeRF360 dataset demonstrate that pruning 80% of neural Gaussians increases the rendering speed from 152 to 178 FPS, as shown in Tab. 5. These results underscore LightGaussian's potential as an effective optimization tool for other Gaussian-based representations.We performed ablation studies on each component of our method to evaluate their individual impacts.Results show that Gaussian Pruning and Recovery effectively removes redundant Gaussians, retaining only those crucial for scene representation. SH Distillation reduces the Spherical Harmonics degree, simplifying the lighting model with minimal quality loss. Furthermore, Vector Quantization efficiently compresses the feature space, creating a more compact representation. Combined, these modules substantially improve the overall efficiency and performance of our framework.Significance Criteria To identify the optimal criteria for measuring each Gaussian's global significance, we evaluated key characteristics: Gaussian-ray interaction count, Gaussian opacity, and functional volume. Tab. 3 illustrates that integrating these three factors -by weighting the hit count with opacity and Gaussian volume -yields the highest rendering quality post zero-shot pruning. A quick Gaussian Co-adaptation (the last row) which optimizes the remaining Gaussians, can effectively recover the rendering accuracy to its pre-pruning level. The visual comparison of rasterized images, before and after pruning, alongside the depiction of pruned Gaussians, is presented in Fig. 5 . Degradation & Speed vs. Compression Ratio We investigate the interplay between rendering quality and speed across various model compression ratios. Specifically, we adjust parameters such as Gaussian Pruning & Recovery, SH Distilling, and VQ. As depicted in Fig 6 , we note a marked decline in rendering quality when the pruning ratio reaches 70%, with a more rapid deterioration as the VQ ratio approaches 65%. In Tab 6, we observe that 2-degree SH compactness reduces the SSIM from 0.927 to 0.926, while 1-degree SH compactness further reduces it to 0.923. 
conclusion, limitations, and broad impact: 
 We present LightGaussian, a novel framework that transforms heavy point-based representations into a compact format for efficient novel view synthesis. Designed for practical use, LightGaussian leverages 3D Gaussians to model large-scale scenes, effectively identifying and pruning the least significant Gaussians generated through densification. It achieves over 15× data reduction, boosts FPS to over 200, and minimally impacts rendering quality. Exploring zero-shot compression across various 3D-GS-based frameworks remains a promising direction for future research.Broadly, LightGaussian's compact representation has the potential to democratize high-quality 3D content for applications in VR, AR, and autonomous driving by reducing resource demands, enabling more accessible and scalable deployment across industries. While we do not foresee any significant risk from this specific technique, 3D reconstruction may infringe on personal privacy when applied in public spaces or with drone footage, contradicting our ethical intentions. 
more experiment results: 
 In addition to realistic indoor and outdoor scenes in the Mip-NeRF360 [7] and Tank and Temple datasets [65] , we further evaluate our method on the synthetic Blender dataset [5] , and provide a scene-wise evaluation on all datasets, accompanied by detailed visualizations.Results on NeRF-Synthetic 360°(Blender) Dataset. The synthetic Blender dataset [5] includes eight photo-realistic synthetic objects with ground-truth controlled camera poses and rendered viewpoints (100 for training and 200 for testing). Similar to 3D-GS [16] , we start training the model using random initialization. Consequently, we calculate the Global Significance of each Gaussian, work to reduce the SH redundancy, and apply the Vector Quantization (codebook size set at 8192) to the learned representation. Overall comparisons with previous methods are listed in Table 7 , where we observe our LightGaussian markedly reduces the average storage size from 52.38MB to 7.89MB, while improving the FPS from 310 to 411 with only a slight rendering quality decrease.Additional Qualitative Results on Mip-NeRF360. We provide extra visualizations for 3D-GS [16] , LightGaussian (ours), and VQ-DVGO [49] , accompanied by the corresponding residual maps from the ground truth. As evidenced in Figure 8 and Figure 9 , LightGaussian outperforms VQ-DVGO, which utilizes NeRF as a basic representation. Furthermore, LightGaussian achieves a comparable rendering quality to 3D-GS [16] , demonstrating the effectiveness of our proposed compact representation.Additional Quantitative Results on Mip-NeRF360. Tables 8, 9 , and 10 present the comprehensive error metrics compiled for our evaluation across all real-world scenes (Mip-NeRF360 and Tank and Temple datasets). Our method not only compresses the average model size from 782MB to 45MB, but also consistently demonstrates comparable metrics with 3D-GS on all scenes. LightGaussian additionally shows better rendering quality than Plenoxel, INGP, mip-NeRF360, and VQ-DVGO.Implementation Details of VQ-DVGO. In the implementation of VQ-DVGO [49] , we initially obtain a non-compressed grid model following the default training configuration of DVGO [9] . The pruning quantile β p is set to 0.001, the keeping quantile β k is set to 0.9999, and the codebook size is configured to 4096. We save the volume density and the non-VQ voxels in the fp16 format without additional quantization. For the joint finetuning process, we have increased the iteration count to 25,000, surpassing the default setting of 10,000 iterations, to maximize the model's capabilities. All other parameters are aligned with those specified in the original VQ-DVGO paper [49] , ensuring a faithful replication of established methodologies. We present the rendering results from 3D-GS [16] , LightGaussian, and VQ-DVGO [49] . The residual maps highlight the differences between the rendered images and ground truth (GT) images.Overall Analysis. We report the performance of the investigation on the proposed modules in Tab. 2 and Fig. 7 . We verify that our design of Gaussian Pruning & Recovery is effective in removing redundant Gaussians (Exp #1→3) with negligible quality degradation while preserving rendering accuracy. This proves that the proposed global significance, based on principle of splatting, accurately represents the critical aspects of each Gaussian. By removing the high-degree SHs and transferring the knowledge to a compact representation (Exp #3 →5), our method successfully demonstrates the benefits from using soft targets and extra data from view augmentation, results in negligible changes We present the rendering results from 3D-GS [16] , LightGaussian, and VQ-DVGO [49] . The residual maps highlight differences between the rendered images and ground truth (GT) images.in specular reflection. In practical post-processing, the vector quantization on the least important Gaussians (Exp #7), showcases the advantage of adopting VQ to further reduce model size. We re-train 3D-GS, Compressed 3D-GS, Compact 3D-GS using their original code and 3D-GS configurations on our platform, to perform a fair comparison.  
 
 Appendix: Algorithm Explanation. We detail the procedures of LightGaussian in Algorithm 1. The trained 3D-GS [16] features a Gaussian location with a dimension of 3, Spherical Harmonics coefficients with a dimension of 48, and opacity, rotation, and scale, whose dimensions are 1, 4, and 3, respectively.Initialize: Training view images I = {Ii ∈ R M } N i=1 and their associated camera poses [16] . 2: # Gi with attributes XYZ, SH-3deg, Opacity, Rotation, Scale. 3:P = {ϕ i ∈ R 3×4 } N i=1 . 1: # Pre-Training 3D-GSG = {Gi ∈ R (3+48+1+4+3) )} N i=1 ← 3D-GS(I, P) ▷ Initial 3D-GS: G 4: #Gaussian Pruning and Recovery: G → G ′ . 5: GS ← CALGS(G) ▷ Calculate Global Significance Score GS 6: G ′ ← PRUNE(G, GS) ▷ Prune Least Significant Ones of G, based on GS 7: G ′ ← RECOVERY(G ′ , P)▷ Gaussian Recovery (Finetune G ′ on P) 8: #Distilling into Compact SHs: Reduce from 3-degree to 2-degree 9: ScoreList.append(score) 14:G ′′ ← REDUCESH(G ′ ) ▷ Reduceend for 15:return ScoreList 16: end function Post-Finetuning of Vector Quantization. Applying vector quantization (VQ) based on the significance score facilitates a reduction in the required codebook size compared to the full model. Nonetheless, this method results in a decrease in accuracy as it groups the attributes into several clusters (codes). We observe that the SSIM decreases from 0.923 (#6) to 0.915 (#7), as illustrated in Table 2 of the main draft. This observation motivates us to joint optimize all other attributes and the codes post the VQ. Specifically, by assigning a unique code to each Gaussian, we proceed to implement differentiable finetuning using photometric loss, which is similar to the methodology employed in 3D-GS, and optimize all the attributes in conjunction with VQ. This approach permits precise adjustments to the codebook and the rest attributes, improving its alignment with the training images (#8, SSIM improves from 0.915 to 0.923). 


Decision: Pre-rebuttal, the paper received mixed scores that required a lot of attention from the authors. Recurring issues were related to missing comparisons to existing 3DGS compression methods (in terms of the speed and memory consumption), and to lack of novelty.

Authors did a good job clearing all concerns related to additional comparisons, and they are strongly encouraged to include this additional analysis in the final version. Regarding the novety concerns, the AC agrees that each individual component is not novel on its own, but the significant effectivity of the proposed component marriage is novel. 

Post-rebuttal, all reviewers raised their scores above the acceptance threshold which sends the paper well above the acceptance bar.
Authors are encouraged to include the additional performance analysis (requested by all reviewers) in the final version, together with the promised improvements to the exposition (reviewer bxja).

Reviews: review: paper_summary: paper_summary: This manuscript presents a pipeline to drastically reduce the size of pretrained 3D Gaussian splatting models in a way that preserves novel view image fidelity and increases rendering speed. This pipeline consists of three parts: 1) pruning based on an introduced global significance score followed by fine-tuning, 2) reducing the spherical harmonic dimension of the Gaussians and using knowledge distillation to recover the lost specular information, 3) vector quantization of the spherical harmonics of gaussians below a certain global significance score threshold. The global significance score per Gaussian is accumulated over all pixel rays that that intersect that Gaussian and is the sum of the opacity of the Gaussian multiplied by a normalized volume of the Gaussian. Experimental results show a marked decrease in the model size and increase in rendering speeds while image quality is only slightly degraded.
strength_weakness: strength_weakness: **Main weaknesses:**

W1: I’m surprised that contribution per pixel ray is sufficient for an effective the global significance score. It seems like a per Gaussian score would need to account for both the Gaussian’s contribution to the input image pixels AND the location of those pixels in 3D space. A Gaussian observed by multiple cameras should hold more importance as it will be more likely to faithfully reconstruct the 3D scene than a Gaussian only observed by a single camera.

W2: I’m also surprised that the global significance score doesn’t need to account for the exponential drop in contribution to the pixel color due to alpha blending from Gaussians in front of the Gaussian whose score is being computed.

W3: Does the global significance score get updated for all Gaussians along a pixel’s ray or is it updated for only those Gaussians that contribute to the pixel color? I suspect it is the latter, but this should be clearly stated in the manuscript.

W4: It’s surprising to me that recovering the higher degree spherical harmonic representation in the lower spherical harmonic degrees via knowledge distillation and sampled views is needed to maintain high image quality after spherical harmonic reduction.  I’d expect that recovering the color via fine-tuning in the just the training views should work quite well and knowledge distillation and sampled views should be ablated against this.


**Cosmetic issues:**

Eq. 4: Volume should always be positive, so there shouldn’t be a need to take the max of normalized volume and 0.

Line 200: “network” should be “model”, 3D-GS models aren’t networks.
questions: questions: I’m rating this manuscript as “Accept”. I encourage the authors to engage with the weaknesses I’ve listed and will consider raising my score if they are addressed in the rebuttal.
limitations: limitations: This manuscript has adequately covered the limitations of the proposed pipeline.
paper_summary: paper_summary: In this paper, the authors delivered a compact 3D Gaussian representation, i.e., LightGaussian for novel view synthesis. There are three technical contributions. Firstly, the authors present Gaussian Pruning and Recovery that measure the significance of each Gaussian to the view quality and then prune 3D Gaussian that has minimal impact on the visual quality. Secondly, the authors propose Spherical Harmonics (SH) distillation that condenses the information from higher-order coefficients to a more compact form. Thirdly, a Vector Quantization step is presented to reduce the redundancy in spatial and lighting representations.
strength_weakness: strength_weakness: i) The story line is not coherent and the motivation may need more explanation. In particular, the three technical contributions, i.e., Gaussian Pruning and Recovery, SH Distillation, and Vector Quantization seem like three independent works. Combining them together without clear motivation make this paper like a technical report. Although I appreciate the work the authors have done, it would be better to improve the writing by establishing a coherent story line and adding clear motivation for the proposed methods.

ii) It is better to add discussion and fair comparison with previous 3DGS compressing works such as compressed 3D-GS [34] and compact 3D-GS [36]. For example, what is the difference between the compress techniques? And what about the performance with the same baseline model?

iii) Some technical details may require more explanation, for example, why designing Eq. (3) like this as the contribution score of each 3DGS? It is interesting to know the process and key insight of the designing process. How you think about this question?

iv) For the main results in table 1, why the Compressed 3D-GS does not have the inference speed? And why LightGaussian is better than SOTA on the Mip-NeRF 360 dataset and performs worse on the Tank and Template datasets? It is better to add analysis and discussion here to explain this performance gap.
questions: questions: I have a question for the presented SH distillation step

Some existing works just use an view-independent component (also low-order SH coefficients) to represent the color of 3D Gaussian [1]. Did you ever try this method? It is interesting to have a discussion between the SH distillation with this option.

[1] Radiative Gaussian Splatting for Efficient X-ray Novel View Synthesis. In ECCV 2024.
limitations: limitations: Yes, the authors have analyzed the limitations and broader impact of the method in section 4.4 of the main paper.
paper_summary: paper_summary: The paper focuses on compressing 3D Gaussian Splatting (3D-GS) models by mainly focusing on reducing the number of Gaussians and compressing the feature size of Gaussians. With three key steps: gaussian pruning and recovery, spherical harmonics distillation, and vector quantization, the paper achieves ~15x reduction in disk usage and ~2x faster rendering speeds with minimal loss in rendering quality. The effectiveness is verified on Mip-NeRF 360, Tank and Temple, and NeRF synthetic datasets.
strength_weakness: strength_weakness: * The proposed vector quantization step lacks novelty, for example both [34] and [36] used vector quantization to compress Gaussian attributes, with minor differences in the method. 
* In comparison to the prior 3DGS compression works, the improvement is limited. As shown in Table 1, the improvement on the Mip-NeRF 360 dataset is not clear compared to [34], and the results are worse than [34] on the Tank and Temple dataset in terms of both rendering quality and model size. Besides, why the rendering speeds of [34] are not available?
* The method requires at least two finetuning steps (Gaussian co-adaptation and VQ finetuning) for the compression, beyond the pre-trained 3DGS model, and each finetuning step requires the same iterations as pre-training. Prior work for example [36] can compress 3DGS in an end-to-end manner, and achieve similar performance. How long does the method take to compress one 3DGS model, and in comparison with the 3DGS compression baselines?

In conclusion, the paper is not significantly better than the 3DGS compression baselines. The proposed two novel compression methods such as the Gaussian pruning by a heuristic metric and the knowledge distillation do not show considerable improvement over the baselines in terms of rendering quality, rendering speed, and storage cost.
questions: questions: Please address the issues I pointed out in the 'weakness' section. Furthermore, I would appreciate clarification on the following questions in the authors' rebuttal.
* Can authors provide the rendering speeds of [34] which are missing in Table 1?
* In Table 2 are the reported FPSs for training (finetuning) or testing? If for testing why the FPS changes after some steps that should only affect the training procedure, for example from step [4] to [5]. Are they just noises?
* In the experiments do you use a universal pruning ratio in the Gaussian pruning step, or is it scene-specific? 
* (Minor) In Algorithm 1 line 12 why use G' rather than G as the teacher? As the rendering quality of G' is slightly worse than G. Is it because G' is faster to render?
limitations: limitations: The authors provide a discussion of limitations in the paper.
paper_summary: paper_summary: LightGaussian introduces a three-stage technique to efficiently reduce the number of Gaussian primitives. In the first stage, redundant Gaussian primitives are pruned based on global significance, rather than opacity. The second stage involves SH distillation, which utilizes data augmentation from synthesized pseudo views. Finally, the third stage employs Vector Quantization of the SH coefficients. By incorporating these three approaches, LightGaussian achieves a remarkable 15x compression while maintaining competitive rendering quality.
strength_weakness: strength_weakness: 1. **Lack of necessary experimental analysis.** Although the authors compare LightGaussian with Compressed 3D-GS and Compact 3D-GS in Table 1, they fail to provide any analysis in the experimental results section, which is quite peculiar. Moreover, most of the experimental analyses focus on NeRF, which seems unnecessary. Furthermore, in Table 1, it is difficult to distinguish the visual quality performance and compressed capacity between LightGaussian and Compressed 3D-GS, and, strangely, there is no qualitative number provided for the FPS of Compressed 3D-GS.
2. **Lack of novelty.** This method appears to be a post-processing approach of 3D-GS and lacks essential characteristics, unlike Scaffold-GS [1], which proposes a hierarchical 3D Gaussian representation and only stores the information of anchors. It would be beneficial for the authors to compare LightGaussian with Scaffold-GS and its subsequent work, HAC [2], in terms of both experimental results and analysis.
3. **Poor layout formatting.** In Figure 7, the author incorrectly typeset the paper, leading to a compromised reading experience. By the way, the comparison in Figure 6 is not clearly distinguishable, as the eight images appear to be nearly identical.

[1] Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering

[2] HAC: Hash-grid Assisted Context for 3D Gaussian Splatting Compression
questions: questions: 1. Please provide a more detailed analysis of compression GS methods, such as Compressed 3D-GS and Compact 3D-GS, in the experimental section. In addition to comparing FPS, it would be valuable to include a comparison of training times as well.
2. It would greatly enhance the paper if the authors compare LightGaussian with Scaffold-GS and HAC in the experimental section for a more comprehensive evaluation.
limitations: limitations: The authors discuss the limitation in Section 5.

Comments: title: Response from the Authors
Response from the Authors
Response from the Authors
Response from the Authors
Response to the author rebuttal
comment: We are pleased that our responses have well addressed your concerns. We sincerely appreciate your willingness to raise your score from 4 to 6 and kindly note that it has not yet been updated in the system.
We are pleased that our responses have well addressed your concerns. We sincerely appreciate your willingness to raise your score from 4 to 6 and kindly note that it has not yet been updated in the system.
Thank you for affirming your positive view of our paper.
Thank you for affirming your positive view of our paper.
Thanks for the detailed experiments and clarification, all my concerns are addressed, I will raise my score to 6.
W2: I’m glad to see that transmittance helped improve the metrics!

W4: If you have not already done so, I’d recommend adding this ablation in the appendix with appropriate links in the main paper.

Overall, I’m satisfied and will keep my rating the same.
Thanks for the authors' sincere reply and effort. 
I agree with the performance of LightGaussian, thanks to detailed experiments, and all my concerns have been tackled. I will raise my score to 6.
Thanks for your reply and effort. All my concerns have been addressed. I keep my original score.