Locality-aware Gaussian Compression for Fast and High-quality Rendering

Authors: Seungjoo Shin, Jaesik Park, Sunghyun Cho
Primary area: applications to computer vision, audio, language, and other modalities
Keywords: point-based rendering, neural fields, compression

Abstract:
We present LocoGS, a locality-aware 3D Gaussian Splatting (3DGS) framework that exploits the spatial coherence of 3D Gaussians for compact modeling of volumetric scenes.
To this end, we first analyze the local coherence of 3D Gaussian attributes, and propose a novel locality-aware 3D Gaussian representation that effectively encodes locally-coherent Gaussian attributes using a neural field representation with a minimal storage requirement.
On top of the novel representation, LocoGS is carefully designed with additional components such as dense initialization, an adaptive spherical harmonics bandwidth scheme and different encoding schemes for different Gaussian attributes to maximize compression performance.
Experimental results demonstrate that our approach outperforms the rendering quality of existing compact Gaussian representations for representative real-world 3D datasets while achieving from 54.6$\times$ to 96.6$\times$ compressed storage size and from 2.1$\times$ to 2.4$\times$ rendering speed than 3DGS. Even our approach also demonstrates an averaged 2.4$\times$ higher rendering speed than the state-of-the-art compression method with comparable compression performance.

Decision: accept (poster)
Reviewer scores: 6, 8, 6, 3

Reviews and discussion:
<thread>
creation date:
1729140611438

thread_type:
official_review

<thread_object>
creator: reviewer_esf7

<content>
summary:
The paper introduces "LocoGS," a locality-aware 3D Gaussian Splatting (3DGS) framework designed for fast and high-quality rendering. This framework capitalizes on the spatial coherence among 3D Gaussians to offer a compact, efficient representation that significantly reduces storage requirements and enhances rendering speed without sacrificing quality. 

Key contributions include:
1. A new locality-aware 3D Gaussian representation that efficiently encodes locally-coherent Gaussian attributes.
2. Implementation of additional components like dense initialization, Gaussian pruning, adaptive spherical harmonics bandwidth scheme, and tailored encoding schemes to maximize compression performance.
3. Demonstrated superior performance over existing methods in terms of compression ratio and rendering speed, verified through extensive experiments.

strengths:
1. The locality-aware strategy for Gaussian attribute representation is a novel approach that leverages spatial coherence effectively.
2. The paper includes detailed comparisons with existing methods, showing improvements in storage efficiency and rendering speed. The reduced storage and increased speed facilitate the practical application of 3DGS in real-time scenarios, like mobile devices.
3. The motivation and clarity of the paper are commendable.

weaknesses:
1. The multiple components introduced (e.g., multi-resolution hash grids, and adaptive SH bandwidths) might complicate the implementation and tuning of the framework.
2. The paper could benefit from more discussion on any potential trade-offs or limitations, particularly in different or more challenging rendering environments. e.g. metal and lighting area.

questions:
1. Are there scenarios or conditions under which the proposed method might not perform as expected? e.g. large scenes.
2. Could you provide more detailed explanations or pseudo-codes for complex components like the adaptive SH bandwidth and pruning strategy? Are you going to publish your code?
3. Does your model also perform better in larger scenes? Can you please add more experiments on how it performs in larger scenes? e,g. Miller-19 dataset.
</content>
</thread_object>

</thread>
<thread>
creation date:
1729935639503

thread_type:
official_review

<thread_object>
creator: reviewer_fma7

<content>
summary:
LocoGS analyzes the local coherence of the attributes of Gaussian primitives and introduces a novel representation that incorporates locality information. By utilizing the locality-aware 3D Gaussian representation along with other compression methods, such as quantization and encoding schemes, LocoGS achieves state-of-the-art compression performance and rendering FPS compared to existing compression methods.

strengths:
1. LocoGS carefully analyzes the relationships among 3D Gaussian attributes and introduces a locality-aware representation based on these relationships.
2. The paper is well written and easy to follow, with an excellent categorization of compression methods.
3. LocoGS outperforms existing methods in both compression performance and rendering speed.

weaknesses:
1. Dense initialization is not related to compression; it is merely a warm-up trick. It would be unfair for the authors to use this trick since all of the baselines still initialize with COLMAP points.
2. As we know, the training time for Nerfacto is significantly longer than for 3DGS. Why do the authors choose to use Nerfacto for warm-up instead of 3DGS? Both methods can generate coarse depth maps, which can then be used to create an initialization point cloud. Additionally, how many iterations do the authors train Nerfacto for dense initialization?
3. In the limitations section, the authors describe that LocoGS requires one hour more training time than the baselines. Which components affect the training efficiency—is it the dense initialization?
4. No demos submitted.

questions:
1. I would like to understand the reason why the rendering performance of LocoGS is lower than that of Scaffold-GS. Aside from the quantization and encoding schemes, do the authors discard the view input of the MLP when obtaining each attribute of the 3D Gaussians?
2. How about the performance in large-scale datasets, like Mega-NeRF, Urbanscene3D or MatrixCity?

If the authors solve these questions, I’ll raise the score.
</content>
</thread_object>

</thread>
<thread>
creation date:
1729967726581

thread_type:
official_review

<thread_object>
creator: reviewer_5dlr

<content>
summary:
The paper aims to achieve both high rendering speed and reduced storage size for 3D Gaussian Splatting-based scene representations. To this end, it analyzes the local coherence of Gaussians across several scenes and proposes exploiting multi-resolution hash grids. More specifically, Gaussians now have only base scale, positions, and base color, while hash grids fill in the details. In addition, the paper employs adaptive SH bandwidth, pruning, dense initialization, quantization, and encoding. With all these methods, the paper demonstrates significant performance improvement.

strengths:
The concept of locality is not novel in 3D Gaussian Splatting, but the paper effectively illustrates this through graphs. Moreover, it clearly addresses the differences between other locality-based methods (anchor-based methods), explaining why the proposed method is important and highlighting key points for readers already familiar with anchor-based representation.

The performance improvements in terms of size, rendering speed, and rendering quality are significant.

Additionally, the paper includes many technical details for adopting various techniques, especially during quantization and encoding, which will benefit those working in this research areas.

weaknesses:
As I mentioned in the strength section, the idea of utilizing locality in 3D Gaussian Splatting is not novel.

questions:
How did you calculate cosine similarity of opacities?
I would also like to know how and why three explicit attributes were selected.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730585232498

thread_type:
official_review

<thread_object>
creator: reviewer_6k9a

<content>
summary:
This paper tackles the high storage demands associated with 3D Gaussian Splatting by introducing an effective compression method. The core idea is to exploit local similarity within Gaussian attributes, representing them through compact, well-encoded local features. The authors start by conducting a statistical analysis of 3D Gaussian Splatting (3DGS) attributes and proceed to design a local feature that captures and encodes these similar patterns within a given region. By combining carefully structured steps—such as pruning, point cloud initialization, quantization, and entropy encoding—the proposed method achieves substantial compression of the 3DGS field. Remarkably, this compression is achieved while maintaining, or even enhancing, the rendering quality and fidelity relative to the original setup.

strengths:
1. The overall solution is comprehensive, incorporating careful designs for initialization, pruning, and compression schemes for different components. 
2. The paper is well-written with a clear logical flow.
3. The dense initialization demonstrates an interesting improvement in compression.
4. The analysis of storage size in Tab.3 provides a valuable indication for the community about the current bottlenecks in compression.

weaknesses:
1. Number of Gaussians in Variants: It would be beneficial to provide the number of Gaussians for different variants, as Ours-Sparse has a lower FPS compared to Ours-Small and Ours. Understanding whether the final number of Gaussians influenced the results is important. Including the point number counts along the training iteration would help illustrate the influence of different initializations. Additionally, it would be interesting to see if such an initialization design could improve the performance of previous approaches.
2. The method still takes extra training time to compress a 3DGS, which will be a limitation in some real applications.

questions:
1. Please include the number of Gaussians for the different ablation versions in Tab.3 to understand the main reason for high FPS.
 
2 Decoding Time: Including the decoding time would help readers understand the applicability of the proposed method.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732200597603

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Global Response by Authors

comment:
## **Global response**

Dear reviewers,

We thank all reviewers for their thoughtful comments. We have tried to respond to all these valuable concerns. In response to the comments, we have revised the manuscript and provided additional supplementary materials. Please note that we have highlighted updates in blue.

---

### **[Q1] Performance on large-scale datasets (FMA7, EsF7)**

As requested by the reviewers, we evaluated the performance of LocoGS on a representative large-scale dataset, the Mill-19 dataset, as shown in Table R1. The large-scale scenes in Mill-19 require an excessive number of Gaussians, resulting in large storage requirements (>1.3GB) for the original 3DGS approach. LocoGS successfully reduces the memory size to less than 34 MB, achieving a **47.2x compression ratio**. Additionally, our effective compression scheme involving Gaussian point pruning also substantially improves the rendering speed.

Nevertheless, despite its high compression ratio, our method requires approximately 1.5 to 2 times more memory space than the original 3DGS during training due to the gradient computation of the hash grid and Gaussian attributes. This increased memory requirement presents some challenges when applying our method directly to large-scale scenes. The slight quality degradation observed in Table R1 is also because of this heavy memory requirement, which limits the number of Gaussians to fewer than necessary. To address this issue, dividing a large-scale scene into smaller subregions could be a promising solution. By breaking the scene into more manageable parts, we can reduce the memory requirement and maintain high rendering quality across the entire scene. Please note that increased memory consumption only applies to the training phase. Our method follows memory usage of the original 3DGS for the rendering, which is proportional to the number of Gaussians.

**Table R1. Evaluation on the Mill-19 dataset.**
|Method|PSNR&uarr;|SSIM&uarr;|LPIPS&darr;|Size (MB)&darr;|FPS&uarr;|#G|
|:---|:--:|:--:|:---:|:---:|:---:|:---:|
|***Building***||
|3DGS|19.14|0.651|0.365|1371.02|83|5.44 M|
|Ours|19.79|0.630|0.354|32.97|113|4.13 M|
|***Rubble***|||
|3DGS|23.75|0.733|0.311|1752.37|69|6.95 M|
|Ours|23.38|0.651|0.366|33.26|122|4.20 M|
</content>
</thread_object>

</thread>
<thread>
creation date:
1732785517619

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Revised Manuscript

comment:
Dear reviewers and AC,  

We sincerely appreciate the reviewers’ insightful comments on how to enhance our paper. To address these valuable suggestions, we have carefully revised the manuscript as follows:

**[Section 2]**  
Locality-exploiting 3DGS approaches (5DLr)

**[Section 4.1 & Figure 1]**  
Analysis of local coherence based on Euclidean distance (5DLr)

**[Table 3]**  
Number of Gaussians (6k9A)

**[Section 7]**  
Limitations for large-scale scenes (FMA7, EsF7)

**[Section A.3.1]**  
Ablation study on the attribute design (5DLr)

**[Section A.3.2]**  
Ablation study on the hash grid size. We relocated this section from Sec. 6.2 due to the page limit.

**[Section A.3.4]**  
Effect of dense initialization on existing compression methods (6k9A, FMA7)

**[Section A.3.5]**  
Ablation study on the dense initialization method (FMA7)

**[Section A.4]**  
Algorithms for locality-aware Gaussian representation learning (EsF7)

**[Section A.5.1]**  
Number of Gaussians (6k9A)

**[Section A.5.2]**  
Decoding time (6k9A)

**[Section A.5.3]**  
Large-scale Scenes (FMA7, EsF7)

Please note that these updates have been highlighted in blue.

Best regards,  
Authors
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper introduces a method for 3D Gaussian compression by leveraging the spatial coherence of 3D Gaussians. The results demonstrate that the method effectively reduces storage requirements and improves rendering speed. While several concerns were initially raised, most were addressed in the rebuttal. The remaining issues are related to the method's novelty and its applicability to large-scale scenes.

In terms of novelty, while similar concepts have been explored, the rebuttal clarified the distinctions between the proposed approach and other locality-exploiting methods. Regarding large-scale scenes, the rebuttal demonstrated that the proposed method effectively reduces the storage size of a scene from 1.3GB to 33MB. While the compression method is more beneficial for larger scenes, as stated in the rebuttal, compressing extremely large scenes would require additional effort in scene and memory management, which goes beyond the scope of this paper. Additionally, it is worth noting that most papers on the same topic conduct experiments on similar datasets.