Fast Feedforward 3D Gaussian Splatting Compression

## Abstract

Abstract With 3D Gaussian Splatting (3DGS) advancing real-time and high-fidelity rendering for novel view synthesis, storage requirements pose challenges for their widespread adoption. Although various compression techniques have been proposed, previous art suffers from a common limitation: for any existing 3DGS, per-scene optimization is needed to achieve compression, making the compression sluggish and slow. To address this issue, we introduce F ast C ompression of 3D G aussian S platting (FCGS), an optimization-free model that can compress 3DGS representations rapidly in a single feed-forward pass, which significantly reduces compression time from minutes to seconds.
To enhance compression efficiency, we propose a multi-path entropy module that assigns Gaussian attributes to different entropy constraint paths for balance between size and fidelity. We also carefully design both inter- and intra-Gaussian context models to remove redundancies among the unstructured Gaussian blobs.
Overall, FCGS achieves a compression ratio of over 20 × 20\times 20 × while maintaining fidelity, surpassing most per-scene SOTA optimization-based methods.
Our code is available at: https://github.com/YihangChen-ee/FCGS.

## 1 Introduction

In recent years, 3D Gaussian Splatting (3DGS) (Kerbl et al., ) has significantly advanced the field of novel view synthesis. By leveraging fully explicit Gaussians with color and geometry attributes, 3DGS facilitates efficient scene training and rendering through rasterization techniques (Zwicker et al., ). However, the vast number of Gaussians poses a considerable storage challenge, hindering its wider application.

To address the storage challenges associated with 3DGS, various compression methods have been developed, as surveyed in (Bagdasarian et al., ). These advancements have significantly reduced the storage requirements, bringing them to an acceptable scale. By reviewing the key developments in this area, we identified two fundamental principles that underpin the compression of 3DGS: the value-based and structure-based principles.

- •
Value-based principle. This principle assesses the importance of parameters through value-based importance scores or similarities. Techniques like pruning and vector quantization are employed to reduce the parameter count by retaining only the most representative values. With this principle, earlier studies (Lee et al., ; Niedermayr et al., ; Fan et al., ) focused on compactly representing unorganized Gaussian primitives based on their parameter values, and discarded less significant parameters for model efficiency.
- •
Structure-based principle. More recent approaches have shifted towards exploiting the structural relationships between Gaussian primitives for improved compression (Lu et al., ; Bagdasarian et al., ). This principle emphasizes leveraging organized structures to systematically arrange unsorted Gaussian primitives, which helps eliminate redundancies by establishing structured connections. For instance, HAC (Chen et al., ) uses the Instant-NGP (Müller et al., ) to organize Gaussian anchor features, SOG adopts a self-organizing grid (Morgenstern et al., ), and IGS (Wu & Tuytelaars, ) uses a triplane (Chan et al., ) structure to efficiently organize Gaussian data.

Figure: Figure 1: Left: Existing compression methods require optimization of the existing 3DGS, leading to the drawback of being time-consuming for training. Our proposed FCGS overcomes this issue by compressing 3DGS representations in a single feed-forward pass, significantly reducing time consumption for compression. Right: Compared to Lightgaussian (Fan et al., ), FCGS achieves improved RD performance while requiring much less execution time on the DL3DV-GS dataset.
Refer to caption: x1.png

Despite the effectiveness of compression techniques for 3DGS, they share a common limitation: for any existing 3DGS, per-scene optimization is needed to achieve compression, as illustrated in the left side of Figure 1.
While the optimization-based compression pipeline benefits from scene-specific training to achieve superior RD performance, it slows down the compression significantly
due to the time-consuming finetuning process.

To address this challenge, we propose a novel approach: an optimization-free compression pipeline that enables fast compression of existing 3DGS representations through a single feed-forward pass, which we call FCGS (Fast Compression of 3D Gaussian Splatting) from now on. Agnostic to the source of the 3DGS (either it is from optimizaiton (Kerbl et al., ) or from feed-forward models (Charatan et al., ; Szymanowicz et al., ; Chen et al., ; Tang et al., )), our FCGS allows for fast compression, offering a convenient and hassle-free solution.
In contrast to previous methods that degrade the parameter values or alter the 3DGS structure, thereby requiring additional finetuning, FCGS aims to preserve both the values and the structure integrity of 3DGS, enabling an optimization-free design. The Multi-path Entropy Module (MEM), designed under the *value-based* principle, deduces masks that determine whether attributes should be directly quantized for compression or processed through an autoencoder.
Building on the masks determined by MEM and inspired by the *structure-based* principle, we propose both inter- and intra-Gaussian context models that effectively capture structural relationships among Gaussian attributes.

It is important to highlight that both pipelines of *per-scene optimization-based compression* (previous methods) and *generalizable optimization-free compression* (our approach) are significant and serve different purposes. The former benefits from per-scene adaptation to achieve superior RD performance but suffers from slow compression speed, making it suitable for permanent data storage or server-side encoding.
In contrast, our pipeline offers a convenient and hassle-free solution, where a single model can *directly* and *rapidly* compress various 3DGS without the need for finetuning, making it well-suited for time-sensitive applications. To the best of our knowledge, our work is the first to achieve a *generalizable optimization-free compression pipeline* for 3DGS.
Although the absence of per-scene finetuning naturally limits our RD performance, we have still achieved a compression ratio exceeding 20$\times$ while maintaining excellent fidelity by meticulously designing MEM and context models.
Our contributions are summarized as follows:

- •
We propose a pioneering fast and generalizable optimization-free compression pipeline for 3D Gaussian Splatting, named FCGS, effectively broadening the application scenarios for 3DGS compression techniques.
- •
To facilitate efficient compression of 3DGS representations, we introduce the MEM module to balance size and fidelity across different Gaussians. Additionally, we meticulously customize both inter- and intra-Gaussian context models, significantly enhancing compression performance by effectively eliminating redundancies.
- •
Extensive experiments across various datasets demonstrate the effectiveness of FCGS, achieving a compression ratio of 20$\times$ while maintaining excellent fidelity, even surpassing most of the optimization-based methods.

## 2 Related Work

The field of 3D scene representation for 3D scenes has seen significant advancements in recent years. Neural Radiance Field (NeRF) (Mildenhall et al., ) models scene information using fully implicit MLPs, which predict opacity and RGB values at 3D coordinates for ray rendering. However, these MLPs are designed to be quite large to capture all the scene-specific details, leading to slow training and rendering times. To improve both fidelity and rendering efficiency, subsequent works (Müller et al., ; Chen et al., ; Sun et al., ) have introduced learnable explicit representations that assign scene-specific information to the input coordinates before they are processed by the MLPs. These methods, however, increase storage requirements due to the use of explicit representations. 3D Gaussian Splatting (Kerbl et al., ) takes this further by representing the scene entirely through explicit attributed Gaussians. Using rasterization techniques, these Gaussians can be quickly rendered into 2D images for a given viewpoint. Unfortunately, this fully explicit representation significantly increases storage demands. To address the storage issues of NeRF and 3DGS, various research efforts have focused on compression techniques, which generally fall into two main design principles: value-based and structure-based.

The value-based principle focuses on the importance scores or similarities of parameters. Using these scores or similarities, pruning and vector quantization can be applied via thresholding or clustering, allowing for compression by retaining only the most representative parameters. However, this process often alters the original parameter values by a noticeable margin, necessitating an additional finetuning step to recover the lost information.
For NeRF, methods like VQRF (Li et al., ) and Re:NeRF (Deng & Tartaglione, ) calculate the significance of parameters based on opacity or gradient values and apply pruning or vector quantization, followed by a finetuning phase. BiRF Shin & Park () investigates parameters’ values and binarizes them.
For 3DGS, similar value-based thresholding and clustering techniques are commonly used, as seen in works such as Lee et al. (); Niedermayr et al. (); Navaneet et al. (); Fan et al. (); Girish et al. (); Wang et al. (); Ali et al. (); Fang & Wang (), which consider Gaussian values. Additionally, scalar quantization and knowledge distillation are implemented in Girish et al. () and (Fan et al., ), respectively.

The structure-based principle represents another major direction that investigates the structural redundancies of parameters. CNC (Chen et al., ) introduced a pioneering proof of concept by applying level- and dimension-wise context models to compress hash grids for Instant-NGP (Müller et al., ) in the NeRF series. This structure-based design is also evident in works like Tang et al. (); Rho et al. (); Girish et al. (); Li et al. ().
In the case of 3DGS, structure-based compression methods have also seen notable progress. The primary challenge is the sparse and unorganized nature of 3DGS, which makes it difficult to identify and utilize structural relationships. Grid-based sorting (Morgenstern et al., ) addresses this by projecting 3D Gaussians onto 2D planes for compression while preserving spatial relationships. Scaffold-GS (Lu et al., ) uses anchors to cluster nearby Gaussians that share common features, and based on this, works such as Chen et al. (); Liu et al. (); Wang et al. () further reduce redundancies for anchors, resulting in improved compression performance. HAC (Chen et al., ) and IGS (Wu & Tuytelaars, ) explore relations among the organized grids and Gaussians. SUNDAE (Yang et al., ) employs spectral Graph to improve the pruning efficiency.

However, these approaches all require per-scene finetuning for compression on a given 3DGS, which can be time-consuming.
In this paper, we explore a novel, optimization-free pipeline for 3DGS compression, innovatively building context models for Gaussian primitives, which are essential for eliminating redundancy and improving compression efficiency.

## 3 Fast Compression of 3D Gaussian Splatting

Our goal is to rapidly compress a 3DGS representation in a single feed-forward pass without finetuning. Inspired by image compression methods (Ballé et al., ), we adopt an autoencoder-based structure, where the Gaussian attributes are encoded into a latent space for compression, as shown in Figure 2. However, we found that treating all Gaussian parameters equally and feeding them all into the same autoencoder could lead to a significant loss of fidelity, as some attributes were highly sensitive to deviations. To address this, we introduce a Multi-path Entropy Module (MEM) to effectively balance compression size and fidelity.
For the context model design, we customize both inter- and intra-Gaussian context models that effectively eliminate redundancies among parameters of Gaussians (Figure 3), which are lightweight and efficient. Additionally, we use a Gaussian Mixture Model (GMM) to estimate the probability distribution. In the following sections, we first discuss the background and characteristics of 3DGS, and then delve into the design of our FCGS.

### 3.1 Preliminaries and Discussions

3D Gaussian Splatting (3DGS) (Kerbl et al., ) employs a large number (e.g., $1$ million) attributed Gaussian blobs to represent a 3D scene. These Gaussian blobs are characterized by color and geometry parameters, and are splatted along given views using rasterization techniques to generate rendered images. Specifically, each Gaussian is defined by a covariance $\bm{\Sigma}\in\mathbb{R}^{3\times 3}$ and a location (mean) $\bm{\mu}^{\text{g}}\in\mathbb{R}^{3}$,

$$ $G(\bm{l})=\exp{\left(-\frac{1}{2}(\bm{l}-\bm{\mu}^{\text{g}})^{\top}\bm{\Sigma }^{-1}(\bm{l}-\bm{\mu}^{\text{g}})\right)}\;,$ (1) $$

where $\bm{l}\in\mathbb{R}^{3}$ is a random 3D location, and $\bm{\Sigma}$ is defined by a scaling matrix $\bm{S}\in\mathbb{R}^{3\times 3}$ and a rotation matrix $\bm{R}\in\mathbb{R}^{3\times 3}$ such that $\bm{\Sigma}=\bm{R}\bm{S}\bm{S}^{\top}\bm{R}^{\top}$. To render a pixel value $\bm{C}\in\mathbb{R}^{3}$, the Gaussians are first splatted to 2D, and rendering is performed as follows:

$$ $\bm{C}=\sum_{i}{\bm{c}_{i}\alpha_{i}\prod_{j=1}^{i-1}\left(1-\alpha_{j}\right)}$ (2) $$

where $\alpha\in\mathbb{R}$ is the opacity, and $\bm{c}\in\mathbb{R}^{3}$ is the view-dependent color of each Gaussian, which is calculated from $3$-degree Spherical Harmonics (SH) $\in\mathbb{R}^{48}$. In this paper, we design our FCGS model based on this SH-based rendering structure of 3DGS.

Discussions. The attributes of Gaussians can be categorized into geometry and color attributes. The geometry attributes (opacity $\bm{\alpha}$, diagonal elements of scaling $\bm{S}$, and quaternion representation of rotation $\bm{R}$) determine the dependencies of the rasterization process (e.g., the coverage range of the Gaussians or the depth to which they should be composed). Once the geometric dependencies are established, the color attributes are used to assign colors via $3$-degree SH following (Kerbl et al., ).
Since the geometry attributes directly influence the rasterization dependencies, they are more sensitive to deviations than the color attributes. For brevity, we denote geometry attributes as $\bm{f}^{\text{geo}}\in\mathbb{R}^{8}$, color attributes as $\bm{f}^{\text{col}}\in\mathbb{R}^{48}$, and the concat of both as $\bm{f}^{\text{gau}}\in\mathbb{R}^{56}$. In the following sections, unless otherwise specified, $\bm{x}$ refers to either $\bm{f}^{\text{geo}}$ or $\bm{f}^{\text{col}}$.

Figure: Figure 2: Our approach is inspired by image compression, where the input Gaussian attributes $\bm{x}$ is mapped into the latent space $\hat{\bm{y}}$ for compression after passing through an analysis transform $g_{a}$ and quantization to eliminate redundancies. To compress $\hat{\bm{y}}$, a hyperprior branch is introduced, using the coarse representation $\hat{\bm{z}}$ to estimate the distribution parameters of $\hat{\bm{y}}$ under a Gaussian distribution assumption, which aids in entropy encoding and decoding. In addition to the hyperprior, various context models are applied to $\hat{\bm{y}}$ to improve the estimation of distribution probabilities. After decoding $\hat{\bm{y}}$, a synthesis transform $g_{s}$ projects it back to the original space as $\hat{\bm{x}}$. A loss function is used to maintain high fidelity between $\hat{\bm{x}}$ and $\bm{x}$ using their rendered images, while minimizing the entropy of $\hat{\bm{y}}$ and $\hat{\bm{z}}$. AE and AD represent Arithmetic Encoding and Arithmetic Decoding, respectively.
Refer to caption: x2.png

### 3.2 Value-Based Principle: Multi-path Entropy Module (MEM)

As shown in Figure 2, we draw inspiration from image compression and apply an autoencoder-based compression approach with a hyperprior structure, where the Gaussian attributes $\bm{x}$ are projected into a quantized latent space $\hat{\bm{y}}$ for compression. After obtaining $\hat{\bm{y}}$ through the arithmetic decoding, a synthesis transform $g_{s}$ is employed to project it back to the original space $\hat{\bm{x}}$, which is the decoded version.
However, in 3DGS representations, Gaussians are not the final output; the actual results are the rendered images obtained through rasterization. Unfortunately, deviations in Gaussian attributes can be amplified during the rasterization process, leading to even greater deviations in the rendered images. Some Gaussians are highly sensitive to these deviations, as they may occupy crucial positions that significantly affect the rasterization process. Simply feeding all attributes into the autoencoder does not yield satisfactory results: even without quantization or entropy constraints, allowing $\hat{\bm{x}}$ to best approximate $\bm{x}$, the potential nonlinearity of the MLPs still causes considerable deviations in the decoded Gaussians. When amplified in the rendered images, these deviations lead to a sharp drop in fidelity, as exhibited in the ablation study (Subsection 4.3).

Geometry attributes are the most sensitive to deviations because they directly impact dependencies during rasterization. For example, if the decoded opacity of a Gaussian is too large, it could block Gaussians behind it, preventing them from contributing as they should. Similarly, deviations in scaling and rotation can have significant effects. To address this, we remove $g_{a}$ and $g_{s}$ for all geometry attributes $\bm{f}^{\text{geo}}$, ensuring that $\bm{y}=\bm{x}$.
Fortunately, color attributes are less sensitive to deviations, although they still impact the final result. For $\bm{f}^{\text{col}}$, we introduce MEM to deduce a binary mask $m\in\mathbb{R}$ that adaptively determines whether an $\bm{x}$ (only for $\bm{f}^{\text{col}}$ here) should pass through the MLPs $g_{a}$ and $g_{s}$ to eliminate redundancies, resulting in $\bm{y}=g_{a}(\bm{x})$, or simply bypass the MLPs, setting $\bm{y}=\bm{x}$ to best preserve the original information,

$$ $\hat{\bm{x}}_{i}=g_{s}(\texttt{quant}(g_{a}(\bm{x}_{i})))\times m_{i}+\texttt{ quant}(\bm{x}_{i})\times(1-m_{i}),\;m_{i}=\texttt{binary}(\text{MLP}_{\text{m} }(\bm{f}^{\text{gau}}_{i}))$ (3) $$

where quant represents the quantization operation: if $\bm{y}_{i}$ is in the latent space ($m_{i}=1$), the quantization step is $1$; otherwise ($m_{i}=0$), the step becomes a trainable decimal parameter. During training, uniform noise is added, while during testing, quantization is applied. The binary operation binarizes the mask values to either 0 0 or $1$ using STE (Bengio et al., ).

Balancing the mask rate is crucial, as it directly impacts the RD trade-off. Previous works like Lee et al. () incorporate an additional loss term with a hyperparameter $\lambda_{m}$ to regulate the mask rate. However, given that the RD loss already includes a trade-off hyperparameter $\lambda$ to balance bits and fidelity, introducing another parameter, $\lambda_{m}$, would unnecessarily complicate the process of finding the optimal RD trade-off. To resolve this, we integrate the mask information directly into the bit consumption calculation, allowing the model to adaptively learn the optimal mask rate, thereby eliminating $\lambda_{m}$. Please refer to Subsection 3.4 for details.

### 3.3 Structure-Based Principle: Autoregressive Context Models

To further enhance the accuracy of probability estimation, context models play a crucial role for $\hat{\bm{y}}$. Specifically, a portion of $\hat{\bm{y}}$ is first decoded and then used to assist in predicting the remaining part based on contextual relationships. In image compression, where data like $\hat{\bm{y}}$ are arranged in a structured grid (i.e., a feature map), designing such context models is relatively straightforward. However, Gaussians in 3DGS are inherently sparse and unorganized (as is $\hat{\bm{y}}$), making it a significant challenge to design efficient and lightweight context models. To tackle this, we analyze the unique properties of Gaussians and develop customized inter- and intra-Gaussian context models, as illustrated in Figure 3.

Figure: Figure 3: Context models of our FCGS approach. We build upon the hyperprior design from image compression (top) and introduce our inter- and intra-Gaussian context models (mid & bottom). Together, they form a GMM that provides a more accurate estimation of the value distribution probability of $\hat{\bm{y}}$ (right).
Refer to caption: x3.png

Inter-Gaussian Context Models. 3D Gaussians collectively represent the scene, leading to inherent relationships among them. To uncover these relationships, we refer to grid-based structures, which are typically compact, organized, and capable of constructing spatial connections among Gaussians. While this “grid” structure does not naturally exist within 3DGS, we propose an innovative method to *create grids* directly from Gaussians themselves.

We divide all Gaussians’ $\hat{\bm{y}}$ into $N^{\text{s}}$ batches ($N^{\text{s}}=2$ as an example in Figure 3 mid) and decode each batch sequentially. During this process, the previously decoded $\hat{\bm{y}}\in\mathbb{Y}_{[0,n^{\text{s}}-1]}$ are first used to create grids, which then provide context for the to-be-decoded $\hat{\bm{y}}\in\mathbb{Y}_{[n^{\text{s}}]}$ via interpolation. Here, $\mathbb{Y}_{[n^{\text{s}}]}$ refers to the set of $\hat{\bm{y}}$ belonging to the $n^{\text{s}}$-th batch out of $N^{\text{s}}$. To generate the feature $\bm{f}^{\bm{v}}$ for a voxel at position $\bm{v}$ on the grids, we investigate $\hat{\bm{y}}$ within $\bm{v}$’s interpolation range. For a voxel located at $\bm{v}_{i}\in\mathbb{R}^{3}$ (i.e., the red point in Figure 3 mid (b)), its feature $\bm{f}^{\bm{v}}_{i}$ is computed as follows:

$$ $\bm{f}^{\bm{v}}_{i}=\frac{\sum_{k:\hat{\bm{y}}_{k}\in\mathbb{Y}_{[0,n^{\text{s }}-1]}^{\bm{v}_{i}}}w_{k}\hat{\bm{y}}_{k}}{\sum_{k:\hat{\bm{y}}_{k}\in\mathbb{ Y}_{[0,n^{\text{s}}-1]}^{\bm{v}_{i}}}w_{k}},\quad\text{where}\;w_{k}=\prod_{ \text{dim}\in\{x,y,z\}}\left(1-|\bm{\mu}^{\text{g}}_{k,\text{dim}}-\bm{v}_{i, \text{dim}}|\right)$ (4) $$

where $\mathbb{Y}_{[0,n^{\text{s}}-1]}^{\bm{v}_{i}}$ denotes the subset of $\mathbb{Y}_{[0,n^{\text{s}}-1]}$ where $\hat{\bm{y}}$ fall into $\bm{v}_{i}$’s interpolation range (i.e., the orange box in Figure 3 mid (a,b,c)).
$k$ is the index of $\hat{\bm{y}}_{k}$, and $w_{k}$ represents the weight that determines the contribution of each $\hat{\bm{y}}_{k}$. The closer $\hat{\bm{y}}_{k}$ is to $\bm{v}_{i}$, the greater its contribution. Regardless of the number of $\hat{\bm{y}}$ within $\bm{v}_{i}$’s interpolation range, we can always compute a weighted average feature $\bm{f}^{\bm{v}}_{i}$ for ${\bm{v}_{i}}$ that integrates information for interpolation in the next step.

By conducting this calculation for all the voxels, we can create the grids, which are organized and structured, enabling them to provide contextual information for any input coordinates via interpolation. For a to-be-decoded $\hat{\bm{y}}_{i}\in\mathbb{Y}_{[n^{\text{s}}]}$, located at $\bm{\mu}_{i}^{\text{g}}$, its distribution parameters $\bm{\mu}^{\text{s}}_{i}$, $\bm{\sigma}^{\text{s}}_{i}$, and $\bm{\pi}^{\text{s}}_{i}$ can be computed from these grids using $\text{MLP}_{\text{s}}$:

$$ $\bm{\mu}^{\text{s}}_{i},\bm{\sigma}^{\text{s}}_{i},\bm{\pi}^{\text{s}}_{i}= \text{MLP}_{\text{s}}({{\oplus}}[\bm{f}^{\bm{\mu}^{\text{g}}}_{i},\texttt{emb} (\bm{\mu}^{\text{g}}_{i})]),\quad\text{where}\;\bm{f}^{\bm{\mu}^{\text{g}}}_{i }=\sum_{k:\bm{v}_{k}\in\mathbb{V}^{\bm{\mu}^{\text{g}}_{i}}}w_{k}\bm{f}^{\bm{v }}_{k}$ (5) $$

where $\mathbb{V}^{\bm{\mu}^{\text{g}}_{i}}$ represents the set of voxels forming $\bm{\mu}_{i}^{\text{g}}$’s minimum bounding box (i.e., the $4$ red points in Figure 3 mid (c)),
and $\bm{f}^{\bm{\mu}^{\text{g}}}_{i}$ is the feature obtained by grid interpolation. emb and ${{\oplus}}$ are sin-cos positional embedding and channel-wise concatenate operation, respectively. The calculated parameters $\bm{\mu}^{\text{s}}_{i}$, $\bm{\sigma}^{\text{s}}_{i}$, and $\bm{\pi}^{\text{s}}_{i}$ are then used to estimate the distribution of $\hat{\bm{y}}_{i}$ in GMM.

In practice, we use both 3D and 2D grids, designed with multiple resolutions to capture varying levels of detail. The 3D grids effectively represent the spatial relationships between the Gaussians and the voxels, though they are of lower resolution due to the higher dimensionality. To complement this, we employ three 2D grids, each created by collapsing one dimension. These 2D grids provide higher resolutions, allowing for a more precise capture of local details.

Intra-Gaussian Context Models. Beyond addressing redundancies across Gaussians, we further utilize our FCGS to eliminate redundancies within each Gaussian. Specifically, we split each $\hat{\bm{y}}_{i}$ into $N^{\text{c}}$ chunks, and $\text{MLP}_{\text{c}}$ is used to deduce the distribution parameters for each chunk from the previously decoded ones. For path $m=1$ in MEM, we split $\hat{\bm{y}}$ into $N^{\text{c}}=4$ chunks along channel dimension. For path $m=0$, $\hat{\bm{y}}$ is split into $N^{\text{c}}=3$ chunks along RGB axis to leverage redundancies of color components. We do not apply intra-context for $\bm{f}^{\text{geo}}$, as its internal relations are trivial.

$$ $\bm{\mu}^{\text{c}}_{i},\bm{\sigma}^{\text{c}}_{i},\bm{\pi}^{\text{c}}_{i}={{ \oplus}}_{n^{\text{c}}=1}^{N^{c}}\{\bm{\mu}^{\text{c}}_{i,n^{\text{c}}},\bm{ \sigma}^{\text{c}}_{i,n^{\text{c}}},\bm{\pi}^{\text{c}}_{i,n^{\text{c}}}\},\; \text{where}\;\bm{\mu}^{\text{c}}_{i,n^{\text{c}}},\bm{\sigma}^{\text{c}}_{i,n ^{\text{c}}},\bm{\pi}^{\text{c}}_{i,n^{\text{c}}}=\text{MLP}_{\text{c}}(\hat{ \bm{y}}_{i,[0,n^{\text{c}}c-c)})$ (6) $$

where $n^{\text{c}}$ and $c$ are chunk index and channel amount per chunk, respectively. $\bm{\mu}^{\text{c}}_{i,n^{\text{c}}},\bm{\sigma}^{\text{c}}_{i,n^{\text{c}}},
\bm{\pi}^{\text{c}}_{i,n^{\text{c}}}$ are the intermediate probability parameters for chunk $n^{\text{c}}$, with a dimension size of $c$ for each.

Gaussian Mixed Model (GMM). In addition to the context models, hyperprior also outputs a set of distribution parameters $\bm{\mu}^{\text{h}}_{i}$, $\bm{\sigma}^{\text{h}}_{i}$ and $\bm{\pi}^{\text{h}}_{i}$ from $\hat{\bm{z}}_{i}$. To this end, we introduce GMM to represent the final probability as a combination of these $3$ sets of Gaussian distribution parameters. For any $\hat{\bm{y}}_{i}$ at $j$-th channel, its probability can be calculated as a combination of Gaussian distribution $\mathcal{N}$,

$$ $p(\hat{y}_{i,j})=\sum_{l\in{\{\text{h},\text{s},\text{c}\}}}{\phi}_{i,j}^{l} \mathcal{N}(\hat{{y}}_{i,j}\mid{\mu}_{i,j}^{l},{\sigma}_{i,j}^{l}),\quad\text{ where}\;{\phi}_{i,j}^{l}=\frac{\exp({\pi}_{i,j}^{l})}{\sum_{l\in{\{\text{h}, \text{s},\text{c}\}}}\exp({\pi}_{i,j}^{l})}$ (7) $$

Note that for geometry attributes $\bm{f}^{\text{geo}}$, $l\in{\{\text{h},\text{s}\}}$. GMM adaptively weights the mixing of the three sets of distributions, providing more accurate probability estimation. For $\hat{\bm{z}}$ in the hyperprior branch, we follow Ballé et al. () to use a factorized module to estimate its $p(\hat{{z}}_{i,j})$. According to information theory (Cover, ), bit consumption of $\bm{x}_{i}$ can be calculated: $bit_{i}=\sum_{j}^{D^{y}}(-\log_{2}{p(\hat{y}_{i,j})})+\sum_{j}^{D^{z}}(-\log_{
2}{p(\hat{z}_{i,j})})$. $D^{y}$ and $D^{z}$ are number of channels of $\hat{\bm{y}}_{i}$ and $\hat{\bm{z}}_{i}$.

### 3.4 Training and Coding Process

Training loss. After obtaining $\hat{\bm{x}}$ from $\hat{\bm{y}}$ through the synthesis transform $g_{s}$, we can evaluate the fidelity between the ground truth $\bm{I}$ and the image $\hat{\bm{I}}$ rendered from $\hat{\bm{x}}$.
To balance the mask rate of $\bm{f}^{\text{col}}$ in MEM, we include the mask $m$ directly in the bit calculation process, rather than introducing an additional loss term. Specifically, we forward *all* Gaussians’ $\bm{f}^{\text{col}}$ in both MEM paths and use $m$ to compute the weighted sum of the output bit from both paths, ensuring the gradients can be backpropagated for both. The overall loss function is:

$$ $L=L_{\text{fidelity}}(\hat{\bm{I}},{\bm{I}})+\lambda\frac{L_{\text{entropy}}}{ N^{\text{g}}\times 56},\;\text{where}\;L_{\text{entropy}}=\sum_{i=1}^{N^{\text {g}}}bit_{i}^{\text{geo}}+m_{i}bit_{i}^{\text{col}\mid m_{i}=1}+(1-m_{i})bit_{ i}^{\text{col}\mid m_{i}=0}$ (8) $$

where ${N^{\text{g}}}$ represents the amount of Gaussians, $56$ is the dimension of $\bm{f}^{\text{gau}}$, and they collectively represent the amount of attribute parameters. $bit_{i}^{\text{geo}}$ and $bit_{i}^{\text{col}}$ represents the bit consumption for a Gaussian’s geometry and color attributes, respectively, with $\bm{x}_{i}$ set as $\bm{f}^{\text{geo}}_{i}$ or $\bm{f}^{\text{col}}_{i}$, respectively. $\lambda$ is a hyperparameter controlling the trade-off between fidelity and entropy. The fidelity loss, $L_{\text{fidelity}}$ includes both MSE and SSIM metrics to evaluate the reconstruction quality of $\hat{\bm{x}}$. Minimizing this loss encourages $\hat{\bm{x}}$ to closely resemble the original $\bm{x}$ while also reducing the bit consumption.

Encoding/decoding process. For encoding, the attributes $\bm{f}^{\text{geo}}$ and $\bm{f}^{\text{col}}$ are encoded using Arithmetic Encoding (AE) (Witten et al., ) in the space of $\hat{\bm{y}}$ and $\hat{\bm{z}}$ given the corresponding probabilities, masks $m$ are binary and encoeded using AE based on the occurrence frequency of $1$, and Gaussian coordinates $\bm{\mu}^{\text{g}}$ are 16-bit quantized and encoded losslessly using GPCC (Chen et al., ). For decoding, Gaussian coordinates $\bm{\mu}^{\text{g}}$, masks $m$, and hyperpriors $\hat{\bm{z}}$ are decoded first. Then, to decode $\hat{\bm{y}}_{i,[n^{\text{c}}c-c,n^{\text{c}}c)}\mid\hat{\bm{y}}_{i}\in\mathbb
{Y}_{[n^{\text{s}}]}$, its value distribution is calculated using GMM based on: $\bm{\mu}^{\text{s}}_{i,n^{\text{c}}}$, $\bm{\sigma}^{\text{s}}_{i,n^{\text{c}}}$, $\bm{\pi}^{\text{s}}_{i,n^{\text{c}}}$ from $\hat{\bm{y}}\in\mathbb{Y}_{[0,n^{\text{s}}-1]}$ using inter-Gaussian context models (channel sliced), and $\bm{\mu}^{\text{c}}_{i,n^{\text{c}}}$, $\bm{\sigma}^{\text{c}}_{i,n^{\text{c}}}$, $\bm{\pi}^{\text{c}}_{i,n^{\text{c}}}$ from $\hat{\bm{y}}_{i,[0,n^{\text{c}}c-c)}\mid\hat{\bm{y}}_{i}\in\mathbb{Y}_{[n^{
\text{s}}]}$ using intra-Gaussian context models, and $\bm{\mu}^{\text{h}}_{i,n^{\text{c}}}$, $\bm{\sigma}^{\text{h}}_{i,n^{\text{c}}}$, $\bm{\pi}^{\text{h}}_{i,n^{\text{c}}}$ from $\hat{\bm{z}}_{i}$ using hyperprior (channel sliced). On obtaining GMM, it is decoded using AD.

## 4 Experiments

### 4.1 Implementation Details

Implementation. Our FCGS model is implemented using the PyTorch framework (Paszke et al., ) and trained on a single NVIDIA L40s GPU. The dimension of $\hat{\bm{y}}$ is set to 256 for color ($m=1$). For $\hat{\bm{z}}$, dimensions are set to 16, 24, and 64 for geometry, color ($m=0$), and color ($m=1$), respectively. Grid resolutions are $\{70,80,90\}$ for 3D grids and $\{300,400,500\}$ for 2D grids. We set $N^{\text{s}}$ to 4, using uneven splitting ratios of $\{\frac{1}{6},\frac{1}{6},\frac{1}{3},\frac{1}{3}\}$, with uniform random sampling. In inference, we maintain a same random seed in encoding and decoding to guarantee consistency. The training batch size is $1$ (i.e., one 3DGS scene per training step). We adjust $\lambda$ from $1e-4$ to $16e-4$ to achieve variable bitrates.
During training, we first train $g_{a}$ and $g_{s}$ of $m=1$ using only the fidelity loss to ensure satisfactory reconstruction quality. We then jointly train the context models for color with $m=1$, and finally, train the entire model in an end-to-end manner. We adopt this training process because the gradient chain for the $m=0$ path is shorter than that of the $m=1$ path. Without sufficient pre-training of the $m=1$ path, the model is prone to collapsing into a local minimum where all $m$ values are zero.

Training Dataset. FCGS requires training on a large-scale dataset with abundant 3DGS. To achieve that, we refer to DL3DV dataset (Ling et al., ), which contains approximately $7K$ multi-view scenes. We train these scenes to generate their 3DGS at a resolution of $960$P, which takes about $60$ GPU days on NVIDIA L40s. After filtering out low-quality ones, we obtain $6770$ 3DGS, and randomly split $100$ for testing and the remaining for training. This dataset is referred to as DL3DV-GS. For more details regarding DL3DV-GS, please refer to Appendix Section A.

Metrics. We assess compression performance in terms of fidelity relative to size. Herein, we present PSNR metric to evaluate fidelity due to page constraints. For additional metrics (SSIM (Wang et al., ) and LPIPS (Zhang et al., )), please refer to Appendix Section E.

### 4.2 Experiment Evaluation

For 3DGS from optimization, FCGS enables rapid compression. Since no prior works customize optimization-free compression for 3DGS, direct comparisons are unavailable. This leaves us to benchmark against optimization-based methods, a comparison that is inherently unfair to FCGS. Nonetheless, we achieve excellent RD performance despite this lack of optimization. Furthermore, FCGS can also compress 3DGS generated by feed-forward models (Chen et al., ; Tang et al., ), demonstrating its versatility.

Figure: Figure 4: Performance comparison. Each scene is initially trained for $30K$ iterations to produce the vanilla 3DGS. Methods marked with * and circle are finetuned from this common 3DGS (our FCGS also compresses the same 3DGS); Methods marked with ** and triangles are trained from scratch due to their modification to structures. We also present the runtime of our method and other approaches at the bottom of the figure (which is also reflected by the size of the marks), where our approach requires significantly less time for compression. For our runtime, it means using multiple/single GPUs. Thanks to our optimization-free pipeline, we divide the 3DGS into chunks, with each chunk containing $1$ million Gaussians, allowing us to easily encode these chunks in parallel using multiple GPUs, further speeding up the process.
Refer to caption: x4.png

Figure: Figure 5: Qualitative comparison between our FCGS and 3DGS. We achieve substantial size reduction while preserving high fidelity. PSNR (dB) / SIZE (MB) are indicated in the bottom-right corner.
Refer to caption: x5.png

Figure: Figure 6: Compression of 3DGS from feed-forward models. Left: MVSplat is a generalizable reconstruction model, whose fidelity is measured between the rendered images and the ground truth. Right: LGM is a generative model, whose fidelity is measured between images rendered from 3DGS before and after compression, as no ground truth cannot be measured due to its generative nature. The x-axis uses a Log10 scale.
Refer to caption: x6.png

For 3DGS from optimization, we employ DL3DV-GS, MipNeRF360 (Barron et al., ), and Tank&Temples (Knapitsch et al., ) for evaluation. For the baseline methods, we compare those built upon the vanilla 3DGS structure, including both finetune from existing 3DGS  (Navaneet et al., ; Niedermayr et al., ; Fan et al., ) or train from scratch (Morgenstern et al., ; Girish et al., ; Lee et al., ). The results are in Figure 4.
Although our FCGS lacks per-scene adaptation, which naturally puts us at a disadvantage for unfair comparison, it still surpasses most optimization-based methods, thanks to the effectiveness of MEM and context models.
The qualitative comparisons are shown in Figure 5, which exhibit high fidelity after compression.

For 3DGS from feed-forward models, we also demonstrate compression capability. To evaluate our performance, we refer to MVSplat (Chen et al., ) and LGM (Tang et al., ). MVSplat is a generalizable model that reconstructs interpolated novel views given two bounded views. LGM is a generative model that creates the 3D scenes from the $4$ multi-view images. We follow LGM to generate the remaining $3$ views from the initial $1$ view using Wang & Shi (), which are then collectively input into LGM for 3DGS creation. This process results in ground-truth cannot be measured for LGM, thus we evaluate the compression fidelity by measuring the similarities of images rendered from 3DGS before or after compression. We utilize $10$ scenes from ACID (Liu et al., ) and $50$ scenes from Gobjaverse (Qiu et al., ; Deitke et al., ) for these two models for evaluation. The results are shown in Figure 6. Although trained on 3DGS from optimization (Kerbl et al., ), FCGS still generalizes in a zero-shot manner to feed-forward-based 3DGS, achieving compression ratios of $15\times$ and $5\times$. Notably, when compressing 3DGS from feed-forward models, we set mask $m$ to all 0 0s for color attributes. Please refer to Appendix Section B for further analysis.

Figure: Figure 7: Ablation studies on the DL3DV-GS dataset. Left: Setting the mask for color to all 0 0s results in a significant increase in bit consumption. Conversely, setting the mask to all $1$s leads to a drastic drop in fidelity, even without applying quantization or entropy constraints to $\bm{y}$. Right: Excluding the proposed context models leads to a substantial increase in bit consumption, due to the absence of mutual dependencies.
Refer to caption: x7.png

### 4.3 Ablation Study

We perform ablation studies to evaluate the effectiveness of our proposed MEM and context models.
First, we investigate the MEM module, which adaptively selects high-tolerance color attributes to eliminate redundancies via an autoencoder structure, while ensuring all geometry attributes are directly quantized. Without MEM, we observe the following:
1) If both color and geometry attributes are processed through the autoencoder, the model collapses because rasterization cannot be performed correctly due to deviations in the geometry information.
2) Fixing the geometry attributes to be directly quantized, we allow all color attributes to be processed by the autoencoder (i.e., all $m=1$). As shown in Fig 7 left, even without quantization or entropy constraints on $\bm{y}$, the rendered images suffer from a significant drop in fidelity due to deviations brought by MLPs.
3) On the other hand, if all color attributes are directly quantized (i.e., all $m=0$), their redundancies are not effectively eliminated, leading to increased storage costs.

Next, we explore the impact of our context models. As demonstrated in Figure 7 right, removing intra- and inter-Gaussian context models progressively decreases RD performance. Compared to the full FCGS model, the bit consumption is $1.5\times$ for the base model under similar fidelity conditions.

Figure: Figure 8: Building on pruning approaches, FCGS can further boost compression performance. The size of the compressed 3DGS using FCGS (at the highest rate) is only $40\%$ that of MSC with the same fidelity. When combined with pruning techniques, FCGS achieves a compression ratio of $100\times$ over the vanilla 3DGS (see Mini**+Ours). Experiments are conducted on the Deep Blending dataset. The x-axis uses a Log10 scale.
Refer to caption: x8.png

### 4.4 Boost existing compression approaches

The vanilla 3DGS (Kerbl et al., ) sometimes struggles with densification issues, leading to suboptimal fidelity. Pruning techniques effectively eliminate trivial Gaussians, enhancing fidelity while reducing size, particularly evident with the Deep Blending dataset (Hedman et al., ). Importantly, FCGS is compatible with these pruning techniques. We refer to Trimming (Ali et al., ) and Mini-Splatting (Fang & Wang, ), both of which apply pruning to Gaussians. As shown in Figure 8, FCGS significantly boosts their compression performance. Notably, MSC is a straightforward optimization-free compression tool utilized in Fang & Wang (). We compare the compression performance of FCGS and MSC by directly applying them to the same pruned 3DGS representations. The superior compression performance of FCGS underscores its significance. Please refer to Appendix Subsection E.2 to find the results on the other three datasets.

### 4.5 Coding and rendering efficiency analysis

Our coding time includes GPCC (Chen et al., ) coding for coordinates $\bm{\mu}^{\text{g}}$ and arithmetic coding for attributes $\bm{f}^{\text{geo}}$ and $\bm{f}^{\text{col}}$ and masks $m$. Approximately $60\%$ of the total time is spent on GPCC due to its complex RDO process during encoding. On average, FCGS takes about $1$ second to encode $100K$ Gaussians when running on a single GPU.
The rendering time of the decoded 3DGS remains consistent with that before compression since FCGS does not alter the number or structure of the Gaussians. For instance, the average FPS is $102$ and $91$ before and after compression ($\lambda=1e-4$) on the MipNeRF360 dataset.

## 5 Conclusion

In this paper, we introduce a pioneering generalizable optimization-free compression pipeline for 3DGS representations and propose our FCGS model. FCGS enables fast compression of existing 3DGS without any finetuning, offering significant time savings. More importantly, we achieve impressive RD performance, exceeding $20\times$ compression, through the meticulous design of the MEM module and context models. Our approach can also boost compression performance of pruning-based methods. Overall, this new compression pipeline has the potential to significantly enhance the widespread application of 3DGS compression techniques due to its numerous advantages.

## Appendix A Statistical Data for DL3DV-GS

Using the DL3DV dataset (Ling et al., ), we generate our DL3DV-GS dataset through per-scene optimization with 3DGS (Kerbl et al., ), resulting in a total of $6770$ 3DGS representations. In this section, we present statistical data for the DL3DV-GS dataset, including its size and fidelity metrics, as exhibited below.
Note that, during per-scene optimization, we adhere to the 3DGS train-test view splitting strategy, using $1$ view for testing and $7$ views for training across every $8$ consecutive views. After optimization, we randomly select $100$ scenes for testing, leaving the rest for training. All the approaches are evaluated on the test scenes. For our FCGS, it is evaluated on test views, while for other training-based methods, they are optimized (or trained) on training views and then evaluated on test views.

Figure: Figure A: Statistical data of the DL3DV-GS dataset. We also mark the mean values of each data in the sub-figures.
Refer to caption: x9.png

We also provide example images of this dataset. The images are randomly selected from test views from test scenes, which presents high fidelity.

Figure: Figure B: We randomly select several test views from test scenes for visualization, which showcase high-fidelity quality. The PSNR (dB) values for these four example scenes are indicated at the bottom-right corner of each image, which exhibit high fidelity.
Refer to caption: x10.png

## Appendix B Analysis of Using All Zero Masks to Compress Generated 3DGS

To compress 3DGS from feed-forward models, we set the mask $m$ to all 0 0s for color attributes, instead of using MEM to adaptively infer the mask $m$. This is because the characteristics in 3DGS from feed-forward models differ significantly from those in our training data (i.e., 3DGS from optimization).
Specifically, for MVSplat (Chen et al., ; Charatan et al., ), it uses an SH of $4$ degrees (which we truncate to $3$ degrees without observing significant fidelity loss), and applies a weighted mask that assigns lower weights to higher-degree SH. This results in the values of high-degree SH in its 3DGS being one order of magnitude smaller than in ours. For LGM (Tang et al., ), it uses an SH degree of 0 0 (i.e., it only has DC coefficients). To make our model compatible with its 3DGS, we need to pad zeros for high-degree coefficients to match the shape.
Due to these differences, their 3DGS values vary greatly from ours, making autoencoders less effective and leading to fidelity drops in the $m=1$ path. Additionally, the variance in values would cause the mask $m$ to be incorrectly inferred, as the mask itself is also deduced from these varying attributes using $\text{MLP}_{\text{m}}$. However, for $m=0$, where the inputs are directly quantized with a small step size, this variance can be well tolerated. To this end, we opt for all $m=0$ to compress these models, which has also shown excellent compression performnce thanks to the design of context models.
Overall, this presents an interesting and meaningful problem in the future, raising the question: how to design optimization-free compression models that can be better generalized to 3DGS with different value characteristics. We leave this as a future work to explore.

## Appendix C Effect of Different Random Seeds on Gaussian Splitting

In our inter-Gaussian context models, we employ a random splitting strategy to uniformly divide all Gaussians into $N^{\text{s}}$ batches and decode them progressively. For decoding, we maintain the same random seed as encoding to guarantee the consistency of the contexts. In this section, we investigate the effect of different random seeds on this splitting process and its impact on the final results. Note that this only affects bit consumption but does not influence reconstruction fidelity. By testing $5$ different random seeds over the test set of DL3DV-GS with $\lambda=1e-4$, we observed a standard deviation of $8e-4$ MB, with a Coefficient of Variation (i.e., standard deviation divided by the mean) of $3.6e-5$. This experiment demonstrates the stability of our splitting approach with respect to the random seed.

We also investigated other sampling strategies like Farthest Point Sampling (FPS) in (Qi et al., ). Since FPS is very time-consuming for massive points in 3DGS, we evaluated only scenes with fewer than $500K$ Gaussians in the test set of DL3DV-GS. We observed an average bit increase of $0.6\%$. This is because FPS tends to sample points located at corners (which have the farthest distances from others), leading to non-uniform splitting and affecting the accuracy of context models.

## Appendix D Mask Ratio Analysis in MEM

We provide statistical data on the ratio of path $m=1$ in MEM. As $\lambda$ increases, the model is expected to achieve lower bitrates at the cost of reduced fidelity. To accomplish this, the model tends to assign more $1$s to the mask $m$, thereby allowing more color attributes $\bm{f}^{\text{col}}$ to eliminate redundancies via the autoencoder (i.e., the path of $m=1$), and vice versa.

**Table B: Ratio of path $m=1$ in MEM.**
| $\lambda$ | DL3DV-GS | MipNeRF360 | Tank&Temples |
| --- | --- | --- | --- |
| $1e-4$ | 0.62 | 0.48 | 0.60 |
| $2e-4$ | 0.67 | 0.55 | 0.66 |
| $4e-4$ | 0.74 | 0.63 | 0.74 |
| $8e-4$ | 0.82 | 0.74 | 0.82 |
| $16e-4$ | 0.89 | 0.85 | 0.90 |

## Appendix E Additional Experimental Results

### E.1 More Fidelity Metrics and Training Time

We provide additional fidelity metrics, including PSNR, SSIM, and LPIPS. We also report the training time for each method. For those marked with *, it represents the time taken for finetuning from a common existing 3DGS. For those marked with **, it represents the time taken for training from scratch. For our method, it is the encoding time using multiple/single GPUs.

**Table C: Experiments on DL3DV-GS dataset. Methods marked with * are finetuned from a common 3DGS (our FCGS compresses the same 3DGS). Methods marked with ** are trained from scratch.**
| Methods | PSNR (dB) $\uparrow$ | SSIM $\uparrow$ | LPIPS $\downarrow$ | SIZE (MB) $\downarrow$ | TIME (s) $\downarrow$ |
| --- | --- | --- | --- | --- | --- |
| 3DGS | 29.34 | 0.898 | 0.146 | 372.50 | 751 |
| Light* | 28.89 | 0.890 | 0.159 | 24.61 | 227 |
| Navaneet* | 28.36 | 0.884 | 0.169 | 13.60 | 546 |
| Simon* | 29.04 | 0.893 | 0.154 | 15.36 | 122 |
| SOG** | 28.54 | 0.888 | 0.156 | 16.50 | 1068 |
| EAGLES** | 28.53 | 0.884 | 0.170 | 24.04 | 518 |
| Lee** | 28.71 | 0.886 | 0.165 | 36.56 | 938 |
| Ours-lowrate | 28.86 | 0.891 | 0.156 | 15.83 | 9 / 16 |
| Ours-highrate | 29.26 | 0.897 | 0.148 | 27.44 | 11 / 20 |

**Table D: Experiments on MipNeRF360 dataset. Methods marked with * are finetuned from a common 3DGS (our FCGS compresses the same 3DGS). Methods marked with ** are trained from scratch.**
| Methods | PSNR (dB) $\uparrow$ | SSIM $\uparrow$ | LPIPS $\downarrow$ | SIZE (MB) $\downarrow$ | TIME (s) $\downarrow$ |
| --- | --- | --- | --- | --- | --- |
| 3DGS | 27.52 | 0.813 | 0.221 | 741.12 | 1583 |
| Light* | 27.31 | 0.808 | 0.235 | 48.61 | 422 |
| Navaneet* | 26.80 | 0.796 | 0.256 | 20.66 | 973 |
| Simon* | 27.15 | 0.802 | 0.242 | 27.71 | 195 |
| SOG** | 27.08 | 0.799 | 0.229 | 41.00 | 2391 |
| EAGLES** | 27.14 | 0.809 | 0.231 | 58.91 | 1245 |
| Lee** | 27.05 | 0.797 | 0.247 | 48.93 | 1885 |
| Ours-lowrate | 27.05 | 0.798 | 0.237 | 34.64 | 10 / 31 |
| Ours-highrate | 27.39 | 0.806 | 0.226 | 64.05 | 14 / 41 |

**Table E: Experiments on Tank&Temples dataset. Methods marked with * are finetuned from a common 3DGS (our FCGS compresses the same 3DGS). Methods marked with ** are trained from scratch.**
| Methods | PSNR (dB) $\uparrow$ | SSIM $\uparrow$ | LPIPS $\downarrow$ | SIZE (MB) $\downarrow$ | TIME (s) $\downarrow$ |
| --- | --- | --- | --- | --- | --- |
| 3DGS | 23.71 | 0.845 | 0.179 | 432.03 | 814 |
| Light* | 23.62 | 0.837 | 0.197 | 28.60 | 241 |
| Navaneet* | 23.15 | 0.831 | 0.205 | 13.64 | 587 |
| Simon* | 23.63 | 0.842 | 0.187 | 17.65 | 120 |
| SOG** | 23.66 | 0.837 | 0.187 | 23.10 | 1219 |
| EAGLES** | 23.28 | 0.835 | 0.203 | 28.99 | 604 |
| Lee** | 23.35 | 0.832 | 0.202 | 39.38 | 1007 |
| Ours-lowrate | 23.48 | 0.832 | 0.193 | 17.89 | 10 / 16 |
| Ours-highrate | 23.62 | 0.839 | 0.184 | 32.02 | 13 / 24 |

### E.2 Boost Compression Performance of Pruning-based Approaches

Mini-Splatting (Fang & Wang, ) and Trimming (Ali et al., ) effectively prune trivial Gaussians, thereby slimming the 3DGS. By applying our FCGS on top of their pruned 3DGS, we can achieve superior compression performance. We further evaluate this scheme across DL3DV-GS, MipNeRF360, and Tank&Temples datasets.

Figure: Figure C: Our FCGS boosts the compression performance of pruning-based approaches. Built on Mini-Splatting (Fang & Wang, ) and Trimming (Ali et al., ), we achieve superior RD performance by directly applying FCGS to their pruned 3DGS, without any finetuning.
Refer to caption: x11.png

### E.3 Ablation Studies on the Hyperprior

We present statistical data from our ablation studies, and also conduct ablation experiment over the hyperprior, as shown in Table G. This hyperprior approach follows the approach of Ballé et al. (), which is originally designed for image compression, and we consistently retain it in our FCGS model. In this subsection, we investigate the effect of the hyperprior by manually setting $\hat{\bm{z}}$ to all zeros to remove the information contributed from the hyperprior, while still preserving a necessary Gaussian probability for entropy estimation (which is now predicted from an all-zero hyperprior $\hat{\bm{z}}$ instead).

Interestingly, our experiments reveal that the hyperprior does not significantly contribute to the compression performance. This suggests that, there is less redundancy within each Gaussian attribute in 3DGS compared to that within each individual image, consequently, directly transplanting a hyperprior design from image compression to 3DGS compression is less effective. Instead, our inter- and intra-context models, which are tailored to the unique characteristics of 3DGS, have proven to be more effective.

**Table F: Statistical data from ablation studies over MEM on the DL3DV-GS dataset. Data are presented as PSNR (dB) / SIZE (MB).**
| $\lambda$ | $1e-4$ | $2e-4$ | $4e-4$ | $8e-4$ | $16e-4$ |
| --- | --- | --- | --- | --- | --- |
| Ours | 29.26 / 27.44 | 29.22 / 24.15 | 29.17 / 21.12 | 29.05 / 18.14 | 28.86 / 15.83 |
| Ours w $m$ all 0 0s | 29.27 / 38.15 | 29.23 / 33.87 | 29.16 / 30.12 | 28.98 / 25.99 | 28.69 / 22.84 |
| Ours w $m$ all $1$s | 28.82 / – |  |  |  |  |

**Table G: Statistical data from ablation studies over context models on the DL3DV-GS dataset. Data are presented as PSNR (dB) / SIZE (MB).**
| $\lambda$ | $1e-4$ | $2e-4$ | $4e-4$ | $8e-4$ | $16e-4$ |
| --- | --- | --- | --- | --- | --- |
| Ours | 29.26 / 27.44 | 29.22 / 24.15 | 29.17 / 21.12 | 29.05 / 18.14 | 28.86 / 15.83 |
| Ours w/o intra | 29.26 / 29.65 | 29.22 / 25.94 | 29.15 / 23.32 | 29.02 / 18.87 | 28.82 / 16.80 |
| Ours w/o intra & inter | 29.26 / 38.55 | 29.22 / 35.63 | 29.14 / 31.15 | 29.02 / 26.50 | 28.81 / 23.53 |
| Ours w/o intra & inter & hyper | 29.25 / 39.41 | 29.22 / 34.79 | 29.17 / 31.29 | 29.05 / 27.47 | 28.84 / 24.66 |

## Appendix F Additional Qualitative Comparisons

Figure: Figure D: Qualitative comparison between our FCGS and 3DGS. We achieve substantial size reduction while preserving high fidelity. PSNR (dB) / SIZE (MB) are indicated in the bottom-right corner.
Refer to caption: x12.png

## Peer Reviews

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: 1. Can authors provide visualization of inter- and intra-Gaussian context models on real input data. It would help reader better understand how this method work. 

2. Will the code or data open-source for reproduction ? the training set is interesting and would benefit many other tasks.
soundness: 3
strengths: 1. It proposes a novel optimization-free pipeline for 3DGS-based compression.
2. MEM module to adaptively control size and fidelity.
3. It also designs inter- and intra-Gaussian context models to enhance compression efficiency. 
4. The proposed method achieves a high compression rate and surpasses the existing optimized-based methods in various experiments.
summary: The paper proposes a new compression model for 3D Gaussian Splatting (3DGS) representations. It aims to reduce compression time in an optimization-free pipeline, which is different from existing work. It introduces a Multi-path Entropy Module (MEM) to adaptive control the compression size and quality. It also designs inter- and intra-Gaussian context models to enhance compression efficiency. Experiments on multiple datasets show a large compression rate and high image quality.
weaknesses: 1. The training data is very large, and it takes 60 GPU days to get. Per-scene optimization does not need such an amount of training data. Will smaller data hurt the effectiveness of the feedforward model?
2. More discussion of failure cases. it could be useful to discuss / show scenarios where the method performs poorly.

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 3
questions: - What are the analysis transform and synthesis transform exactly? Architecture? Space? 
- Why can "the potential nonlinearity of the MLPs still cause considerable deviations in decoded gaussians", line 222?
- How are the the set of Gaussian split and what is the criterion to split them?
soundness: 3
strengths: - The paper provides a complete overview of related works on 3DGS with compression and structure it according to the methodology, see Section 1 and 2.
- The overall goal of the paper is clear and the authors motivate compression potential for 3DGS scenes well.
- The quantitative evaluation is very good, as the authors provide a comparison to several relevant methods, like light gaussians and Niedermayr etal.,  on three different datasets (MipNeRF360, T&T and DL3DV-GS) and present results in expressive graphs that highlight the trade-off of size and quality, see Figure 4.
- In addition to comparisons to compression works, the author further extent t0 feed forward reconstruction models and find a more efficient representation of their approach.
- Overall the experiments indicate that the proposed methods is an effective approach for compression performs favourable to most other approaches in terms of (PSNR/SIZE), however less effective than, Niedermayr et al.. See Figure 4.
summary: This paper presents a novel, optimization-free compression pipeline for 3D Gaussian Splatting (3DGS) representations, called FCGS (Fast Compression of 3D Gaussian Splatting). FCGS uses a single feed-forward model to compress 3DGS representations, drastically reducing compression time compared to pruning and optimization-based approaches. The paper further details the Multi-path Entropy Module (MEM) and context models that FCGS employs to balance size and fidelity during compression. Finally, the authors demonstrate the efficacy of FCGS through extensive experiments, achieving a compression ratio exceeding 20x while preserving high fidelity.
weaknesses: - The methodology is very complex and the respective section in the paper is hard to understand even for someone familiar with 3DGS. The reason is that there is information missing at the appropriate position in the text. For example, figure 2 in line 262 is is unclear how the Gaussians are divided into batches (randomly, per grid cell stays unclear). Hence it appears to hard to reproduce. Please see the Questions sections for more open points.
- The method's focus lies very much on increasing the compression speed and it does not improve over existing methods in terms of PSNR/Size, see Figure 4 (Simon*). One could argue that there is limited interests in compression speed, since a 3D GS reconstruction takes several minutes to hours and a follow up compression of a few minutes (e.g. Niedermayr et al.) does not impact the overall reconstruction time significantly.
- For qualitative evaluation, I'd have expected more visual comparisons to baseline methods. There is only comparisons to 3DGS in paper and supplementary material. Please provide more qualitative evidence.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 4
questions: - How much time is needed for encoding and decoding parameters?
- It could be helpful to add compression performance for each attribute by demonstrating the storage size of each attribute before and after compression. (or storage size of color attributes and geometry attributes)
- How many scenes are required to acquire a feasible compression performance?
- An optimization-based compression work, ContextGS [1], has already applied an image compression-based pipeline for 3DGS using hyper-prior similar to the design of inter-Gaussian context models. However, the authors do not need to consider it, since ContextGS has not been published yet, accepted to NeurIPS 2024. Therefore, it would be helpful to mention this paper as a concurrent work.


(+minor question)
- Do you have any plans to release DL3DV-GS, which is a training dataset for this work? This could be helpful for various generalizable 3DGS approaches, including compression.


---
### Reference
[1] Wang et al., ContextGS: Compact 3D Gaussian Splatting with Anchor Level Context Model, NeurIPS 2024
soundness: 3
strengths: - This paper suggests a novel framework that efficiently compresses the 3D Gaussian in a feed-forward manner, whereas existing optimization-based compression methods require additional adaptation steps.

- Experimental results demonstrate that the framework achieves high compression performance with comparable rendering quality with existing compression methods.

- The authors validate that the proposed hyper-prior design, including inter-/intra-Gaussian context models, further improves the compression performance.
summary: This paper proposes a generalizable 3D Gaussian compression framework incorporated with an effective entropy minimization module inspired by 2D image compression. The main contribution of this work is compressing 3D Gaussians within a minute, using a feed-forward pipeline. To achieve this, the authors first divide Gaussian attributes into a) geometry attributes, and b) color attributes. Then, these attributes are encoded and decoded using hyper-priors from inter-/intra-Gaussian context models, along with context design from the image compression approach. Consequently, this paper achieves a compact size and comparable rendering quality compared to the existing compression method, while requiring only several seconds.
weaknesses: - Despite the effectiveness, the design of context models comes from heuristic priors that the Gaussians share inter-/intra-information. It could be helpful to show those relations of Gaussians to understand the justification of the proposed hyper-prior designs.
- There is no ablation on the strategy for dividing chunks in the inter-/intra-Gaussian context model. It could be helpful to show the results for several splitting designs.
- Due to arithmetic coding-based compression, it might yield notable computational costs to encode and decode the parameters.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 2
questions: It seems that the process of generating m by MLP_m is not illustrated in Figure 2. By reading the method section, it is somewhat difficult for readers to understand the role and position of m in the pipeline (although there is a formula provided). It is suggested to draw a more detailed pathway in Figure 2.
soundness: 3
strengths: As far as I know, this is the first generalizable optimization-free 3DGS compression pipeline. Without the need of  original  training images, this generalizable compression method enables the compression of 3DGS, expanding the application scope of 3DGS compression algorithms.  

This article provides a novel approach to context modeling for exploring the correlations between Gaussians.
summary: This paper introduces an optimization-free model, called FCGS, for 3D Gaussian Splatting (3DGS) compression, which allows compressing an optimized 3DGS representation in a feed-forward process. The model incorporates both intra- and inter-Gaussian context information. The authors project most Gaussian attributes into a latent space and utilize arithmetic encoding and decoding for data compression. Additionally, they employ an end-to-end progressive training scheme for the proposed pipeline.
weaknesses: I find the experimental content of this paper somewhat disorganized and difficult to follow. The authors present the main comparative experimental results using figures, but there are no numerical comparisons in the main paper. Although detailed data is provided in the Appendix, the absence of tables in the main paper causes some inconvenience for readers. 

Additionally, in Tables C-D, the authors report the running time, but I believe this comparison may not be entirely reasonable. The running time for methods that involve training from scratch includes the time for reconstruction. FCGS also requires a pre-trained 3DGS, and the time spent pre-training the 3DGS should be reflected in the table. 

In the comparative experiments, the authors only compare methods with worse rendering quality than FCGS. However, there are existing 3DGS compression methods that offer better rendering quality and higher compression rates than FCGS. While these methods are not generalizable, readers should still be informed of the current gap between FCGS and the state-of-the-art.

Meta-review: This paper presents an efficient 3D Gaussian Splatting (3DGS) compression method. The proposed approach is novel, offering an optimization-free pipeline for 3DGS compression in a feed-forward manner. A multi-path entropy module controls the tradeoff between size and quality, while inter- and intra-Gaussian context models further enhance compression performance. The method demonstrates strong compression rates across multiple datasets while significantly reducing execution time.

The main weakness is that the compression size and quality may not surpass optimization-based methods. However, the proposed method compresses 3DGS in a feed-forward manner and reduces per-scene compression time. It is the first generalizable optimization-free 3DGS compression method and offers potential applications that previous methods cannot address.

Decision: Accept (Poster)