ContextGS : Compact 3D Gaussian Splatting with Anchor Level Context Model
Authors: Yufei Wang
Zhihao Li
Lanqing Guo
Wenhan Yang
Alex Kot
Bihan Wen
Venue: NeurIPS 2024 poster (2024)
Keywords: 3D scene compression
3D Gaussian Splatting
Primary area: machine_vision

Abstract: Recently, 3D Gaussian Splatting (3DGS) has become a promising framework for novel view synthesis, offering fast rendering speeds and high fidelity. However, the large number of Gaussians and their associated attributes require effective compression techniques. 
Existing methods primarily compress neural Gaussians individually and independently, i.e., coding all the neural Gaussians at the same time, with little design for their interactions and spatial dependence. Inspired by the effectiveness of the context model in image compression, we propose the first autoregressive model at the anchor level for 3DGS compression in this work. We divide anchors into different levels and the anchors that are not coded yet can be predicted based on the already coded ones in all the coarser levels, leading to more accurate modeling and higher coding efficiency. To further improve the efficiency of entropy coding, e.g., to code the coarsest level with no already coded anchors, we propose to introduce a low-dimensional quantized feature as the hyperprior for each anchor, which can be effectively compressed. Our work pioneers the context model in the anchor level for 3DGS representation, yielding an impressive size reduction of over 100 times compared to vanilla 3DGS and 15 times compared to the most recent state-of-the-art work Scaffold-GS, while achieving comparable or even higher rendering quality.

ContextGS: Compact 3D Gaussian Splatting with Anchor Level Context Model
Yufei Wang'
Zhihao Li'
Lanqing Guo' Wenhan Yang² Alex C. Kot'
Bihan Wen
Nanyang Technological University, Singapore
2 PengCheng Laboratory, China
(yufei001, zhihao.li, lanqing.guo, eackot, bihan.wen] ntu.edu..g yangwh@pellac.cn
Homepage: http::///tthhub..o/////00///////
10x Compression!
Scaffold-GS
(a) Rendered image
(b) Anchor division
(PSNR 5008 Size:
ContextGS (Ours) (PSNR 25,00D Size: 21881BB
(c) Cosine similarity
(d) Bit saving
Figure 1: An illustration of the necessity of using autoregressive model in the anchor level. While Scaffold-GS (20) greatly reduces the spatial redundancy among adjacent 3D Gaussians by group- ing them and introducing a new data structure anchor to capture their common features, spatial redundancy still exists among anchors. Our method, ContextGS, first proposes to reduce the spatial redundancy among anchors using autoregressive model. We divide anchors into levels as shown in Fig. (b) and the anchors from coarser levels are used to predict anchors in finer levels, i.e., predicts then together predict Fig. (c) verifies the spatial redundancy by calculating the cosine similarity between anchors level O and their context anchors in levels and 2. Fig. (d) displays the bit savings using the proposed anchor-level context model evaluated
entropy coding based strong baseline built on Scaffold-GS [20). Compared with Scaffold-GS, we achieve better rendering qualities, faster rendering speed, and great size reduction of up 15 times averaged over all datasets used.
Abstract
Recently, 3D Gaussian Splatting (3DGS) has become a promising framework for novel view synthesis, offering fast rendering speeds and high fidelity. However, the large number Gaussians and their associated attributes require effective compression
compression techniques. Existing methods primarily compress 3D Gaussians
individually and independently, i.e., coding all the 3D Gaussians at the same time, with little design for their interactions and spatial dependence. Inspired by the effectiveness of the context model in image compression, we propose the first autoregressive model the anchor level for 3DGS compression in this work. We divide anchors into different levels and the anchors that are not coded yet can be predicted based the already coded ones in all the coarser levels, leading more
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

accurate modeling and higher coding efficiency. To further improve the efficiency of entropy coding, e.g., to code the coarsest level with no already coded anchors, we propose to introduce a low-dimensional quantized feature as the hyperprior for each anchor, which can be effectively compressed. Our work pioneers the context model in the anchor level for 3DGS representation, yielding an impressive size reduction of over 100 times compared to vanilla 3DGS and 15 times compared to the most recent state-of-the-art work Scaffold-GS, while achieving comparable even higher rendering quality.
1
Introduction
Over the past few years, novel view synthetic has rapidly progressed. As a representative work, Neural Radiance Field (NeRF) [22) uses a Multilayer Perceptron (MLP) to predict the attributes of quired points in the 3D scene. While good rendering qualities are achieved, the dense querying process results in slow rendering, which greatly hinders their applications in practical scenarios. Significant efforts have been made to enhance training and rendering speeds, achieving notable progress through various techniques, such as factorization 44] g [0 [3] and hash grids [24] [2]. However, they still face challenges in the real-time rendering of large-scale scenes due to the intrinsic limitations of volumetric sampling. Recently, 3D Gaussian Splatting (3DGS) [5] has achieved state-of-the-are art (SOTA) rendering quality and speed. As an emerging alternative strategy for representing 3D scenes, 3DGS represents 3D scene using set of 3D Gaussians initiated from Structure- foom--roootit (SfM) with learnable attributes such as color, shape, and opacity. The 2D images can be effectively rendered using differentiable rasterization and end-to-end training is enabled. Meanwhile, benefiting from efficient CUDA implementation, real-time rendering is achieved.
Despite its success, 3DGS still encounters limitations in storage efficiency. Representing large scenes requires millions of 3D Gaussian points, which demand several GBs of storage, e.g., an average of 1.6 GB for each scene in the BungeeNerf [32 dataset. The huge storage overhead greatly hinders the applications of 3DGS 15 thus an efficient compression technique is required. However, the unorganized and sparse nature of these 3D Gaussians makes it highly challenging to effectively reduce data redundancy. To address this issue, various techniques have been proposed to enhance the storage efficiency of 3D Gaussian models. For example, 77 18 25 266 proposed discretize continuous attributes of 3D Gaussians to cluster of attributes stored in the codebooks; 7 18] proposed to prune neural Gaussians with little effect. Entropy coding is also used reduce the storage overhead by
further encoding neural Gaussian features into bitstream [ [ 18 23 Although space utilization has greatly improved, they focus more on individually compressing each Gaussian point and do not well explore the relationship and reduce the spatial redundancy among neural Gaussians. To further reduce the spatial redundancy, most recently, [20] proposed divide anchors into voxels and introduced an anchor feature for each voxel to grasp the common attributes of neural Gaussians in the voxel, i.e., the neural Gaussians are predicted by the anchor features. While the spatial dependency has been significantly reduced, as shown in Fig. (c), the similarity among anchors remains high in certain areas, indicating that spatial redundancy still exists.
To further enhance the coding efficiency of 3D scenes, we propose a novel framework named ContexIGS for 3DGS compression. Inspired by the effectiveness of context models (3T) in image compression (21). we introduce an autoregressive model at the anchor level into 3DGS. Specifically, building on Scaffold-GS (20), we divide anchors into hierarchical levels and encode them progres- sively. Coarser level anchors are encoded first, and their decoded values are used to predict the distribution of nearby anchors finer levels. This approach leverages spatial dependencies among adjacent anchors, allowing already decoded anchors to better predict the distribution of subsequent anchors. leading to significant improvements in coding efficiency. Additionally, anchors decoded at coarser levels can be directly used in the final fine-grained level, reducing storage overhead. To further enhance coding efficiency, especially for encoding the coarsest level anchors without already decoded ones, we employ quantized hyperprior feature as an additional prior for each anchor. It is worth noting that the proposed method can also support vanilla 3DGS renderers by simply removing the view-dependent features from the anchor feature. Our contributions can be summarized as follows:
We propose the first context model for 3DGS at the anchor level. By predicting the properties of anchors that are not coded yet given already coded ones, we greatly eliminate the spatial redundancy among anchors.
2

---

We propose unified compressing framework with the factorized prior, enabling end-to-end entropy coding of anchor features. Besides, a strategy for anchor layering is proposed. which allows already decoded anchors to quickly locate adjacent anchors that are to be decoded. Meanwhile, the proposed method avoids redundant coding of anchors by the proposed anchor forward.
The experimental results on real- world datasets demonstrate the effectiveness of the proposed method compared with SOTA and concurrent works. On average across all datasets, model achieves a compression ratio of 15x x compared to the Scaffold-GS model we used the backbone and x00x compared to the standard 3DGS model, while maintaining comparable or even enhanced fidelity.
2 Related works
2.1 Neural radiance field and 3D Gaussian splatting
Early 3D scene modeling often employs the Neural Radiance Field (NeRF) 22 as a global ap- proximator for 3D scene appearance and geometry. These approaches [2] 29 30 use multi-layer perceptron (MLP) to implicitly represent the 3D scene by predicting attributes of queried points. However, the dense querying process results in extremely slow rendering. Various methods have been developed to speed up the rendering process significantly 6 27), such as plane factorization-bant based techniques like K-Planes [8] and TensoRF 4 and the use of hash grid features in InstantNGP (24). While these methods enable high-quality rendering with a much smaller MLP compared to the vanilla NeRF, rendering single pixel still requíres numerous queries. This can lead to increased storage requirements for the grid-based features and difficulties in efficiently rendering empty space large-scale scenes. To achieve real-time and efficient rendering while maintaining high fidelity, 3DGS [15] introduces an innovative approach by representing the scene explicitly with numerous learnable 3D Gaussians. By employing differentiable splatting and tile-based rasterization (17). 3DGS 15] optimizes these Gaussians during training in an end-to-end manner.
2.2 Deep compression
Despite the effectiveness of 3DGS 15] in rendering speed and fidelity, the large number of Gaussians and their associated attributes result significant storage overhead. Many techniques are proposed to reduce the storage requirements of 3DGS. For example, 18 proposes to prune neural Guassians (aka 3D Gaussians) with minimal impact. 7 [8] Gaussian parameters. Entropy coding is also used in 11 5 18
26 propose to utilize codebooks to cluster 23 to encode neural Gaussians into bit streams by modeling their distributions. While remarkable performances are achieved, they mainly focus on improving the efficiency of single neural Gaussian and neglect the spatial redundancy among neighbor neural Gaussians. Most recently, Scaffold-GS 220 proposes to introduce an anchor level to capture common features of nearby neural Gaussians in the same voxel, and successive work [5] demonstrates its effectiveness by further introducing hash-feature as a prior for entropy coding. However, 5 codes all the anchors the same time, and its spatial redundancy can be further reduced. In the image compression task, an important category of methods to improve the coding efficiency is the context model [21 BI], which greatly reduces the spatial redundancy by predicting the distribution of latent pixels based on an already coded one. Inspired by the context models used in compression, we propose encode the anchor features in an autoregressive way, i.e. predict the anchor points from already coded ones at coarser levels. As far as we know, we are the first to reduce the storage redundancy of 3DGS using a context model the anchor level.
25
3 Preliminary
3DGS [155] utilizes a collection of anisotropic 3D neural Gaussians to depict the scene so that the scene can be efficiently rendered using a tile-based rasterization technique. Beginning from a set of Aimuctre-from-MMotie (SIM) points, each Gaussian point is represented as follows
G(x) = e - (x- u) (x-#)
(I)
where x is the coordinates in the 3D scene, μ and Σ are the mean position and covariance matrix of the Gaussian point, respectively. To ensure the positive semi-definite of Σ, Σ is represented as
3

---

Anchor division (Frem fine courso)
pollict
M. M:Pypp (1-)
4f
O.. X
Ancher furward
Videe Canile (ppicty a (itrhhai
Andhor
Dooding
Aachor
Nearal Geoesion
wi bianan
Deduplication anchors
Vooc
Leersblc offictc
diffecan lavels
ga-n (ve" wetr (i-1)
1 (xP-1-92-)
(a) Anchor points with associated neural Goussians.
(b) The proposed multi-level unchor division.
Figure 2: (a): illustration of the data structure we used following Scaffold-GS 19], where anchor points are used to extract common features of their associated neural Gaussians. (b): The proposed multi-level division of anchor points. The decoded anchors from higher (coarser) levels are directly forwarded to the lower (finer) level to avoid duplicate storage. Besides, taking decompression as an example, the already decoded anchors are used to predict anchors that are not decompressed yet, which greatly reduces the spatial redundancy among adjacent anchors. (Best zoom in for details.)
E =SRSS'R RT, where and S are scaling and rotation matrixes, respectively. Besides, each neural Gaussian has the attributes of opacity E R' and view-dependent color E R modeled by spherical harmonic [34]. All the attributes, i.e., [is, R. S, o, c), in neural Gaussian points are learnable and optimized by the reconstruction loss of images rendered by the tile-based rasterization.
Scaffold-GS [20]. While representing scenes with neural Gaussians greatly accelerates the rendering speed, the large amount of 3D Gaussians leads to significant storage overhead. To reduce the redundancy among adjunct 3D Gaussians, the most recent work, Scaffold-GS [19], proposes to introduce anchor points to capture common attributes of local 3D Gaussians shown in Fig. 2 (a). Specifically, the anchor points are initialized from neural Gaussians by voxelizing the 3D scenes. Each anchor point has context feature f 32 location E R3, scaling factor I e R and & learnable offset O E R Given a camera at x., anchor points are used to predict the view-dependent neural Gaussians in their corresponding voxels as follows,
{c' r
where o. = ||x Xcl12. d.
x-x ||2" the superscript i represents the index of neural Gaussian in the voxel, c' R are the scaling and color respectively, and I' R4 is the quaternion for rotation. The positions of neural Gaussians are then calculated as
(1e
1== x +
0 1
where is the learnable positions of the anchor and I is the base scaling of its associated neural Gaussians. After decoding the properties of neural Gaussians from anchor points, other processes are the same the 3DGS [15]. By predicting the properties of neural Gaussians from the anchor features and saving the properties of anchor points only, Scaffold-GS [19 greatly eliminates the redundancy among 3D neural Gaussians and decreases the storage demand.
4 Methodology
The overall framework is shown in Fig. 3 We first introduce how to divide anchors into levels with traceable mapping relationships among adjacent levels in Sec 4.1 Based that, we present the entropy coding in an autoregressive way in 4.2 and the overall training objective in Sec 4.3
4.1 Anchor partitioning strategy
We attempt to divide N anchors V (v ((x,, O,) into K disjoint levels, i.e.,
V = t U VL U... U V-1 and V n V 2, vi + j. For each anchor set V it is expected to spawn over the whole scene, and relatively uniform downsampling of finer set Vi-1, Assume that we encode/decode the scene using the order from level K to level where level K is

---

Anchor ()
010--101
Feature
MLP
10
Context feature (from
Codoc
(x V.
Coding
Distribution cstimation
Aeseropy (f)
Senling
Posttio
Officet
Adaplive quanization
Hysperpeior
Enerey estimation (Triining)
Leartable unchor atributes
Quantized propertics
(b) Anchor Feature Coding
N
Mescrosy ( (Training)
by bosiN
O..
Hyperprice (au))) t
O..
MLP
010-=101
Rendeting
i
(Tessing)
Refirence
Renderigg resuk
Level dependent neural Caussians
Distanoc
Hypeeprior Codlng
(c) Neural Gaussian Splatting d a-blending
Figure 3: The overall framework of the proposed method includes three levels, Le., K = 3, to encode the anchors. The decoded anchors from a coarser level + 1 used to encode the anchors in level i. Besides, hyperprior features are used to predict the properties of anchors at all levels. For training, after finishing the coding of all levels, the anchor features after adaptive quantization are used to predict properties of neural Gaussians. The rendering loss is calculated and optimized together with the entropy coding loss Centropy- For testing, after we decode anchor features from the bit stream, the rendering is exactly the same with Scaffold-GS [20] without introducing overhead.
the coarsest level, we expect the mapping from V to Vi-1 is traccable and easy to obtain. other Of) from coarser level k, we expect to quickly locate where N the number of adjacent anchors,
words, given anchor v = (x
v adjacent anchor set { ( No- i.., the anchors in the same voxel of level i 1 with v². As such, after decoding anchors coarser level, we can scatter the already decoded features to to-be-processed anchors as the prior for better coding efficiency.
Ji=0 in level
To achieve the requirements above, we propose to utilize a simple yet effective way to divide anchors into levels using "bottom-up" method. As shown in Fig. (b), given set of anchors of the scene, we partition them into different fine-grained sets based on different voxel sizes E, as follows
4 =
<><=M(((V MM( x x})
: 1.2,....... V 1/--1}}.
where | is the counting operation, X is the anchor position after quantization using the voxel size of level k. and M PP( - v (where P is the power set) is mapping function that selects representative anchor to level k - from set of anchors {vē-1 x = =} that has the same position after quantization. The definition of x and M are follows
ch
x E V
ch
(5)
M(I x x))
vi- s.t. minfi i x x),
where 5o is initialized as the whole anchor set V. We select the anchor with the minimum index minfi: x = x} among set of anchors in level that are the same voxel. Besides, we filter out the repeated anchors among different levels follows
v V2 V V v V
(6)
where \ is the set difference operation. We keep the voxel size of the finest level (level 0) the same as the initial value e, and set the voxel size of level to e; R; Since different scenes have different initial voxel sizes and anchor point distributions, using fixed set of voxel scaling parameters { leads to suboptimal performance. To avoid finetuning pyram-easameters for
each scene, propose conduct me-timee parameter search after initializing anchors. Instead of
5

---

| &
directly setting the scale Кig, we target ratio between level i and + and expect
TV
Since V*I decreases monotonically with K;, can easily and efficiently determine the values of {si [ using a binary search. We empirically find that the performance of models among different senses is relatively robust to the selection of (refer to Fig. 6
4.2 Coding with entropy models
After dividing the anchors into multi-levels, in this section, we discuss how the already decoded anchors to predict ones that are not decompressed yet and how to encode attributes of anchors improve the coding efficiency.
Context modeling in anchor levels. To encode an anchor point v; = (x,, 4, O,) into bitstreams efficiently using entropy coding, we need to estimate its distributions accurately. The core idea of the proposed method is to predict the properties of anchors additionally conditioned on already decompressed anchors. Taking the modeling of anchor feature f from the anchor vž example, the details are as follows
Prz (xb)) = (N(H, u( - (E)))) Al
FE
where Fi, is MLP belonging to the level k. and 2pt is the prior of the anchor v as follows
[x x
-=K-1 -1 k<K-1
(8)
1 are the feature and where [-] is the concatenation operation among the channel dimension, r. .15
scaling from the adjacent anchor / in level + 1 that are already decoded as shown in Fig. (b).
Hyperprior feature for anchor. While introducing the position x contributes to predicting the distribution of anchor features, it still lacks enough freedom to eliminate spatial redundancy. There- fore, we introduce a learnable hyperprior vector z; E R250//h for each anchor vi where h, is hyper-parameter control the length of the hyperprior features. The hyperprior z; is modeled using the non-parametric, fully factorized density model I as follows:
30//ha-1 II
1 2' (-2
(-11) (E))
(2)e(2,
where 2; represents z, with quantization noise, is the channel index and θ is the network parameters for modeling the hyperprior. Since the hyperprior feature z, is quantized into integers å and jointly optimized reduce the size the bitstream, only occupies small portion of storage shown in Table 4
The final prior for coding features of the anchor v is & [2,; +b!1
4.3 Training objective
The training objective of the proposed method to jointly optimize the bitrate of coded anchor features and rendering loss measured by SSIM and L1 loss. The final training loss is
L == fold fold + XeCentropy + XmnL
(10)
where Loonf fold is the training loss of [19] Lm is the masking loss from 18] to regularize the masking loss of offsets of neural Gaussians O and Centropy is the overall entropy loss that measures the cost of storing anchor properties defined as follows,
K-1
1
Lentropy = E[- loga Pa (2) +
Σ 1=0
loga
II
pr(f.luB/)
(II)
where the first term measures the cost of coding the hyperprior feature while the second term the cost of coding features of anchor points in all the levels, and of is the context feature that includes both the hyperprior feature x and the feature from already coded nearby anchor in level + 1 as illustrated in Eq.

---

The quantitative results obtained from the proposed method ContexIGS and other com- petitors. Baseline methods, namely 3DGS [13] and Scaffold-GS [20], are included for reference. The intermediary approaches are specifically designed for 3DGS compression. Our methodology showcases two results representing varying size and fidelity tradeoffs, achieved through adjustment Highlighted in red and yellow cells are the best and second-best results, respectively. Size
Table 1:
of Xe.
measurements are expressed in megabytes (MB).
Datasets
Nip-NeR1360 8
Tank& Temples
DeepBlending
11
RangeNNeRF
32
methods
psur
ssimt
lpips]
size]
peart
ssim?
Ipips
stzel
poart
ssimt lpips).
sizcl
psur
soam?
lpips.,
stze_
3DGS [S]] [SIGGRAPH 23)
27.19
0.313
0.223
744.7
23.09
0.844
0.178
431.0
29,42
0.899
0.247
663.9
24.37
0.841
00:205
1616
Scaffald-GS 2 (CVPR
27.50
0.806
0.252
253.9
23.96
0.853
0.177
86.50
30.21
0.906
0.254
66.00
26.62
0.865
0.241
183.0
EAGLES
21.15
23.41
0.540
0.,00
02.00
LightGinssian
X
27.00
0.799
0.249
44.54
22.83
0.822
0.242
22.43
27.01
0.872
0.308
33.94
24.52
0.825
0.255
87.28
DDSS
[
(4YPR224
27.08
0.79%
0.247
48.80 80
23,32
0.83
0.201
39.43
29.79
0.901
0.258
43.21
23.36
0,788
0.251
82.60
ComppressedD (CERR224)
26.98
0.832
0.194
17.28
25.30
24.13
Morgenstem of ml 23
26.01
0.772
0.259
23:90
22.78
0.817
0:211
13:05
28.92
0.891
0.276
8.40
Navancet ml. 25
27.16 0.808
0.228
50, 30
23,47
0.840
0.188
27.97
29.75
0.903
0.244
42.77
24.63
0.823
0.239
104.3
27.53
0.807
21.04
0.816
0.187
1.35
18.19
Ours (Jom-rate)
27.62
0.50%
0.237
12.68
24.20
0.822
0.184
7.05
30.11
0.907
0.265
3.45
26.90
0.866
0.222
14.00
Ours (high-mte)
27.75
0811
0.231
18.41
24.29
0.855
0.176
11,80
30.39
0.909
0.258
6.60
27.15
0,875
0.205
21.80
30.25
HAC
Scaffold-68 65 yoc
27.0
HAC
Scaffeld-GS Navaneet al.
Scaffald-Gs
30G5
PSNR (dB)
30.00
EAGLES Compressed3d Ravanect
EAGLES #Corpect33DS
29.75
o CompacttDG
26.0
27.0
ComprossedRD
29.50
25.5
Compact3DGGs of Compressed3
26.5
29.25
Morgenstem
25.0
EAGLES
3DGS
Navaneet et al, Mongenster et al.
28.00
Morgenstem
10a
10
10a
10a
DeepBlending
Bumge=NeRF
Mip-NeRF360
(iz)
Figure 4: The Rate-Distortion (RD) curves for quantitative comparison between our method with most recent SOTA competitors. It is worth noting that the x-axis is in log scale for better visualization.
5
Experiments
5.1
Implementation details
We build our method based on Scaffold-GS (0). The number of levels is set to 3 for all experiments and the target ratio among two adjacent iterations is 0.2. he is set to 4, i.e., the dimension of the hyper-prior feature is a fourth of the anchor feature dimension. For a fair comparison, the dimension of the anchor feature is set 50 following (5) and we set the same Am. 5e 4. The setting of Ae is discussed in Appendix AA.3 since different values are used to evaluate different rate-distortion tradeoffs. For a fair comparison, we use the same training iterations with Scaffold-Gr (20 and HAC [5). i.e., 30000 iterations. Besides, we use the same hyperparameters for anchor growing as Scaffold-GS (20) so that the final model has similar or even smaller number of anchors, leading to faster rendering speed. More implementation details are in the supplementary materials.
5.2 Comparison with baselines
Baseline, metric, and benchmark. We compare our method with 3DGS 15]
Scaffold-GS 20 and
some representative 3DGS compression works, including Compact3DGS [8] Compressed3D 26
EAGLES (T]) LightGaussian (7]. Morgenstern al. (23), Navaneet er al. 25 and HAC (5) The bascline methods include existing mainstream techniques, e.g., pruning [Z codebooks 7 18
25 26), and entropy coding [ 18 23), and includes the most recent works. We utilize PSNR
SSIM, and LPIPS 35 to evaluate the rendering qualities of different methods and report the storage size measured in MB. We evaluate the performance of the models on several real-scene datasets, including BungeeNeRF [32], DeepBlending (14), Mip-NeRF360 [3 and Tanks& Temples [16]. To more comprehensively evaluate the performance of our method, following the previous prototype (5), we use all 9 scenes in Mip-NeRF360 3 The detailed results of each scene are reported in the Appendix AA.3] To further evaluate the performance models among a wide range of compression ratios, we use Rate- sate-Dsitorrtion (RD) curves as additional metric.
7

---

(A) Rederence image [The Thmples dhismes)
16t Compaet SEGGS
(s) ScafiakiGs
(di HAC
(m) Ouns
(PSNut: 23 Stane Mus)
prasii 22-47 auans
(Ussat 22.35 Staw 7.00us)
(1SNa 22.42 Sinee 6.4041))
Redervenn imasg Taapl&/Tompte dotaset)
[b) CompactsDDSS
(c) BoatfolCSS
(4) ILAC
(c) Ouzs
(PSMR: 46.4MB)
171MB) (PENR: Siee 17.8M) (PBNR: Siee
Figure 5: Visual comparisons between method and baselines including Scaffold-GS (00 HAC [5), and Compact3DGS [18] on Bungeenerf [32] and Tank& Temples [16]. We report the PSNR (dB) of the image and the size of the 3D scene. (Best zoom in for details.)
The ablation study of each component we pro- posed measured on BungeeNerf [32] dataset. "HP" and "CM" represent the hyperprior and anchor level context model respectively. Ours w/o HP w/o CM can be roughly regarded Scaffold-GS (20) model with entropy coding
Table 2:
26
24
22
FOTT
20
amsterdam P5K (db)
amsterdam Size (MB)
and masking loss (8].
15
Size (MB)
PSNR
SSIM
L.IPS
0.2
0.8
ScaffoldGSS 20
183.0
26.62
0.865
0.241
Figure 6: The ablation of different tar- get ratio among different scenes. The PSNR remains relatively stable while the size of the scenes keeps increasing when increasing the т, which demon- strates the effectiveness.
Ours w/o HP w/o CM
18.67
26.93
0.867
0.222
Ours w/o CM
15.03
26.91
0.866
0.223
Ours w/o HP Ours
15.41
26.92
0.867
0.221
14.00
26.90
0.866
0.222
Results. As shown in Table the proposed method achieves significant improvement compared to our backbone method Scaffold-GS (20) in terms of the size of the model, with size reduction of 15x in average. Besides, the proposed method also achieves higher storage efficiency compared to the most recent competitors for the 3DGS compression, e.g., HAC (5). Compressed3D [26), and Compact3DGS [8]. is worth noting that the proposed method also significantly improves rendering quality, compared with the backbone model use, i.e., Scaffold-GS [20). This further verifies the observation from previous works that appropriate constraints on neural Gaussians can contribute the rendering quality, e.g., entropy constraints ( and pruning 33 Visual comparisons between the proposed method and other competitors are shown in Fig. 5 As shown the figure, the proposed method achieves better rending quality with greatly reduced size compared with most recent 31) compression works and also the backbone model. Besides, a comparison of the RD curves among the proposed method and most recent competitors is shown Fig. where the proposed method achieves better performance in wide range of compression ratios.
5.3 Ablation studies and discussions
Evaluation of target ratio 7 among adjacent levels. To evaluate the performance of the proposed strategy that encodes anchors in progressive way, evaluate the performance of models trained under different ratios among adjacent levels. We disable the hyperprior feature to better explore the effect of different target ratios As shown in Fig. 6 the PSNR remains relatively stable and the size gets relatively converged at the low ratio area. We select = 0.2 for all experiments.
Ablation of each component. We verify the effectiveness of two main components in our methods, iee, anchor level context model and hyperprior features, and the results are shown in Table 2 We build all the models Scaffold-GS [20] and set the model with the entropy constraint and the

---

Table 3: The ablation study of our method w/ and w/ reusing anchors from coarser
levels, i.e., anchor forward technique, measured on BungeeNerf [32 dataset.
Size (MB)
PSNR
SSIM
I.PIPS
Ours w/o dividing into levels Ours w/o reusing anchors Ours
14.73
26.91
0.867
0.222
15.54
26.91
0.862
0.230
13.80
26.89
0.867
0.222
Table 4: The storage cost of each component and rending qualities of our method and baselines evaluated on the scene rome BungeeNeRF [32] dataset. "w/ APC" represents using anchor position
coding, ie., using the hyperprior features to code the anchor positions. (The encoding/decoding time measured on an RTX3090.)
Number Anchors
Storage Cost (MB)
Fidelity
Speed (s)
Hyper Position
Feature
Scaling
Offsct
Mask
MLPs
Total
PSNR
SSIM Encode
Decode
Scaffold-GS GS (20
61.9
NIA
7.08
75.16
14.18
70.88
0.047
26.25
0.872
N/A
N/A
Ours (w/ APC)
52.3
1,066
1,954
5,708
1.603
2.556
0.452
0.320
13.62
26.38
0.871
41.33
51.58
Ours
52.5
0.778
2.543
5.808
1.586
2.563
0.452
0,316
14.06
26.38
0.871
20.40
17.85
masking loss 18 as our baseline model, i.e., "Ours w/o HP w/o CM". It worth noting that our baseline model significantly improves the storage efficiency compared with Scaffold-GS (20 and even the latest SOTA methods. Both the proposed anchor-level context model and hyperprior feature for anchors significantly improve the compression rate compared with our strong baseline model, reducing the file size by 21% and 10.17%, respectively. Besides, using them together can further boost the performance with storage savings of 26.1 1% and 92.5% compared with the baseline we introduced above and Scaffold-GS 20 respectively.
Ablation of anchor forward. A main difference between the proposed method and existing works for Level-of-Detail (LOD) techniques (28) is that the proposed method can reuse the anchors of different levels, i.e., anchor forward in Fig. 2 (b). For example, the anchors from different levels in [28] are stored separately. In contrast, the anchors from coarser levels are used in the final level (level 0) in our method, i.e., in an autoregressive manner. To verify the effectiveness of the proposed method that reuses the anchors in coarser levels, we do an ablation study in Table As shown in the table, the model w/o reusing anchors of coarser levels to the finest level leads to serious redundancy. even slightly worse than the model w/o dividing anchors into different levels. This demonstrates the effectiveness of the proposed anchor forward technique for the anchor-level context model.
Discussion on compressing anchor positions. One can utilize the hyperprior feature z to predict the distribution of anchor positions and the anchor position can thereby be compressed using entropy coding. However, we find that the precision of the anchor position is essential to the performance of the model and an adaptive quantization strategy leads to serious performance degradation. While fixed quantization width is feasible and can retain the fidelity performance while effectively compressing the size of anchors, it leads to a greatly increased number of symbols that greatly decreases the coding speed. Since the anchor position only occupies a small portion of bitstreams as shown in Table we do not encode anchors into bitstreams in all the experiments.
Analysis of inference and decoding time. The rendering speed after decompression is the same as or even faster than Scaffold-GS (20) when the number of anchors is the same since we use the same data structure. However, as shown in Table 4 we can achieve higher rendering quality with fewer anchors due to the use of masking loss (18) therefore achieving faster rendering speed. For the decoding time, while the proposed method involves autoregressive coding, which usually very slow image compression tasks (21) due to its serial characteristics, it adds neglectable overhead to our method in both training and decompression compared to other entropy-coding based 3DGS compression methods, such as HAC [5]. This is because, unlike autoregressive coding in image compression that predicts pixels/latent features one by one, introducing a loop of at least thousands of operations, we perform autoregressive coding group by group, introducing only loop of 3 iterations. Additionally, there is no overlap of anchors among the coding of different levels, so the overall number of anchors to be processed remains the same without dividing into levels.

---

6 Conclusion
In this work, we introduce pioneer study into utilizzig the anchor-level context model in the compression of 3D Gaussian splatting models. We divide anchors into different levels and the anchors from coarser levels are first coded and then used to predict anchors that are not coded yet. Additionally, a hyperprior feature is used for each anchor that further reduces the channel-wised redundancy. Besides, we demonstrate that utilizing the proposed anchor forward technique, i.e., directly reusing the anchors from coarse levels to the final level, can achieve better performance than just using anchors of coarse levels as a prior. Extensive experiments demonstrate that the proposed methods achieve better performance than SOTA and concurrent works.
Acknowledgements
This research was done in ROSE Lab, NTU and was supported in part by the NTU-PKU Joint Research Institute, collaboration between the Nanyang Technological University (NTU) and Peking University (PKU) that was sponsored by a donation from the Ng Teng Fong Charitable Foundation, the National Research Foundation Singapore Competitive Research Program (award number CRP29- 2022-0003), and Guangdong Basic and Applied Basic Research Foundation (2024A1515010554
References
[1] J. Ballé, D. Minnen, S. Singh, S. J. Hwang, and N. Johnston. Variational image compression with a scale hyperprior. In International Conference on Learning Representations, 2018.
[2] J. T. Barron, B. Mildenhall, M. Tancik, P. Hedman, R. Martin-Brualln and P. P. Srinivasan. Mip-nerf:
A multiscale representation for anti-aliasing neural radiance fields. In Proceedings of the IEEE/CVF International Conference Computer Vision, pages 5855-5864, 2021.
[3] J. T. Barron, B. Mildenhall, D. Verbin, P. P. Srinivasan, and P. Hedman. Mip-nerf 360: Unbounded anti-aliased neural radiance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5470-5479, 2022.
[4] A. Chen, Z. Xu, A. Geiger. J. Yu, and H. Su. Tensorf: Tensorial radiance fields. In European Conference on Computer Vision, pages 333-350. Springer, 2022.
[5] Y. Chen, Q. Wu, J. Cai, M. Harandi, and W. Lin. Hac: Hash-grid assisted context for 3d gaussian splatting compression. arXiv preprint arXiv:2403. 14530, 2024.
[6] K. Deng, Liu, J.-Y. Zhu, and Ramanan. Depth-supervised nerf: Fewer views and faster training for free. Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 12882-12891, 2022.
[7] Z. Fan, K. Wang, K. Wen, 7. Zhu, Xu, and Wang. Lightgaussian: Unbounded 3d gaussian compression with 15x reduction and 200+ fps. arXiv preprint arX1v:2311.17255, 2023.
[8] S. Fridovich-Keil. G. Meanti, F. R. Warburg, B. Recht, and A. Kanazawa. K-planes: Explicit radiance fields in space, time, and appearance. In Proceedings of the IEEF/CV Conference Computer Vision and Pattern Recognition, pages 12479-12488, 2023.
[9] S. Fridovich- Keil,, A. Yu, M. Tancik, Q. Chen, B. Recht, and A. Kanazawa. Plenoxels: Radiance fields without neural networks. In Proceedings of the JEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 5501-5510, 2022.
[10] Q Gao, Q. Xu, H. Su, U. Neumann, and Z. Xu. Strivee: Sparse tri-vector radiance fields. In Proceedings of the JEEE/CVF International Conference on Computer Vision, pages 17569-17579, 2023.
[11] S. Girish, K. Gupta, and A. Shrivastava. Eagles: Efficient accelcrated 3d gaussians with lightweight
encodings. arXiv preprint arXiv:2312.0455 2023.
[12] S. Girish, A. Shrivastava, and K. Gupta. Shacira: Scalable hash-grid compression for implicit neural representations, In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 17513-17524, 2023.
[13] K. Han and W. Xiang. Multiscale tensor decomposition and rendering equation encoding for view synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4232 4241, 2023.
10

---

[14] P. Heddaan, J. Philip. T. Price, Frahm, G. Drettakis, and G. Brostow. Deep blending for free-viewpoint image-based rendering. ACM Transactions on Graphics (ToG), 37(6):1-15, 2018.
[15] B. Kerbl, G. Kopanas, T. Leimkühher, and G. Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trunsactions on Graphics, 42(4):1-14, 2023.
[16] A. Knapitsch, J. Park, Q--Y Zhou, and Koltun. Tanks and temples: Benchmarking large-scale scene reconstruction. ACM Transactions on Graphics (ToG), 36(4): 11, 2017.
[17] C. Lassner and M. Zollhofer. Pulsar: Efficient sphere-based neural rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1440 1449, 2021.
[18] J. C. Lee, D. Rho, X. Sun, H. Ko, and E. Park. Compact 3d gaussian representation for radiance field. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.
[19] T. Lu, M. Yu, L. Xu, Y. Xiangli, L. Wang, D. Lin, and B. Dai. Scaffold-gs: Structured 3d gaussians for view view-adaptive rendering. arXiv preprint arXiv:2312.00109 2023.
[20] T. Lu, M. Yu, L. Xu, Xiangli, L. Wang. Lin, B. Dai. Scaffold-gs gs: Structured 3d gaussians for view-adaptive rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.
[21] F. Mentzer, E. Agustsson, Tschannen, Timofte, and Van Gool. Conditional probability models for deep image compression. In Proceedings of the IEEE Conference on Computer Vision and Pateenn Recognition, pages 4399-4402, 2018.
[22] B. Mildenhall, P. Srinivasan, M. Tancik, T. Barron, R. Ramamoorthi, and R. Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1):99 106, 2021.
[23] W. Morgenstem, F. Barthel, A. Hilsmann, and P. Eisert. Compact 3d scene representation via self-organizing gaussian grids. arXiv preprint arXiv:2312.
2023.
[24] T. Müller, A. Evans, C. Schied, and A. Keller. Instant neural graphics primitives with a multiresolution hash encoding, ACM Transactions on Graphics (ToG), 41(4):1-15, 2022.
[25] K. Navaneet, K. P. Meibodi, A. Koohpayegani, and Pirsiavash. Compact3d: Compressing gaussian splat radiance field models with vector quantization. arXiv preprint arXiv:231 181159, 2023.
[26] S. Niedermayr, J. Stumpfegger, and R. Westermann. Compressed 3d gaussian splatting for accelerated novel view synthesis. Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, 2024.
[27] C. Reiser, S. Peng, Y. Liao, and A. Geiger. Kilonerf: Speeding up neural radiance fields with thousands of tiny mlps. In Proceedings of the IEEE/CVF international conference on computer vision, pages 14335-14345, 2021.
[28] K. Ren, L. Jiang, T. Lu, M. Yu, Xu, Z. Ni, and B. Dai. Octree-gs: Towards consistent real-time rendering with lod-structured 3d gaussians. arXiv preprint arXiv:2403. 17898, 2024.
[29] D. Rho. B. Lee. S. Nam. J. C. Lee, J. H. Ko, and E. Park. Masked wavelet representation for compact neural radiance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20680-20690, 2023.
[30] T. Takikawa, A. Evans. J. Tremblay, T. Müller, M. McGuire, A. Jacobson, and S. Fidler. Variable bitrate neural fields. In ACM SIGGRAPH 2022 Conference Proceedings, pages 1-9, 2022.
[31] A. Van den Oord, N. Kalchbrenner, L. Espeholt, O. Vinyals, A. Graves, et al. Conditional image generation with pixelenn decoders. Advances in neural information processing systems, 29, 2016.
[32] Y. Xiangli. L. Xu, X. Pan. N. Zhao, A. Rao, C. Theobalt, Dai, and D. Lin. Bungeenerf: Progressive neural radiance field for extreme multi-scale scene rendering. In European conference on computer vision, pages 106-122. 122. Springer, 2022.
[33] R. Yang, Z. Zhu, Jiang, Yc, X. Chen, Y. Zhang, Y. Chen, Zhao, and Zhao. Spectrally pruned gaussian fields with neural compensation. arXiv preprint arXiv:2405 00676, 2024.
[34] Q. Zhang, S.-H. Back, S. Rusinkiewicz, and F. Heide. Differentiable point-based radiance fields for efficient view synthesis. In SIGGRAPH Asia 2022 Conference Popers, pages 1-12, 2022.
[35] R. Zhang, P. Isola. A. A. Efros, E. Shechtman, and O. Wang. The unreasonable effectiveness of deep features as perceptual metrie. In CVPR, 2018

---

A Appendix supplemental material
A.1 Broader impacts
The social impacts of our work are mainly in three folds:
1. Accessibility: This work contributes to making advanced 3DGS L5 models more accessible to wider audience. The reduced size of the 3DGS representation means that it can be more easily stored, transmitted, and processed on various devices, potentially democratizing access to high-quality rendering capabilities.
2. Applications: Improved compression techniques for 3DGS could enhance various applica- tions across industries, including virtual reality, gaming, medical imaging, and architectural visualization. These advancements may lead to more immersive experiences, better medical diagnostics, and more efficient design workflows.
3. Open Access:
The commitment releasing the code fosters transparency and collaboration within the research community. Open access to the code allows other researchers and practitioners build upon this work, accelerating innovation in the field of view synthesis and compression.
We do not find serious negative impacts since our work is only for compression. There no concerns regarding the misuse of our models or generating fake data.
A.2 Limitations
A main and inevitable limitation of the proposed method is that the entropy coding process in- troduces extra computational costs to estimate the entropy of the anchor features during training encoding/decoding when saving/loading the 3D scene. For example, it requires extra time decode the data of 3D scenes from the bitstream, making it challenging to start rendering at once when clicking the file.
A.3 More experimental details and results
We report the detailed comparison of our method w/ and w/o using the coding of anchor position in T To evaluate the performance among different rate-distortion (RD) tradeoffs, utilize different A,. i.e., a larger A, leads a smaller model size. For a fair comparison, we use the same normalization of the hyper-parameter Xa (5] by additionally utilizing the dimension of anchor features as the divisor so that the same A can lead to the similar bias towards the RD tradeoffs. The detailed results of each scene with different >e are reported in Table 6 Table
8 Table 7 and Table 9
Table 5: Quantitative results the effect anchor positon coding on our method. For our approach,
we give two results of different size and fidelity tradeoffs by adjusting Xe. A smaller Xe results in larger size but improved fidelity, and vice versa.
Datasets methods
Mip-NeRF360 3
Temples [66
DeepBlending
14
3p BungeeNeRF
psart ssim? lpips! sizel psart ssim? lpips.. size.] psnrt ssim? lpips sizel
psart ssim? lpips.. size
w/ encoding anchors
Ours (low-rate)
27.61 0.809 0,236 11.32
24.16 0.851 0.185 6.91 30.11 0.907 0.269 3.31
26.89
0.867 0.222 13.80
Ours (high-rate)
27.86 0.813 0,230 21.07 24.29 0.855 0.178 11.47 30.42 0.910 0.261
6.40
27.15
0.876
0.202 25.23
w/o encuding anchors
Ours (low-rate)
27.62 0,808
0.237
12.68
24.20
0.852 0.184 7.05 30.11 0.907
0.265
3.45
26.90
0.866
0.222
14.00
Ours (high-rate)
27.75 0.811 0.231
18.41
24.29 0.855 0.176 11.80 30.39 0.909
0.258
6.60
27.15
0.875
0.205
21.80
12

---

Table 6: Per-scene results of our method on BungeeNerf 32
dataset.
Scene
Size
PSNR
SSIM
LPIPS
rome
13,9631
25.9751
0,8669
0,2143
quebec
11.5909
30.1941
0.9337
0.1669
pompidou
15.2258
25,4676
0.8469
0.2446
low-rate
hollywood
13.5183
24.5154
0.7772
0.3164
(A 0.004)
bilbao
13.1537
28.0842
0.8877
0,1932
amsterdam
16.5549
27.1694
0.8842
0.1956
Average
14.0011
26.9010
0.8661
0.2218
rome
21.7033
26.6297
0.8825
0,1929
quebec
18.1517
30,5271
0.9400
0.1510
pompidou
23.4748
25.6218
0.8546
0.2330
high-rate 0.001)
hollywood
20.7082
24.7170
0.7876
0.3002
bilbao
20.7686
27.9842
0.8928
0.1770
amsterdam
26,0079
27.3979
0.8949
0.1754
Average
21.8024
27.1463
0.8754
0.2049
Table 7: Per-scene results of our method DeepBlending
14
dataset.
Scene
Size
PSNR
SSIM
LPIPS
drjohnson
3.94
29.68
0.906
0.261
low-rate (A = 0.004)
playroom
2.96
30.53
0.907
0.269
Average
3.45
30.11
0.907
0.265
drjohnson
7.80
29.86
0.909
0.249
high-rate (A 0.0005)
playroom
5.41
30.93
0.910
0.268
Average
6.60
30.39
0.909
0.258
Table 8: Per-scene results our method on Tank& Template [6 dataset.
Scene
Size
PSNR
SSIM
LPIPS
train
6.39
22.40
0.818
0.217
low-rate
truck
7.72
26.00
0.885
0.150
(A = 0.004)
Average
7.05
24.20
0.852
0.184
train
10.55
22.53
0.823
0.208
high-rate
truck
13.06
26.05
0.888
0.143
(A = 0.0005)
Average
11.80
24.29
0.855
0.176
13

---

Table Per-scene results of our method on the Mip-NeRF360 dataset.
Scene
Size
PSNR
SSIM
LPIPS
bicycle
21.82
25.08
0.738
0.271
bonsai 7.17
32.67
0.946
0.186
counter
6.30
29.38
0.912
0.197
flowers
16.71
21.31
0.576
0.377
low-rate
garden
18.78
27.32
0.845
0.148
(A = 0.004)
kitchen
7.00
31.28
0.925
0.131
room
4.50
31.71
0,923
0.207
stump
14.86
26.58
0.762
0.268
treehill
17.00
23.29
0.647
0.349
Average
12.68
27.62
0.808
0.237
bicycle 38.09
24.97
0.740
0.263
bonsai
12.43
32.93
0.951
0.182
counter
10.67
29.60
0.916
0.190
flowers
28.27
21.18
0.571
0.378
high-rate
garden
31.26
27.39
0.851
0.136
(A 0.0005)
kitchen
12.44
31.69
0.931
0.123
room
8.09
32.03
0.928
0.196
stump
24.22
26.56
0.761
0.263
treehill
28.78
23.09
0.645
0.346
Average
21.58
27.72
0.811
0.231
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
We clearly summarized the contributions in both abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations in the Appendix A.2
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
15

---

Justification:
We do not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We included all necessary information to reproduce the main results. Besides, will release the code upon acceptance.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
16

---

Answer: [Yes]
Justification: We will release the code and pretrained models on acceptance. The data we used are all public data.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cr cc/
public/guides ottp:////eooooeeebeeassssooooon.
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to reproduce the results. See the NeurlPS code and data submission guidelines (https wetgguddddeeeuuubbueeee ssinoolicyy for more details.
//nips.co
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
The code will be released on acceptance. We included the necessary details for training and testing.
Justification:
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: We evaluate our methods over different datasets and rate-distortion tradeoffs, resulting in a large set of experiments. The proposed method outperform other SOTA methods by large margin, which far beyond the standard deviation of the experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
17

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma enror bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: All our experiments are done using a server with RTX3090s. The requirements of compute resources small.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: Our work does not have special ethics concerns. All data and experiments comply with ethical guidelines.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss broader impacts in Appendix A.I
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
18

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our method does not have a high risk of misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We cite the original papers and state which version of the dataset we used.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in package should be provided. For popular datasets, paperswithcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
19

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do not introduce new data.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/moodl as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
20


Reviews:
[{"review_id": "T6beAtRBAu", "reviewer": "Reviewer_fG6F", "summary": "This paper proposes ContextGS, a compact 3D Gaussian Splatting (3DGS) framework that requires only a minimal amount of storage size while demonstrating high rendering quality. \nUpon the neural Gaussian-based 3DGS framework Scaffold-GS, the authors construct a multi-level anchor structure to reduce the spatial redundancy and adopt context modeling proposed in image compression tasks.\nConsequently, it achieves a 15$\\times$ compression ratio compared to Scaffold-GS and a 100$\\times$ compression ratio compared to 3DGS.\nDespite the minimal storage size, it outperforms the existing compact 3DGS approaches in rendering quality.", "questions": "- What does the ‘mask’ of Scaffold-GS in Table 4 mean? To my knowledge, there is no masking parameter in Scaffold-GS. I am confused that the masking strategy of Lee et al. is applied to Scaffold-GS.\n\n- In L271-L272, the authors have argued that this method has fewer anchors due to the use of masking loss. Please clarify this part.\n\n- As they have mentioned in L392-394, there exists additional computational costs for entropy minimization during training. I wonder about the exact training time for this method compared to Scaffold-GS and HAC.\n\n- Also, the exact quantitative evaluation of rendering speed is needed to support the faster rendering time of ContextGS compared to Scaffold-GS as described in L271-L272. Please provide a comparison of rendering speed to prove the argument.\n\n- The multi-level partitioning strategy is similar to the multi-resolution structure of Scaffold-GS. Please clarify the difference between the multi-level strategy of this paper and Scaffold-GS.\n\n- Does the encoding anchor in Table 5 denote that APC in Table 4?", "limitations": "The additional computational costs cannot be alleviated for achieving a small storage size. Also, the format of this representation does not fit to standard 3DGS, thus the existing applications such as real-time interactive renderers cannot be used. Therefore, it has disadvantages to be used as a practical 3DGS representation.", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The quantitative evaluation shows that this method outperforms in rendering quality compared to existing compact 3DGS frameworks, including Scaffold-GS and HAC while requiring minimal storage usage.\n\n- The proposed entropy modeling of neural Gaussian features further reduces the storage size with minor degradation of rendering quality.", "weaknesses": "Although it achieves high compression performance, there are several critical points to concern.\n\n- There are limited technical contributions to their method. The concept of multi-level (or multi-resolution) anchor structure has already been proposed in Scaffold-GS and context modeling of neural Gaussians has been introduced in HAC. There are only minor technical contributions compared to the previous work.\n\n- It requires noticeable encoding and decoding time resulting in a bottleneck in the practical application of 3DGS. Especially, the encoding/decoding time of 'w/ APC' requires more than 40 sec for a single scene. Also, Scaffold-GS needs a per-view decoding process for view-adaptive rendering, resulting in slower rendering speed than 3DGS.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your valuable comments!\n\n&nbsp;\n## Novelty\n\nWe want to highlight that our method has **significant and essential differences** from both HAC and ScaffoldGS. The **context model**, in the generally accepted sense, **does not require additional storage**. We use already coded anchors (which are part of **existing anchors instead of newly created ones**) to model uncoded ones in our work. **HAC did not use the context model** in this definition since they need additional storage for hash features; actually, the termed \"context\" in HAC is a kind of **hyperprior** model [1].\n\nThe differences in \"multi-level\" between different papers are summarized as follows:\n\n| | Scaffold (Not designed for entropy coding) |HAC|The Context Model of Ours |\n|---|---|---|---|\n|Multi-level/Extra level|Introduces and stores **new data type** Anchor compared with 3DGS|Introduces and stores **new data type** grid hash feature compared with ScaffoldGS|In the anchor level of ScaffoldGS **itself**|\n|**Additional storage cost** of the extra level |Yes|Yes|**No**|\n\nTaking decompression as an example, HAC decompresses the hash grid feature first and then uses it to help decode the anchor features, i.e., all anchors are decoded **at the same time**. (As mentioned above, this acts as the role of **hyperprior** [1].) In our work, we decode some anchors first and use the already decoded anchors to decode the undecoded ones **like in an autoregressive manner**. (This is the commonly termed **context model** and no additional bitstream cost is needed for the \"extra levels\".)\n\n&nbsp;\n### Encoding and Decoding Time\n\nThe **encoding and decoding time are required by all entropy-coding-based 3DGS compression methods**, e.g., HAC (ECCV'24 Oral) and Compact3DGS(CVPR'24 Oral).\n\nFor example, compared with Compact3DGS, our decoding speed is much faster as shown in the following table. Besides, the decoding time (17.85s) is worth it compared to the reduced model size (187MB->14MB).\n\n| Measured on Rome (Bungernerf) | PSNR (dB) | Decoding Time (s) | Size (MB)|\n| --- | --- | --- | --- | \n| HAC (ECCV'24) | 25.68 | 22.77 | 19.3 | \n| Compact3DGS (CVPR'24 Oral) | 25.17 | 613.5 | 51.3 |\n| Ours | **26.38** | **17.85** | **14.1** | \n\n&nbsp;\n### \"Especially, the encoding/decoding time of 'w/ APC requires more than 40 sec for a single scene\"\n\n**This might be a misunderstanding**. As indicated in L267-268, the results in \"w/ APC\" only aim to explain why we do not encode the position, even though it leads to some improvements in rate-distortion performance. Thanks for your comments, but actually **we did not use \"APC\" in all the experiments** as indicated in L267-268.\n\n&nbsp;\n### ScaffoldGS vs. 3DGS\n\nWhile Scaffold-GS needs a per-view decoding process for view-adaptive rendering, it can achieve **comparable** (or even faster speed in some cases) by limiting the prediction of neural Gaussians to anchors within the view frustum. Scaffold-GS also has **better rendering quality** than vanilla 3D-GS. It is worth noting that we do not involve encoding or decoding for view-adaptive rendering. \n\n&nbsp;\n### Mask in Scaffold-GS\n\nThanks for pointing out the typo. The \"mask\" of Scaffold-GS originally aims to represent the property \"opacity (float32, dim=1)\" in the saved checkpoint. Since it is not used in the rendering, we will revise it to \"N/A\" in the revised paper. Other \"mask\"s in Table 4 refer to the encoded 3D Gaussian-level mask using Lee et al.'s pruning strategy.\n\n&nbsp;\n### Fewer Anchors and Faster Speed\n\nLee et al. [14] demonstrate that using their proposed masking loss can significantly reduce the number of Gaussians and increase the rendering speed. In our work, any anchor for which all its 3D Gaussians are masked will also be removed. As a result, as shown in the \"Number of anchors\" in Table 4, compared with ScaffoldGS, which utilizes 61.9K anchors, the proposed method uses only 52.5K anchors, approximately 15% less, and achieves better PSNR.\n\nOur rendering is exactly the same as ScaffoldGS so fewer anchors lead to faster rendering speed when other hyperparameters are the same. We re-trained a model with similar rendering quality to ScaffoldGS, and the results are as follows. **We achieve slightly faster rendering speed due to a smaller number of anchors and Gaussian points**. \n\n| Rome (Bungernerf) | ScaffoldGS | Ours |\n| --- | --- |--- |\n| FPS | 202.8 | 205.4 |\n| PSNR | 26.25 | 26.24 | \n\n&nbsp;\n### Training Speed\n\nWhile entropy coding is relatively slow in the inference due to its serial properties, estimating the entropy loss in the **training phase** is **parallel and fast**. \n\n*As mentioned in our limitations section*, estimating the entropy during training will indeed slightly increase the training time. Nevertheless, the additional minor training cost is worthwhile for the reduced size. For example, many existing concurrent works also explicitly estimate the entropy during training, e.g.,\n- End-to-End Rate-Distortion Optimized 3D Gaussian Representation, ECCV'24 \n(The training time is not reported, and the code is not released yet)\n- Hash-grid Assisted Context for 3D Gaussian Splatting, ECCV'24\n\nCompared with other models for compression, we have a similar training speed.\n\n| | ScaffoldGS | HAC (ECCV'24) | Compact3DGS (CVPR'24) | Ours |\n| --- | --- | --- | --- | --- |\n| Training Time (mins)| ~25 | ~40 | ~60  | ~60 |\n| Size (MB) | 186.7 | 19.30 | 51.27 | 14.06 | \n| PSNR (dB)| 26.25 | 25.68 | 24.80 | 26.38 | \n\n&nbsp;\n### \"Does not fit standard 3DGS\"\nThe core idea of our context model is not limited to ScaffoldGS, as it does not involve new data formats and does not make any assumptions about the basic elements of the 3D scene. Applying to vanilla 3DGS may be left for future exploration.\n\n&nbsp;\n### Does the encoding anchor in Table 5 denote that APC in Table 4?\n\nYes, the encoding anchor is the “w/ APC” (w/ **A**nchor **P**oint **C**oding) in Table 4. \"w/ APC\" in Table 5 is only for reference, and **we do not use it in the main paper**.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8VL327zVPl", "reviewer": "Reviewer_otkA", "summary": "In this paper, the authors propose ContextGS to reduce spatial redundancy among anchors using an autoregressive model. Specifically, the authors divide anchors into three levels, performing entropy coding from the top (coarse) level to the bottom (fine) level. Anchors from coarser levels are utilized as context to assist in the entropy coding of anchors at finer levels. Experimental results show that the proposed method achieves a size reduction of 15 times compared to Scaffold-GS and 100 times compared to 3DGS.", "questions": "1.\tIt’s better for the authors to make furhter clarification regarding how anchors are divided into different levels. Are anchors that have the same position after quantization selected for a higher level (according to lines 154-155)? But Figure 2(b) contradicts this, as v_3^{k-1}, which is far from the quantization center, is selected as the anchor for upper level.\n2.\tI wonder about the effects of the learnable hyperprior feature z_i and would like to see the performance without it.", "limitations": "This paper seems to lack discussions of the limitations or the potential negative societal impact.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 1, "contribution": 2, "strengths": "1.\tThe authors have explored the correlation between anchors and, for the first time, introduce autoregressive entropy models for spatial prediction of anchors.\n2.\tThe entire pipeline is trained end-to-end with joint rate-distortion optimization, supporting multiple bitrates by adjusting λ.", "weaknesses": "1.\tThe size reduction brought by entropy model is limited. The authors claim that the proposed method can achieve a size reduction of 15 times compared to Scaffold-GS. However, Table 2 in the ablation study indicates that the size reduction primarily stems from entropy coding and masking loss (from 183.0 MB to 18.67 MB), while the main contribution of the paper, the hyperprior and anchor level context model, results in approximately a 25% bitrate reduction (from 18.67 MB to 14.00 MB), which is relatively modest.\n2.\tThe description of the anchor division method is somewhat confusing, and the equations are difficult to understand.\n3.\tThe authors design a learnable hyperprior vector for each anchor as an additional prior. However, this approach may introduce additional spatial redundancies between anchors.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We deeply appreciate your thorough review and valuable feedback on our submission. Here are our detailed responses to your comments and suggestions:\n\n&nbsp;\n&nbsp;\n### Performance Improvement\n\n**As illustrated in the summary of the rebuttal**, we argue that the proposed main components indeed bring significant improvements.\n\nAny method of compression with SOTA performance relies on entropy coding, and most of them contribute to the improvement of entropy coding as well. Different papers use various entropy-based backbones as the baseline. Our entropy baseline (i.e., Ours w/o HP w/o CM) is well-designed with very strong performance. The baseline method we used in the ablation study even outperforms the **concurrent** work HAC (ECCV'24).\n\n| Tested on BungeeNeRF | PSNR | SSIM | LPIPS | SIZE |\n| --- | --- | ---- | --- | --- |\n| HAC | 26.48 | 0.845 | 0.250 | 18.49 |\n| **Baseline** We Proposed for Ablation | 26.93 | 0.867 | 0.222 | 18.67 |\n| ContextGS (Ours) | 26.90 | 0.866 | 0.222 | 14.00 |\n\nA 25% improvement on such a strong baseline is already significant. If we use a plain baseline entropy-based method like the one used in HAC (removing the additionally used anchor position as the prior), the size is ~30 MB, with around a 50% performance improvement. Besides, as mentioned previously, there is only ~11% accumulated improvement over 3 years for image compression.\n\n\n&nbsp;\n&nbsp;\n### Additional Cost from Hyperprior\n\nWhile introducing the hyperprior requires additional bitstream and storage costs, it is also compressed and optimized in an end-to-end manner. The allocated bit number to them is jointly optimized. As shown in Table 2, **using the hyperprior model further contributes to size reduction**. Besides, **almost all** image compression works utilize a hyperprior model to improve their performance since its proposition [a] . \n\n[a] Variational image compression with a scale hyperprior, ICLR 2018\n\n&nbsp;\n&nbsp;\n### Anchor Division\n\nYes, the anchors that have the same position after quantization are selected for a higher level. The reason that \"v_3^{k-1} in Fig. 2(b), which **may be** far from the quantization center, is selected as the anchor for the upper level\" is as follows:\n\n- The **quantization center** of the voxel **does not** necessarily **have a corresponding anchor**.\n- If we create an anchor for it, it goes against the core idea of our work, which is using the context model to improve coding efficiency (since the context model does not involve new storage requirements).\n- Currently, we select the anchor in the voxel that has the minimum index as elaborated in Eq. 5. This is due to the concern of high efficient implementation and this strategy already demonstrates significant improvements.\n\nBesides, we modified Fig. 2(b) to make it clearer (by swapping the positions of $v_0^{k-1}$ and $v_1^{k-1}$ and adding text description below \"Anchor forward\") as in the attached PDF. The reason for these modifications is to highlight that the index of anchors in the same voxel is unsorted since all the anchors are stored discretely and unordered, e.g., $\\{v_0^{k-1}, v_1^{k-1}, v_2^{k-1}\\}$ in the figure. We select the anchor with the minimal index, i.e., $v_0^{k-1}$, which may be either on the border or center of the voxel.\n\nHope the modified figure is clearer.\n\n&nbsp;\n&nbsp;\n### The Effects of the Learnable Hyperprior\n\nWe included the ablation study of the learnable hyperprior in Table 2. Removing the learnable hyperprior increases the size from 14.00 MB (Ours) to 15.41 MB (Ours w/o HP), demonstrating its effectiveness and complementarity with the context model. The baseline model (Ours w/o CM w/o HP) plus learnable hyperprior (Ours w/o CM) leads to a size reduction from 18.67 MB to 15.03 MB (~20% reduction).\n\nIt is worth noting that, different from HAC, the baseline model in our paper (Ours w/o CM w/o HP) already uses the anchor position as a kind of hyperprior. This is one of the reasons why the proposed baseline method (Ours w/o CM w/o HP) can already achieve comparable or even slightly better performance than HAC.\n\n&nbsp;\n&nbsp;\n### Limitations\nWe discussed limitations in L391-396 in the submitted paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eUl4azYx13", "reviewer": "Reviewer_gvyt", "summary": "The paper aims to compress Gaussian Splatting-based neural rendering representations.\nTo achieve higher representation performance with a smaller size, the paper proposes hierarchical anchors, where coarser level anchors work as context to achieve a higher compression rate.\nAdditionally, the paper introduces hyperprior coding for improved performance.", "questions": "Line 167: Is “senses” a typo of “scenes”?\n\nLines 201-202: It is mentioned that the number of levels was set to three. Did you run experiments with different numbers of levels, and how did they affect the performance?\n\nTab. 5 (appendix): Does “coding” refer to the hyperprior coding used in the main paper? If so, could you explain why “w/o encoding anchors” results in similar or smaller sizes compared to “w/ encoding anchors”?", "limitations": "The paper provides limitations of the proposed method.", "rating": 7, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "**Novelty**\nThe main idea is novel within the 3DGS framework.\n\n**Performance**\nThe performance improvement is significant.\n\n**Experiments**\nThe ablation study has been conducted thoroughly, making it easy to understand the contribution of each part in improving the rate-distortion curve.", "weaknesses": "**Mathematical Notations (minor)**\nThe meaning of ^ is unclear. In Eq. 5, $\\hat{x}$ denotes that $x$ has been quantized.\nHowever, in Eq. 6 and line 156, $\\hat{V}$ does not seem to refer to a quantized set of anchors.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for your review and valuable suggestions on our paper.\n\n&nbsp;&nbsp;\n### Typos\nThank you for pointing out the typo; we have corrected it.\n\n&nbsp;&nbsp;\n### Ablation of Different Number of Levels\nThank you for your suggestions. We conducted an ablation study on the Rome-Bungeenerf dataset without using the learnable hyperprior. The results are shown in the following table:\n\n| Level num | PSNR  | SSIM  | LPIPS  | Size  |\n|-----------|-------|-------|--------|-------|\n| 1         | 26.38 | 0.8731 | 0.2079 | 18.26 |\n| 2         | 26.27 | 0.8706| 0.2130 | 15.99 |\n| 3 (default) | 26.43 | 0.8730| 0.2107 | 15.12 |\n| 4         | 26.32 | 0.8712| 0.2113 | 15.27 |\n\nWe find that decreasing the level number to 2 leads to an obvious degradation in performance. However, increasing the level number slightly increases the storage cost. We observe that increasing the level number does not significantly enhance the compression rate of anchors while it increases the size of MLPs by approximately 0.1MB for the new level.\n\n&nbsp;&nbsp;\n### Notations\nThank you for pointing out the duplicate use of $\\hat{\\mathbf{V}}$. We will use $\\tilde{\\mathbf{V}}$ instead to avoid confusion.\n\n&nbsp;&nbsp;\n### Without Encoding Anchors in Table 5 (Appendix)\n**\"Coding\" in Table 5 refers to using entropy coding techniques to encode the anchor positions**, i.e., the detailed results of (w/ APC) in Table 4. We do not encode the anchor positions in the main paper because, as shown in Table 4, it significantly slows down the coding speed.\n\nSpecifically, we find that the anchor position is very important and difficult to compress. Through training with adaptive quantization width, we find the anchor position requires high numerical precision for storage. This results in a limited compression ratio and a slow coding speed due to the large number of symbols required for Arithmetic (entropy) Coding. \n\nSince anchor positions only occupy approximately 15% of the storage space and are crucial for rendering, compressing them sometimes does not contribute to significant improvements in performance.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "e2eWWkqLrN", "reviewer": "Reviewer_8ybR", "summary": "This paper presents Context-GS, a method designed to reduce the memory overhead of 3D Gaussian Splatting (3DGS). Inspired by context modeling in image compression, the authors introduce a similar concept into Scaffold-GS, which uses anchors to predict 3D Gaussian distributions. The method encodes anchors at a coarse-to-fine level, significantly enhancing storage efficiency. Experimental results on real-world datasets demonstrate that the proposed method achieves a high compression ratio while maintaining comparable fidelity.", "questions": "The teaser figure is hard to understand and does not effectively convey the main idea. Upon reading the caption, my initial impression was that (b), (c), and (d) are from the proposed Context-GS. However, the caption stating \"(c) verifies the spatial redundancy\" and the main text at L48 stating \"spatial dependency has been significantly reduced\" confused me about what exactly is being reduced. Is one of the figures taken from Scaffold-GS? It would be more effective to first provide a reference scene so readers can understand what is actually being reduced. Additionally, the points are too small and chaotic, making it difficult to convey the ideas effectively.\n\nMinors:\n- L21: \"quired\" should be \"queried.\"\n- L29: \"Neural Gaussian\"? It is unclear why this is referred to as \"Neural Gaussian\" instead of \"3D Gaussian.\" 3D Gaussian typically does not involve neural networks. The paper might define \"neural\" as meaning differentiable, but this could confuse some readers into thinking it is related to neural networks.", "limitations": "The paper discusses limitations in encoding and decoding costs, but the primary limitation appears to be highlighted in the weaknesses section. The proposed method also seems difficult to apply to more generic Gaussian splatting techniques. \n\nOne of the authors' main motivations is that \"these papers mainly focus on improving the efficiency of a single neural Gaussian and neglect the spatial redundancy among neighboring neural Gaussians.\" However, this motivation does not resonate with me, as the ablation study does not indicate significant improvements from considering spatial redundancy.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.The method is novel, integrating the concept of context modeling from the image compression domain.\n\n2.The results are strong. It achieves significantly better compression while retaining high rendering quality, comparable to the original Scaffold-GS and clearly superior to other counterparts. The evaluation is comprehensive, and the paper conducts thorough ablation studies that fairly analyze the actual compactness.\n\n3.The overall presentation is easy to follow.", "weaknesses": "1. The main concern is that the proposed main components have a very minor effect on performance. As shown in Table 2, the primary contribution to compression is adopted from Compact-3DGS, while the proposed \"HP\" and \"CM\" components reduce the memory by only up to 4 MB.\n\n2. The proposed method appears complicated and specifically customized to Scaffold-GS, which limits its extendability. One of the most appealing features of 3DGS is its compatibility with many graphics engines. However, the use of neural networks and view-dependent Gaussian prediction undermines this advantage.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for your thorough review and valuable suggestions\n\n&nbsp;&nbsp;\n### \"The primary contribution to compression is adopted from Compact-3DGS\"\n\n**Our method is significantly different from Compact-3DGS.** We only adopt the masking loss from Compact-3DGS. We also have significantly better performance than Compact-3DGS, i.e., **3.5dB improvements using only 16.9\\% bitstream measured on Bungeenerf**.\n\nWhile Compact-3DGS used entropy coding, they only used entropy coding as a **post-processing** step. Specifically, they use Huffman coding to encode the index after vector quantization (VQ) when completing training. As a comparison, we **optimize** the estimated entropy of the 3D scenes by learning the distribution of anchors **during training**. \n\nTo demonstrate the difference between Compact-3DGS and the effectiveness of the proposed method, we set the weight of the masking loss used in Compact-3DGS in Eq. 10 to $0$, and the results are as follows:\n\n| Measured on Rome (BungeeNerf) | PSNR | SSIM | LPIPS | Size |\n| --- | --- | --- | --- | --- |\n| $\\lambda_m$ = 5e-4 (default) | 26.38 | 0.871 | 0.214 | 14.06 |\n| $\\lambda_m$ = 0 | 26.42 | 0.872 | 0.212 | 13.97 |\n\nSince the entropy loss is trained in an end-to-end manner, we can achieve a similar or even better rate-distortion trade-off compared with using the masking loss proposed in Compact-3DGS in our framework. This strongly highlights the difference between our end-to-end entropy framework and the Huffman coding as a post-process in Compact-3DGS. \n\n&nbsp;&nbsp;\n### 4MB Improvement\nThe improvement from 18.67MB to 14MB is indeed a significant improvement. As illustrated **in the highlight of the summary of the rebuttal**, the relative improvements depend on the selection of the baseline. It is worth noting that while HAC and the proposed method both use Scaffold-GS as the backbone model, the baseline model we used for the ablation study is much stronger than that used by HAC, as shown in the following table. Our well-designed baseline model can achieve **much better** rendering quality with **almost the same size** as HAC. On top of the SOTA method/such a strong baseline, a 25% performance improvement is already very significant (as mentioned previously, there is only ~11% accumulated improvement over 3 years for image compression).\n\n|Tested on BungeeNeRF | PSNR | SSIM | LPIPS | SIZE |\n|---|---|---|---|---|\n|HAC|26.48|0.845|0.250|18.49|\n|**Baseline** We Proposed for Ablation|26.93|0.867|0.222|18.67|\n|ContextGS (Ours)|26.90|0.866|0.222|14.00|\n\n&nbsp;&nbsp;\n### \"Specifically customized to Scaffold-GS\"\nThe core idea of the proposed context model is **not limited to ScaffoldGS**, as it **does not involve new data formats** and **does not make any assumptions about the basic elements** representing the 3D scene, such as anchors or 3D Gaussians. The context model aims to explore the relationship among existing elements by predicting the probability distribution of the current context based on given decoded contexts. In principle, **it could also be applied to vanilla 3DGS or new backbones in the future**, which may be left for future exploration.\n\n&nbsp;&nbsp;\n### Teaser figure\n\nYes, **figures (b, c, d) in the teaser figure are all from the proposed ContextGS.** \n\nOur method **does not aim to reduce the similarity** between different levels, i.e., whether using the proposed context model does not significantly affect the similarity among different levels of anchors.\n\nIn fact, we **need to use the high similarity to better model the relationship among two levels**, i.e., predicting the probability distribution of the current context based on given decoded contexts. Usually, the higher similarity between the current context and decoded contexts leads to more significant improvements from the context model as follows. \n\n- Higher similarity $\\rightarrow$ More accurate distribution modeling $\\rightarrow$ Reduced entropy (bitstream) \n\nFig. (c) aims to visualize the similarity to **illustrate the feasibility of implementing the context model** in 3DGS. It shows that many anchors in level 0 have highly similar or almost the same features as their corresponding anchors in a coarser level. \n\n\nThe detailed explanations are as follows:\n\n- **The relationship between similarity and redundancy.** Accurately speaking, high similarity is a prerequisite but not a sufficient condition for context modeling. Fig. (c) shows that although the representation of a 3D scene is sparse, similar to natural images where flat areas (areas with similar values) exist that make the context modeling work, 3D scenes also have flat areas that make the context modeling possible.  \n\n- **Visualization of the bit-saving map.** The bit saving is calculated by estimating the number of bits to encode each anchor with and without using the information of already coded anchors from coarser levels (similar to previous works that calculate the bit-saving maps, e.g., Fig. 4 in [R1]).\n\n- **The anchors of which storage costs are reduced** are those who have **high feature similarities** with their corresponding anchors in the coarser levels and are **difficult to directly represent using existing hyperprior** features. This is the reason why the bit-saving map in Fig. (d) and the similarity map in Fig. (c) do not necessarily match exactly.\n\nWe have enlarged the point size of anchors in coarser levels to make them more visible since they only occupy a small portion. However, it is still challenging to avoid the anchors being small and chaotic due to the large number of anchors representing a scene. \n\n\n[R1] Checkerboard Context Model for Efficient Learned Image Compression, CVPR 2020\n\n\n&nbsp;&nbsp;\n### Typos and Neural Gaussian\nWe use \"neural Gaussian\" to convey their differentiable nature. We follow the symbols and definitions in ScaffoldGS, which mixes the use of \"3D Gaussian\" and \"Neural Gaussian.\" To avoid confusion, we will unify it to \"3D Gaussian\" in the revised paper", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
# Thanks to All the Reviewers for the Insightful Comments

We would like to thank the reviewers for their efforts and insightful comments. We appreciate the reviewers’ acknowledgment regarding the **novelty/motivation** and **performance** of the proposed method. For example:

**Novelty/motivation**:
- "For the first time, introduce autoregressive entropy models" from Reviewer otkA10,
- "The main idea is novel within the 3DGS framework." from Reviewer gvyt,
- "The method is novel, integrating the concept of context modeling from the image compression domain" from Reviewer 8ybR26.

**Performance**:
- "The performance improvement is significant." from Reviewer gvyt,
- "Outperforms in rendering quality while requiring minimal storage usage" from Reviewer fG6F,
- "The results are strong.", "Significantly better compression while retaining high rendering quality" from Reviewer 8ybR.

The questions or weaknesses mentioned by each reviewer are answered separately. Please feel free to discuss with us if you have any further concerns or questions.

&nbsp;
&nbsp;
&nbsp;
&nbsp;

# Highlights
Some reviewers may have concerns regarding the improvements in the ablation study. We want to emphasize that these improvements (**~25%**) are indeed very significant, outperforming those gains achieved in recent published papers on image/representation compression (**~11% accumulated improvements over 3 years**).

&nbsp;
&nbsp;
## The Baseline We Used for Ablation is Very Strong

We want to highlight that the selected baseline method for the ablation study is **strong enough** as shown in the following table. Its performance is **even better** than the **most recent** SOTA method HAC (ECCV'24).

| Tested on BungeeNeRF | PSNR | SSIM | LPIPS | SIZE |
| --- | --- | ---- | --- | --- |
| HAC | 26.48 | 0.845 | 0.250 | 18.49 |
| **Baseline** We Proposed for Ablation | 26.93 | 0.867 | 0.222 | 18.67 |
| ContextGS (Ours) | 26.90 | 0.866 | 0.222 | 14.00 |

A 25% improvement on such a strong baseline/the most SOTA method is already **a very significant improvement**.

&nbsp;
&nbsp;
## The Relative Improvements Depend on How We Select/Claim the Baseline

The improvement upon the baseline highly **depends on how we design/select the baseline**. As shown in Eq. 8 of the paper, all the experiments include the anchor position as the hyperprior. Even in the ablation study, "w/o hyperprior" means we do not use the proposed learnable hyperprior **but still use** the anchor position as the hyperprior.

If we remove the anchor position from the input of the baseline model (**similar to the one used in HAC**), the results are as follows:

| Tested on Rome (BungeeNeRF) | PSNR | SSIM  | SIZE |
| --- | --- | ---- | --- |
| Scaffold-GS | 26.25 | 0.872 | 186.7 |
| Ours* w/o CM w/o HP | 24.83 | 0.850 | 27.85 | 
| Ours* w/o Learnable Hyperprior | 26.53 | 0.873 | 16.54 |
| Ours* w/o Context Model | 26.44 | 0.872 | 19.99 |
| Ours* | 26.43 | 0.872 | 13.74 |
| Ours | 26.38 | 0.871 | 14.06 |

("Ours*" represents that we do not use the anchor position as the hyperprior. "Ours" represents that we use the anchor position as the additional hyperprior, which is the result in Table 4 of the paper.)

Compared with the baseline model without any hyperprior, both the proposed learnable hyperprior and the proposed context model have very significant improvement (**~50% compression rate and 1.+ dB PSNR improvement in total**). Besides, we find that our method is not affected by removing the anchor position. We did not use such a baseline in our paper previously since we think using a strong baseline can **better represent the true performance** of the model and **benefit society**.


&nbsp;
&nbsp;
## \~25% is a Significant Improvement in Entropy-Based Compression

Thirdly, we want to highlight that in the deep entropy-based compression field, \~25% improvement is already very significant. For example, taking CVPR'23 Oral work [a] and CVPR'20 work [b] as a comparison, over **3 years** development, the **accumulated improvements** in the image compression domain is only around **~11%** on the standard benchmark.
(Shown in PSNR/bpp subfigure of Fig.7 in [a]. Under the same PSNR, the bpp of [a] is 0.4 while the bpp of [b] is around 0.45.)

This strongly demonstrates the difficulty of improving performance on a strong entropy baseline and supports the significance of our improvements.

[a] Learned Image Compression with Mixed Transformer-CNN Architectures, CVPR 2023 Oral

[b] Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules, CVPR 2020

Meta-review: This work proposed an autoregressive model at the anchor level for 3DGS compression - which is of certain importance giving the prevailing trends using 3dgs for rendering and acknowledging the value of effective compression technique. The overall idea is of certain novelty and the evaluation is comprehensive. All reviewers give positive scores with high confidence. The AC's recommendation is consistent with the reviewers: acceptance.
Decision: Accept (poster)