3D Scene Compression through Entropy  Penalized Neural Representation Functions
Authors: Thomas Bird
Jona Ballé
Saurabh Singh
Philip Chou
Venue:  (2021)
Keywords: scene representation
compression
neural rendering
entropy coding

Abstract: Some forms of novel visual media enable the viewer to explore a 3D scene from essentially arbitrary viewpoints, by interpolating between a discrete set of original views. Compared to 2D imagery, these types of applications require much larger amounts of storage space, which we seek to reduce. Existing approaches for compressing 3D scenes are based on a separation of compression and rendering: each of the original views is compressed using traditional 2D image formats; the receiver decompresses the views and then performs the rendering. We unify these steps by directly compressing an implicit representation of the scene, a function that maps spatial coordinates to a radiance vector field, which can then be queried to render arbitrary viewpoints. The function is implemented as a neural network and jointly trained for reconstruction as well as compressibility, in an end-to-end manner, with the use of an entropy penalty on the parameters. Our method significantly outperforms a state-of-the-art conventional approach for scene compression, achieving simultaneously higher quality reconstructions and lower bitrates. Furthermore, we show that the performance at lower bitrates can be improved by jointly representing multiple scenes using a soft form of parameter sharing.

Under review as a conference paper at ICLR 2021
3D SCENE COMPRESSION THROUGH ENTROPY PENALIZED NEURAL REPRESENTATION FUNCTIONS
Anonymous authors Paper under double-blind review
ABSTRACT
Some forms of novel visual media enable the viewer to explore a 3D scene from essentially arbitrary viewpoints, by interpolating between a discrete set of original views. Compared to 2D imagery, these types of applications require much larger amounts of storage space, which we seek to reduce. Existing approaches for com- pressing 3D scenes are based on separation of compression and rendering; each of the original views is compressed using traditional 2D image formats; the re- ceiver decompresses the views and then performs the rendering. We unify these steps by directly compressing an implicit representation that maps spatial coordinates to a radiance vector field, which can then be queried to render arbitrary viewpoints. The function is implemented as a neural network and jointly trained for reconstruction as well as compressibility, in end-to-end manner, with the use of an entropy penalty on the parameters. Our method signif- icantly outperforms state-of-the -frtt--artt conventional approach for scene compres- sion, achieving simultaneously higher quality reconstructions and lower bitrates. Furthermore, we show that the performance at lower bitrates can be improved by jointly representing multiple scenes using soft form of parameter sharing.
the scene, a function
INTRODUCTION
The ability to render 3D scenes from arbitrary viewpoints can be seen as a big step in the evolution of digital multimedia, and has applications such as mixed reality media, graphic effects, design, and simulations. Often such renderings are based on number of high resolution images taken of some original scene, and it is clear that to enable many applications, the data will need to be stored and transmitted efficiently over low- low-bandwidth channels (e.g., to mobile phone for augmented reality).
Traditionally, the need to compress this data is viewed as a separate need from rendering. For exam- ple, light field images (LFI) consist of a set of images taken from multiple viewpoints. To compress the original views, often standard video compression methods such as HEVC (Sullivan et al.) 2012) repurposed (Jiang et al., 2017 Barina et al.) 2019). Since the range of views is narrow, light field images can be effectively reconstructed by "blending" a smaller set of representative views (Astola & Tabus 2018 Jiang al. 2017 Zhao al. 2018 Bakir et al. 2018 Jia et al. 2019). Blending based approaches, however, may not be suitable for the more general case of arbitrary- 3D) scenes, where a very diverse set of original views may increase the severity of occlusions, and thus would require storage of prohibitively large number of views to be effective.
A promising avenue for representing more complete 3D scenes is through neural representation functions, which have shown a remarkable improvement in rendering quality (Mildenhall et al. 2020 Sitzmann al. 2019: Liu et al. 2020;; Schwwaz al., 2020). In such approaches, views from scene are rendered by evaluating the representation function at sampled spatial coordinates and then applying a differentiable rendering process. Such methods are often referred to as implicit representations, since they do not specify explicitly the surface locations and properties within the scene, which would be required to apply some conventional rendering techniques like rasterization (Akenine- Möller et al. 2019). However, finding the representation function for a given scene re- quires training a neural network. This makes this class methods difficult to use as a rendering method in the existing framework, since it is computationally infeasible on low-powered end de- vice like a mobile phone, which are often on the receiving side. Due to the data processing inequality, it may also be inefficient to compress the original views (the training data) rather than the trained

---

Under review as a conference paper at ICLR 2021
Views 3D scene
Neural representation function
Loss
42
Entropy
Rate
length(1000010...
Poses }
Sender
Reconstruction error
Radiance field
Images {Xx}
Novel rendering
11000010...
Pose
Radiance field
Receiver
Figure I: Overview of cNeRF. The sender trains an entropy penalized neural representation function on set of views from scene, minimizing combination of rate and distortion. The receiver can use the compressed model to render novel views.
representation itself, because the training process may discard some information that is ultimately not necessary for rendering (such redundancy in the original views, noise, etc.).
In this work, we propose to apply neural representation functions to the scene compression problem by compressing the representation function itself. We use the NeRF model (Mildenhall et al. 2020). a method which has demonstrated the ability to produce high-quality renders novel views, as representation function. To reduce redundancy of information in the model, we build upon the model compression approach Oktay et al. (020), applying an entropy penalty to the set of discrete reparameterized neural network weights. The compressed NeRF (cNeRF) deseribes radiance field, which is used in conjunction with a differentiable neural renderer to obtain novel views (see Fig. [ To verify the proposed method, we construct a strong baseline method based the approaches seen in the field of light field image compression. cNeRF consistently outperforms the baseline method, producing simultaneously superior renders and lower bitrates. We further show that cNeRF can be improved in the low bitrate regime when compressing multiple scenes at once. To achieve this, introduce novel parameterization which shares parameters across models and optimize jointly across scenes.
2 BACKGROUND
We define multi-view image dataset as a set of tuples D = ((V,,,,))) X,) where V, is the camera pose and X,, the corresponding image from this pose. We refer the 3D ground truth that the views capture the scene. In what follows, we first provide a brief review of the neural rendering and the model compression approaches that we build upon while introducing the necessary notation.
Neural Radiance Fields (NeRF) The neural rendering approach of Mildenhall et al. (2020) uses neural network to model a radiance field. The radiance field itself is a learned function go R + ( +)+ mapping 3D spatial coordinate and 2D viewing direction to a RGB value and a corresponding density element. To render view, the RGB values are sampled along the relevant rays and accumulated according to their density elements. The learned radiance field mapping go is parameterized with two multilayer perceptrons (MLPs), which Mildenhall et al. (2020) refer to as the "coarse" and "fine" networks, with parameters 9. and 0 respectively. The imput locations to the coarse network are obtained by sampling regularly along the rays, whereas the input locations to the fine network sampled conditioned on the radiance field of the coarse network. The networks
2

---

Under review as a conference paper at ICLR 2021
trained by minimizing the distance from their renderings to the ground truth image:
(9(00iV Vo) Xnll 18 Vn, 0.) x
(e,)
Ly(0g:
Where || ||2 is the Euclidean norm and the Xn are the rendered views. Note that the rendered view from the fine network X relies on both the camera pose V, and the coarse network to determine the spatial locations to query the radiance field. We drop the explicit dependence of f on 0g in the rest of the paper to avoid cluttering the notation. During training, we render only a minibatch of pixels rather than the full image. We give a more detailed description of the NeRF model and the rendering process in Appendix Sec. A
Model Compression through Entropy Penalized Reparameterization The model compression work of OKtay et al. (2020) reparameterizes the model weights é into a latent space as Φ. The latent weights are decoded by a learned function F, i.e. é F($). The latent weights + are modeled as samples from learned prior q, such that they can be entropy coded according to this prior. To minimize the rate, i.e. length of the bit string resulting from entropy coding these latent weights, a differentiable approximation of the self-information [ weights is penalized. The continuous the straigh-tthrouge estimator (Bengio et al., 2013) used to obtain surrogate gradients of the loss function. Following Ballé et al. (2017). uniform noise added when learning the continuous prior ( + u) where u; (122.2) 7) V This uniform noise is stand-in for the quantization, and results in a good approximation for the self-information through the negative log-likelihood of the noised continuous latent weights. After training, the quantized weights are obtained by rounding, = Φ 1. and transmitted along with discrete probability tables obtained by integrating the density the quantization intervals. The continuous weights t and any parameters in q itself can then be discarded.
loga (g(()) of the latent are quantized before being applied the model, with
METHOD
To achieve a compressed representation of a scene, we propose to compress the neural scene repre- sentation function itself. In this paper we use the NeRF model as our representation function. To compress the NeRF model, we build upon the model compression approach of Oktay et al. (2020) and jointly train for rendering as well as compression in an end-to-end trainable manner. We subse- quently refer to this approach cNeRF. The full objective that we seek to minimize is:
L(Φ, v = L.(FF((.))) + ( +A E
T())
(2)
Distortion
Rate
where y denotes the parameters of as well any parameters the prior distribution g. and we have explicitly split & into the coarse 4. and fine o, components such that +- = {T... c. o }. X is a trade-off parameter that balances between rate and distortion. A rate-distorte distortton (RD) plot can be traced by varying À explore the performance of the compressed model at different bitrates.
Compressing a single scene When training cNeRF to render a single scene, we have to choose how to parameterize and structure F and the prior distribution q over the network weights. Since the networks are MLPs, the model parameters for a layer / consist of the kernel weights and biases [W1, bi). We compress only the kernel weights Wi. leaving the bias uncompressed since is much smaller in size. The quantized kernel weights W are mapped the model weights by F,, i.e. Wi = F(Wi). Fi constructed as an affine scalar transformation, which is applied elementwise to Wr:
F( ou Wi,6 -3
We take the prior to be factored over the layers, such that we learn prior per linear kernel gi- Within each kernel, take the weights in Wi be i.i.d. from the univariate distribution qı, parameterized by a small MLP, as per the approach of Ballé et al. (2017). Note that the parameters of this MLP can be discarded after training (once the probability mass functions have been built).
3

---

Under review as a conference paper at ICLR 2021
Ground teuth
32 bita per paramelar
4.38 byp
0.99 byp
Lego
Ground pec paramelec
hop
1.09 bpp
Pern
Figure 2: Renderings of the synthetic Lego scene and real Fern scene from the uncompressed NeRF model, at 32 bits per parameter (bpp), and from cNeRF with À E {0.0001, 0.01].
Compressing multiple scenes While the original NeRF model is trained for a single scene, we hypothesize that better rate-distortion performance can be achieved for multiple scenes, especially they share information, by training joint model, For dataset of M scenes, we parameterize the kernel weights of model m, layer as:
W F.th (W7"" 3:)
(4) Compared to Eqn. B we have added a shift, parameterized as scalar linear transformation of M..
a." W," 31th +
discrete shift S that is shared across all models m {I,.....} Si has the same dimensions as the kernel W and as with the discrete latent kernels, S; is coded by learned probability distribution. The objective for the multi-scene model becomes:
M
C( V = Σ
L.h (F
m #2) + LF Tm ())) +
Σ +E0
((0)
+
Σ $E
I(Ø)
(5)
where 4- is the set of all discrete shift Š parameters, and the losses, latent weights and affine transforms are indexed by scene and model m. Note that this parameterization has more parameters than the total of the M single scene models, which at first appears counter-intuitie, since we wish to reduce the overall model size. It is constructed as such so that the multi-scene parameterization contains the M single scene parameterizations they can be recovered by setting the shared shifts to zero. If the shifts are set to zero then their associated probability distributions can collapse to place all their mass at zero. So expect that if there is little benefit to using the shared shifts then they can be effectively ignored, but there a benefit to using them then they can be utilized. As such, interpret this parameterization inducing soft form of parameter sharing.

---

Under review as a conference paper at ICLR 2021
oNeRF
HEVO LLFF
Figure 3: A comparison of four (zoomed in) renderings from cNeRF with À = 0.0001 and HEVC L.LFF with QP=30. HEVC L.L.F shows obvious artifacts such as ghosting around edges and an overall less crisp rendering.
EXPERIMENTS
Datasets To demonstrate the effectiveness of our method, we evaluate on sets of scenes used by Mildenhall et al. (2020):
Synthetic. Consisting of 800 x 800 pixel views taken from either the upper hemisphere or entire sphere around object rendered using the Blender software package. There are 100 views taken to be in the train set and 200 in the test set.
Real. Consisting of a set of forward facing 1008 x 756 pixel? photos of a complex scene. The number of images varies per scene, with 1/8 the images taken as the test images.
Since we are interested in the ability of the receiver to render novel views, all distortion results (for any choice of perceptual metric) presented are given on the test sets.
Architecture and Optimization We maintain the same architecture for the NeRF model as Milden- hall et al. (2020), consisting of 13 linear layers and ReLU activations. For cNeRF we use Adam (Kingma & Ba] 2015) optimize the latent weights & and the weights contained the decoding functions F. For these parameters we use initial learning rate of 5 x 10 and learning rate decay over the course of learning, as per Mildenhall et al. (2020). For the parameters of the learned prob- ability distributions q. find it beneficial to
lower learning rate of 5 x 10 (0,5, such that the distributions do not collapse prematurely. We initialize the latent linear kernels using the scheme of Glorot & Bengio (2010), the decoders F near the identity.
Baseline We follow the general methodology exhibited light field compression and take the compressed representation of the scene to be compressed subset of the views. The receiver then decodes these views, and renders novel views conditioned on the reconstructed subset. We use the video codec HEVC compress the subset of views, is done by Tiang al. (2017). To render novel views conditioned on the reconstructed set of views, we choose the Local Light Field Fusion (LLFF) approach of Mildenhall al. (2019). LLFF a novel view is rendered by promoting nearby views to multiplane images, which are then blended. We refer the full baseline subsequently as HEVC LLFF.
state-of-the-art learned approach in which
4.1 RESULTS
Single scene compression To explore the frontier of achievable rate-distortion points for cNeRF, we evaluate at range of entropy weights À for four scenes two synthetic (Lego and Ficus) and two real (Fern and Room). To explore the iite-distortion frontier for the HEVC LLFF baseline evaluate at range of QP values for HEVC. We give more thorough description of the exact specifications of the HEVC LLFF baseline and the ablations we perform to select the hyperpa- rameter values in Appendix Sec. We show the results in Fig. 4 We also plot the performance of
5

---

Under review as a conference paper at ICLR 2021
32
Forn (compressed NeRF)
30
Ferm (HEVC LUFF)
Room (compressed NeRF)
Lego (compreszed NeRF)
Room (HEVC LLFF)
Lago (HEVC LLFF)
Ficus (compressed NeRF)
Ficus (HEVC. + LLFF)
22
Size
Size
Figure 4: Rate-distortion curves for both the cNeRF and HEVC LLFF approaches, on real (left) and two synthetic (right) scenes. We truncate the curves for HEVC + LLFF, since increasing the bitrate further does not improve PSNR. See Fig. for the full curves.
NeRF PSNR
cNeRF (A = le
HEVC LLFF (QP=30)
Scene
PSNR
Size (KB)
Reduction
PSNR
Size (KB)
Chair
33.51
32.28
621
8.32x
24.38
5,778
Drums
24.85
24.85
870
5.94
20.25
5,985
Ficus
30.58
30.35
701
7.37x
21.49
3,678
Hotdog
35.82
34.95
916
5.65
30.18
2.767
Lego
32.61
31.98
707
7.31x
23.92
4,665
Materials
29.71
29.17
670
7.71x
22.49
3,134
Mic
33.68
32.11
560
9.23
28.95
3,032
Ship
28.51
28.24
717
7.21
24.95
5,881
Room
31.06
30.65
739
7.00
26.27
886
Fern
25.23
25.17
990
S.22x
22.16
2,066
Leaves
21.10
20.95
1.154
4.48x
18.15
3,162
Fortress
31.65
31.15
818
6.32x
26.57
1,149
Orchids
20.18
20.09
1,218
4.24x
17.87
2,357
Flower 27.42
27.21
938
5.51x
23,46
1,009
T-Rex
27.24
26.72
990
5.22x
22.30
1,933
Homs
27.80
27.28
995
5.20x
20.71
2,002
Table I: Results comparing the uncompressed NeRF model,cNeRI and HEVC LLFF baseline.
We pick À and QP to give reasonable trade-off between bitrate and PSNR. The reduction column the reduction in the size of cNeRF compared to the uncompressed NeRF model, which has size of 5,169KB. Note that for all scenes, cNeRF achieves both higher PSNR and lower bitrate than HEVC LLFF.
the uncompressed NeRF model - demonstrating that by using entropy penalization the model size be reduced substantially with relatively small increase in distortion. For these scenes plot renderings at varying levels of compression in Fig. 2 and Fig. g The visual quality of the renderings does not noticeably degrade when compressing the NeRF model down bitrates of roughly 5-6 bits parameter (the precise bitrate depends on the scene), At roughly bit per parameter, the visual quality has degraded significantly, although the renderings are still sensible and easily recognisable. We find this to be a surprising positive result, given that assigning a single bit parameter is ex- tremely restrictive for such complex regression task as rendering. Indeed, to our knowledge no binary neural networks have been demonstrated to be effective on such tasks.
Although the decoding functions FF (Eqn. 3 are just relatively simple scalar affine transformations, do not find any benefit to using more complex decoding functions. With the parameterization given, most the total description length of the model is in the coded latent weights, not the param-
eters of the decoders entropy models. We give full breakdown in Tab. 5
Fig. shows that cNeRF clearly outperforms the HEVC LLFF baseline, always achieving lower distortions (roughly) equivalent bitrate. Reconstruction quality is reported peak signal-to- noise ratios (PSNR). The results are consistent with earlier demonstrations that NeRF produces much better renderings than the L.LFF model (Mildenhall at. 2020). However, it still interesting

---

Under review as a conference paper at ICLR 2021
31
27.0
26.5
26.0
29
25.5
28
27
Lego + Ficus (Separate)
24.5
Fern Room (Separate)
Lego Ficus (oint, shared shift
Forn Room (jaint, shared shitt)
0.5
1.0
1.5 Size (MB)
2.0
2.5
0.5
1.0
1.5
2.0 Size (MB)
2.5
3.0
3.5
Figure 5: Rate-distortion curves for comparing the multi-scene model with single shared shift the single scene models. The models are shown for two synthetic (left) and two real scenes (right).
to see that this difference persists even at much lower bitrates. To evaluate on the remaining scenes, we select a single value for cNeRF and QP value for HEVC + LLFF. We pick the values to demonstrate reasonable trade-off between rate and distortion. The results are shown in Tab. [ For every scene the evaluated approaches verify that cNeRF achieves a lower distortion at lower bitrate. We can see also that cNeRF consistently able reduce the model size significantly without seriously impacting the distortion. Further, we evaluate the performance of cNeRF and HEVC + LLFF for other perceptual quality metrics in Tab. 3 and 4 Although eNeRF is trained to minimize the squared error between renderings and the true images (and therefore maximize PSNR), cNeRF also outperforms HEVC + LLFF in both MS AS-SSIM (Wang et al., 2003) and LPIPS (Zhang et al. 2018). This is significant, since the results of Mildenhall et (2020) indicated that for SSIM and LPIPS, the LLFF model had a similar performance to NeRF when applied to the real scenes. We display comparison of renderings from cNeRF and HEVC LLFF in Fig,
Multi-scene compression For the multi-scene case we compress one pair of synthetic scenes and one pair of real scenes. We train the multi-scene cNeRF using single shared shift per linear kernel, as per Eqn. To compare the results to the single scene models, we take the two corresponding single
Fs sum the sizes and average the distortions. We plot the resulting rate- dittoiisoo frontiers in Fig. 5 The results demonstrate that the multi-scene cNeRF improves upon the single scene cNeRFs low bitrates, achieving higher PSNR values with smaller model. This meets our expectation, since the multi-scene cNeRF can share parameters via the shifts (Eqn. 4 and so decrease the code length of the scene-specific parameters. At higher bitrates we see benefit to using the multi-scene parameterization, and in fact see slightly worse performance. This indicates that in the unconstrained rate setting, there is no benefit using the shared shifts, and that they may slightly harm optimization.
RELATED WORK
Scene Compression A 3D scene is typically represented as a set of images, for each view. For a large number of views, compressing each image individually using a conventional compression method require large amount of space. As result, there is body of compression research which aims to exploit the underlying scene structure of the 3D scene to reduce space requirements. lot of research has been focused on compressing light field image (LFI) data (Astola & Tabus 2018 Jiang et al.] 2017; Bakir et al., 2018 Jia et al., 2019) Zhao et al. 2018). LFI data gener- ally consists of multiple views with small angular distances separating them. This of views can be used to reconstruct signal on the 4D domain of rays of the light field itself, thus permitting post-procsssing tasks such as novel view synthesis and refocusing.
majority of works select representative subset of views to transmit from the scene. These are compressed and transmitted, typically using a video codec, with the receiver decoding these images and then rendering any novel view for unobserved (during training) camera pose. Reconstruction for novel camera poses can be performed using traditional methods, such as optical flow (Jiang et al. 2017). or by using recent learned methods that employ convolutional neural networks (Zhao et al. 2018 and generative ad- versarial networks (Jia et al. 2019). A contrasting approach to multi- view image compression is proposed by Liu (2019) which pair of images from two viewpoints is compressed by con-
7

---

Under review as a conference paper at ICLR 2021
ditioning the coder of the second image on the coder of the first image. It is important to emphasise that we are not studying this kind of approach in this work, since we wish the receiver to have the ability to render novel views.
Neural Rendering is an emerging research area which combines learned components with render- ing knowledge from computer graphics. Recent work has shown that neural rendering techniques can generate high quality novel views wide range of scenes (Mildenhall et al.) 20200 [Sizzmann 2020; 2020 Schwarz al.) 2020). In this work we build upon the method Milden- hall et al. (2020), coined as a Neural Radiance Field (NeRF), for single scene compression and then extend it with a novel reparameterization for jointly compressing multiple scenes. Training neural representation networks jointly across different scenes (without compression) has been explored by Sitzmann et al. (219)) and [iu et al. (2020), who use a hypernetwork (Ha et al., 2017) to map a latent vector associated with each scene to the parameters of the representation network. Liu et al. (2020) note that the hypernetwork approach results in significant degradation of performance when applied to the NeRF model (a loss of more than 4 dB PSNR). In contrast, our approach of shared reparameterization significantly different from these methods.
Liu |
al. [019
Model Compression There is body of research for reducing the space requirements of deep neural networks. Pruning tries to find a sparse set of weights by successively removing a subset of weights according to some criterion (Han et al. 2016 Li ct al. (0117). Quantization reduces the precision used to describe the weights themselves (Courbariaux et al. 2016 Li et al., (016). In this work we
focus instead on weight coding approaches (Havasi et al. 2019 Oktay et al. 2020) that code the
model parameters to yield compressed representation.
6 DISCUSSION AND CONCLUSION
Our results demonstrate that cNeRF produces far better results as a compressed representation than state-of-the-art baseline, HEVC+LLFF, which follows the paradigm of compressing the original views. In contrast, our method compresses a representation of the radiance field itself. This is important for two reasons:
Practically, compressing the views themselves bars the receiver from using more complex and better-performen rendering methods such as NeRF, because doing this would require training to be performed at the receiving side after decompression, which is computation- ally infeasible in many applications.
Determining the radiance field and compressing it on the sending side may have coding and/or representational benefits, because of the data processing inequality: the cNeRF pa- rameters are a function of the original views, and as such must contain equal to or less information than the original views (the training data). The method thus relieved of the need to encode information in the original views that is not useful for the rendering task.
It is difficult to gather direct evidence for the latter point, as the actual entropy of both representations difficult to measure (we can only upper bound by the compressed size). However, the substantial performance improvement of our method compared to HEVC+LLFF suggests that the radiance field a more economical representation for the scene.
Linked to our choice is also the fact that we adopt more realistic evaluation methodology than many scene compression techniques. Rather reporting the bitrate and reconstruction quality of the original views, we evaluate our method (and the baseline) by reporting the reconstruction quality of a held-out set of views of each scene, which was not used for training. Since in a free-viewpoint scenario, the vast majority of rendered views will not correspond to one of the original ones, we believe this more accurately measures success of the compared methods.
The encoding time for cNeRF is long, given that new scene must be trained from scratch. Impor- tantly though, the decoding time is much less, as it is only required to render the views using the decompressed NeRF model. cNeRF enables neural scene rendering methods such as NeRF to be used for scene compression, as it shifts the complexity requirements from the receiver to the sender. In many applications, it is more acceptable to ineur high encoding times than high decoding times, as one compressed data point may be decompressed many times, allowing amortization of the en- coding time, and since power-constrainel devices are often at the receiving side. Thus, our method represents a big step towards enabling neural scene rendering in practical applications.
8

---

Under review as a conference paper at ICLR 2021
REFERENCES
T. Akenine-Möller E. Haines, and N. Hoffman. Real-time rendering. Crc Press, 2019.
Astola and Tabus. Wasp: Hierarchical warping, merging, and sparse prediction for light field image compression. In 2018 7th European Workshop on Visual Information Processing (EUVIP), pp. 1-6, 2018.
N. Bakir, W. Hamidouche, O. Déforges, K. Samrouth, and M. Khalil. Light field image compression based on convolutional neural networks and linear approximation. In 2018 25th IEEE Interna- tional Conference on Image Processing (ICIP), pp. 1128-1132, 2018.
Ballé, Laparra, and E. Simoncelli. End-to-end optimized image compression. In 5th Int. Conf. on Learning Representations (ICLR), 2017.
D. Barina, T. Chlubna, M. Solony, D. Dlabaja, and Zemcik. Evaluation of 4d light field compres- sion methods. In 177x:11905077,,,,, 2019.
Y. Bengio, N. Léonard, and A. Courville. Estimating propagating gradients through stochastic neurons for conditional computation. In
2013.
M. Courbariaux, Y. Bengio, and J.-P. David. Binaryconnect: Training deep neural networks with binary weights during propagations. In arXiv:/511.00330 2016.
X. Glorot and Y. Bengio. Understanding the difficulty of training deep feedforward neural networks. volume 9 of Proceedings of Machine Learning Research, pp. 249-256, 2010.
D. Ha, A. Dai, and Q. Le. Hypernetworks. In International Conference Learning Representa- tions, 2017.
S. Han, H. Mao, and W. J. Dally. Deep compression: Compressing deep neural network with pruning, trained quantization and huffman coding. 4th International Conference on Learning Representations, ICLR 2016, 2016.
M. Havasi, R. Peharz, and J. M. Hernández-Lobato. Minimal random code learning: Getting bits back from compressed model parameters. In International Conference on Learning Representa- tions, 2019.
C. Jia, X. Zhang, S. Wang, S. Wang, and S. Ma. Light field image compression using generative adversarial network-based view synthesis. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 91):117--18, 2019.
X. Jiang, M. Pendu, and C. Guillemot. Light field compression using depth image based view synthesis. In 2017 IEEE International Conference on Multimedia Expo Workshops (ICMEW), pp- 19-24, 2017.
D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. In 3rd International Confer- ence on Learning Representations, ICLR 2015, 2015.
F. Li, B. Zhang, and B. Liu. Ternary weight networks. In arXiv:1605.04711 2016.
H. Li, A. Kadav, I. Durdanovie, H. Samet, and H.P. Graf. Pruning filters for efficient convnets. In a6Xiv: 08710, 2017.
J. Liu, S. Wang, and R. Urtasun. Dsic: Deep image compression. In 2019 IEEE/CVE Inter- national Conference on Computer Vision (ICCV, pp. 3136-3145, 2019.
L. Liu, J. Gu, K. Z. Lin, T.SS. Chua, and C. Theobalt. Neural sparse voxel fields. In arXiv:2007. 1571, 2020.
B. Mildenhall, P. Srinivasan, R. Ortiz-Cayon, N. K. Kalantari, R. Ramamoorthi, R. Ng. and A. Kar. Local light field fusion: Practical view synthesis with prescriptive sampling guidelines. ACM Transactions Graphics (TOG), 2019.

---

Under review as a conference paper at ICLR 2021
B. Mildenhall, P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng. Nerf: Represent- ing scenes as neural radiance fields for view synthesis. In ECCV, 2020.
D. Oktay, J. Ballé, S. Singh, and A. Shrivastava. Scalable model compression by entropy penalized reparameterizattoo.. In International Conference on Learning Representations, 2020.
K. Schwarz, Y. Liao, M. Niemeyer, and A. Geiger. Graf: Generative radiance fields for 3d-aware image synthesis. In arXin:2007.02442, 2020.
V. Sitzmann, M. Zollhöfer, and G. Wetzstein. Scene representation networks: Continuous 3d- structure-awar neural scene representations. In Advances in Neural Information Processing Systems, 2019.
V. Sitzmann, J. Martel, A. Bergman, D. Lindell, and G. Wetzstein. Implicit neural representations with periodic activation functions. In arXiv, 2020.
G.J. J. Sullivan, J. Ohm, W. Han, and T. Wiegand. Overview of the high efficiency video coding (hevc) standard. IEEE Transactions on Circuits and Systems for Video Technology. 22(12):1649-166 2012.
M. Tancik, P. Srinivasan, B. Mildenhall, S. Fridovich-Keit N. Raghavan, U. Singhal, R. Ramamoor- thi, J. T. Barron, and R. Ng. Fourier features networks learn high frequency functions low dimensional domains. In arXiv:2006.10739, 2020.
Z. Wang, E. P. Simoncelli, and A. C. Bovik. Multiscale structural similarity for image quality assessment. In The Thrity-Seventh Asilomar Conference on Signals, Systems Computers, 2003, volume 2, pp. 1398-1402 Vol.2, 2003.
R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang. The unreasonable effectiveness of deep features as perceptual metric. In arXiv: 033924, 2018.
Z. Zhao, S. Wang, C. JJa, X. Zhang, S. Ma, and J. Yang. Light field image compression based on deep learning. In 2018 IEEE International Conference on Multimedia and Expo (ICME), pp. 1-6, 2018
10

---

Under review as a conference paper at ICLR 2021
A NEURAL RADIANCE FIELDS
The neural rendering approach of Mildenhall et al. (2020) uses a neural network to model a radiance field. The radiance field itself is a learned mapping ge IR + (R3 t),, where the input is a 3D spatial coordinate = (xr y, 2) E IR' and a 2D viewing direction d = (0, 6) E RR.. The NeRF model also makes use of positional encoding into the frequency domain, applied elementwise to spatial and directional inputs
<(p) (p))), cos(2"mp)) (p), sinn xp), cos (2s(22='Tp))
(6)
This type of encoding has been shown to be important for implicit models, which take as input low dimensional data which contains high frequency information TTancik et al. 2020 Sitzmann et al.) 2020).
The network output is an RGB value (r. 9. E IR3 and a density element E RR. To render a particular view, the RGB values are sampled along the relevant rays and accumulated according to their density elements. In particular, the color c(r) of a ray r = (o + td : 2 0}, in direction from the camera origin o, is computed as
c(r) Tx((-ex((-oro))
where T; exp (-E o,6;)
where (c,, 0;) is the output of the mapping evaluated at (Pr, d), where p; = ttd, t, the distance of sample i from the origin along the ray, and 6; = f,+1 - t; is the distance between samples. The color c(r) can be interpreted as the expected color of the point along the ray in the scene closest to the camera, if the points in the scene are distributed along the ray according to inhomogeneous Poisson process. Since in a Poisson process with density o; the probability that there are no points in an interval of length 8, is exp(-orç)) Thus T; is the probability that there are points between i and 4i, and (1 - exp(-a,5)
is the probability that there is a point between 4 and 44+1- The rendered view X comprises pixels whose colors e(r) are evaluated at rays emanating from the same camera origin o but having slightly different directions d, depending on the camera pose V.
B HEVC + LLFF SPECIFICATION AND ABLATIONS
There are many hyperparameters to select for the HEVC + LLFF baseline. The first we consider is the number of images to compress with HEVC. If too many images are compressed with HEVC then at some point the performance of L.FFF will saturate and an unnecessary amount of space will be used. On the other hand, if too few images are compressed with HEVC, then LLFF will find it difficult to blend these (de)coopeessed images to form high quality renders. To illustrate this effect, we run ablation on the Fern scene where we vary the number of images we compress with HEVC, rendering a held out set of images conditioned on the reconstructions. The results are displayed in Fig. [ We can clearly see the saturation point at around 10 images, beyond which there is no benefit to compressing extra images. Thus when picking the number of images to compress for new scenes, do not use more than 4 per test image (which corresponds to compressing 12 images in our ablation).
The second effect we study the order in which images are compressed with HEVC, which affects the performance as HEVC is a video codec and thus sensitive to image ordering. It stands to reason that the more the sequence of images resemble a natural video, the better coding will be. As such, we consider two orderings: firstly the "snake scan" ordering, in which images are ordered vertically by their camera pose, going alternately left to right then right to left. The second the "lozenge" ordering (Jiang et al.) 2017), in which images are ordered by the camera pose in a spiral outwards from their centre. Both orderings appear sensible since they always step from a given camera pose to an adjacent pose. We compare results compressing and reconstructing set of images using HEVC across a range of Quantization Parameter (QP) values for the Fern scene in Tab. 2 The difference between the two orderings is very small. Since snake scan is simpler to implement, all our experiments.
this in
The effect of changing QP is demonstrated in Fig. 7 and select QP=30 for the experiments in which we choose one rate rate- distortin point to evaluate, since it achieves almost the same performance as QP=20 and QP=10 with considerably less space.

---

Under review as a conference paper at ICLR 2021
23
22
PSNR
8
10
12
16
Number comoressed with LEEV
Figure 6: Test performance of the HEVC LLFF baseline across different number of images com- pressed with HEVC. The decompressed images are used by LLFF to reconstruct the test views.
Fem
26
Room
Lego
25
Ficus
PSNR
QP=30
QP=20
11=10
4P=40
23
QP-50
22
21
20
25
Size (MB)
Figure 7: Full rate-distortion curves for HEVC LLFF, with labels showing effect of the QP parameter. To avoid clutter, only the Lego QP labels given, and the other scenes are similarly ordered from QP=10 on the right to QP=50 on the left.
C EXTRA RESULTS
Here we present some further results from our experiments, including results on different perception metrics, breakdown of the cNeRF model size and extra comparisons of renderings.
12

---

Under review as a conference paper at ICLR 2021
QP
Snake scan
Lozenge
10
50,9
50.8
20
42.5
42.4
30
33.9
33.8
40
26.7
26.8
50
22.6
22.5
Table 2: PSNR values from compressing and reconstructing a set of images from the Fern scene, with two different orderings on the images.
cNeRE (A le 1e PSNR(UV)
HEVC + LLFF (QP=30)
Scene
PSNR(Y)
PSNR(Y)
PSNR(UV)
Chair
36.64
45.21
33.41
42.46
Drums
26.68
37.17
21.72
34.72
Ficus
31.77
43.50
24.05
37.00
Hotdog
36.31
42.70
31.98
41.62
Lego
29.89
40.56
24.94
35.92
Materials
26.82
38.53
16.23
35.04
Mic
33,36
48.02
29.87
48.64
Ship
28.27
38.46
26.48
38.48
Room
32.59
44.94
27.05
42.50
Fern
25.16
40.33
22.15
38,06
Leaves
21.17
36.09
18.43
34.52
Fortress
31.60
44.70
27.31
41.67
Orchids
20.43
34.65
18.17
32.11
Flower 27.91
38.25
24.27
33.90
T-Rcx
26.77
42.64
22.42
40.07
Horns
27.68
42.65
22.57
40.13
Table 3: PSNR values comparing cNeRF to HEVC + LLFF. The images are rendered in the YUV color encoding and the PSNR is computed the channel and average of the UV channels. cNeRF is superior to HEVC LLFF in PSNR(Y) and PSNR(UV) for all scenes except PSNR(UV) for Mic and Ship. Note that the bitrates are the same as for Tab.
eNeRF (A = 1e-1)
HEVE + LLFF (QP=30)
Scene
MS-SSIM(Y)
MS- MS-SSIM(RGB)
LPIPS
MS-SSIM(Y)
MS-SSIM(RGB)
LPIPS
Chair
0.997
0,997
0.014
0.989
0.989
0.026
Drums
0,953
0.952
0.070
0.910
0.909
0.109
Ficus
0.986
0.986
0.023
0.925
0.924
0.069
Hotdog
0.992
0,990
0.041
0,983
0,981
0.070
Lego
0.983
0.980
0.034
0,951
0.94S
0.080
Materials
0,971
0.970
0,047
0.833
0.832
0.172
Mic
0.992
0.992
0,022
0.988
0.988
0.026
Ship
0.891
0.889
0.201
0.879
0.883
0.174
Room
0,979
0,977
0,087
0.950
0.948
0.168
Fern
0.934
0,932
0.187
0.862
0.862
0.239
Leaves
0.909
0,905
0.204
0.843
0.842
0.244
Fortress
0.966
0.962
0.090
0.918
0.914
0.165
Orchids
0.852
0.849
0,220
0.720
0.721
0.320
Flower
0.945
0.941
0.138
0.911
0.906
0.171
T-Rex 0.962
0.961
0.101
0.905
0.903
0.182
Horns
0.953
0.951
0.169
0.886
0.884
0.238
Table 4: MS-SSIM and LPIPS values comparing cNeRF to HEVC + LLFF MS-SSIM is given in both the Y channel (from the YUV color encoding) and RGB. LPIPS is given for RGB, as it only defined for such. In all cases, cNeRF is superior to HEVC LLFF in MS-S and LPIPS. Note that the bitrates are the same as for Tab.
13

---

Under review as a conference paper at ICLR 2021
Entropy weight
Rate (KB)
Overhead (KB)
10
119
23
10-3
293
27
1 X 10-4
673
34
X 10-5
1061
42
Table 5: Breakdown of the cNeRF size across four entropy weights trained on the Lego scene. The size is divided into the size of the coded latent weights (the rate) and everything else (the overhead). The overhead consists of description lengths the probability built from the prior y, the parameters of the decoding functions F and any bias parameters.
Ground teuth
32 bis per parameter
6.64 byp
1.83 hpp
Ficus
Ground per parameler
bop
1.15 bpp
Room
Figure 8: Renderings the synthetic Ficus scene and real Room scene from the uncompressed NeRF model, at 32 bits per parameter (bpp), and from cNeRF with X E (0.0001,0.0) 0.013.
14


Reviews:
[{"review_id": "dp-fG2Va-95", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "I think the paper is well written, and explains the details of the method well. I addition I think choices made in the method are intelligent and well justified. \n\nMy concerns with the paper lie in two area. The first is that I am not convinced that the problem this paper seeks to solve, ie  compression of a 3D scene, is so relevant to the ML research community that is justifies the lack of novelty in the method. From reading this paper it feel like 2 approaches have been found , ie, NeRF and model compression using entropy penalty, which happen to work well together, but the actual degree of novel research contribution seems low.  If 3D scene compression was an established area of research in the ML research community, with many previous works proposing high performing solutions,  this approach may be justified as sufficiently better across various metrics. However, given the lack of this prior work, the solution feels far more like an engineering solution to allow NeRF to work well on cellphones then machine learning research. \n\nMy second concern is the lack of comparison to other approaches to scene compression. From the way I view the problem you have proposed, you are assuming practically explicit 3D scene information as input and attempting to transfer this information between devices in as small a size as possible such that it images of the scene can be sampled on the new device in as high a quality as possible.  There are other ways of describing a 3D scene then with multiple images. The experiment I really want to see here is if this method is better then if the explicit scene information is compressed and rendered on the second device. If you can provide this in new experiments I will be happy to raise my score. Examples of this would be compressing  a 3D mesh, or surface defined by voxels with lighting parameters and textures.  My main concern here is that for simple scenes I am very sure that compressing or even transferring directly the 3D scene explicitly is a far superior, though there is probably some level of scene complexity at which your method overtakes it. I think it is important to establish this, and understand where your method is applicable. ", "extra_scores": {}, "extra_text": {}}, {"review_id": "_oyaIfM066f", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "**Paper Summary**:\n\nCompressing 3D scene is an interesting problem to explore. The paper proposes to add entropy penalized reparametrization (Oktay et al. (2020)) technique into Nerf (Mildenhall et al. (2020)) and compress the neural network in Nerf. Experiments also show some compressing rates improvement with the proposed baseline, which I have some concerns in the weakness below.  \n\n**Strength**:\n1. The problem this paper is attacking is interesting, compressing 3D scene plays an important role in real-time applications. \n2. The paper also showed the approach to compress multiple scenes in one network (though I have some concerns for the experiments in weakness below).\n\n**Weakness**:\n1. Lack of novelty. The main technique from this paper is merging two methods together (Oktay et al. (2020), and Mildenhall et al. (2020)),  with some improvements by extending the nerf to multiple scenes. However, the multiple scenes experiments are not performed well to demonstrate the effectiveness of the proposed method (see below), so the overall novelty is not enough. \n\n2. Experiments are not great enough to show the effectiveness of the full pipeline. \n\n   a) In single scene experiment, I think the main reason why HEVC+LLFF has lower PSNR is that LLFF is interpolating multiple training views, and this is obvious and has already been shown in the original Nerf paper (Table 1, Fig 5 in  Mildenhall et al. (2020)). Therefore, this is not a valid experiment to show the pipeline in this paper is better in compression. A more valid baseline should be: receiver receives the training images compressed by HEVC -> decode the images -> receiver train a new Nerf on the decoded images -> run the trained model on the test set. \n\n  b) In multiple scenes experiment. The paper only showed the experiments on compressing two scenes, it's not clear how will the performance be and conclusion generalize to more scenes, e.g. train only one model on all 8 synthetic scenes together. \n\n3. The paper also didn't consider the efficiency problem, training a Nerf model takes a long time, and also the author did not report the running time when doing inference on a trained nerf model, and do not have a comparison with HEVC+LLFF baseline. In literature, Nerf model takes more than one day to converge, while HEVC is very fast for both encoding and decoding.\n\nOverall, considering the novelty and the experiment section in this paper, I don't think they are enough to reach the bar of ICLR, so I vote for rejection. \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "CYnYni84S-", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\n\nThis paper proposes to compress nerf models with entropy loss, where instead of directly training nerf model parameters, it trains a new function F which takes some compressed information and decodes to the nerf models. Then it did the same things as nerf, which render scenes in novel views. The authors show that the function F could largely compress the original nerf models while keeping similar PSNR.\n\nComments:\nThe paper combines network compression and neural renderings, which is pretty interesting. However, I have several concerns :\n\nNovelty:  The paper seems to combine two methods together, where in network compression, it adopts Oktay et al. (2020) while in neural renderings, it relies on nerf. Though it introduces compression to neural rendering, such a combination seems to be not very creative.\n\nGeneralization. While one can train such a network for nerf scenes, the network may get overfit to these scenes since the whole dataset has no more than 20 scenes.  It would be great to validate on shapenet datasets, or some dataset has at least 100 models.\n\nComaprison. The authors compare their methods with HEVC+LLFF method, which is pretty unfair. Since HEVS is a traditional video compression method while LIFF is also a traditional blending view rendering method. The authors are encouraged to compare with some neural network compression methods + neural rendering methods.\n\nConclusion: Overall, I think this paper proposes an interesting idea and shows good results. However, due to lack of creativity and unfair comparison, I rate it below the acceptance bar.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "5cE0tr60YSZ", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a method of compressing neural radience fields (NeRF's) by learning mappings from latent codes to model parameters such that both distortion/reconstruction quality and the rate get minimized. While maintaining the same level of quality, this method is able to compress NeRF models for more efficient sender-to-receiver transmission.\n\nThe strong aspects of this paper include:\n(1) it addresses the valid problem of compressing NeRF models, which is particularly valid given that one usually trains one NeRF per scene. Without compression, the storage of NeRF's will grow dramatically as more scenes are considered;\n(2) it achieves the same level of rendering quality, while compressing the model by 8-9x;\n(3) it might have implications to future work that attempts to render NeRF's in real time or under a sender-receiver setup.\n\nIn terms of drawbacks, this paper can be made stronger by addressing the following points:\n(1) while model sizes are compressed significantly at almost no cost of rendering quality, there is no improvement in rendering speed. In fact, because of the additional decompression, rendering novel views might very well take longer than the original NeRF. Since improving the NeRF rendering speed is vital to eventually achieving real-time rendering, I consider this a major drawback of this work;\n(2) another major drawback in my opinion is that the work does not demonstrate its usefulness under the multi-scene setup. As the original NeRF is per-scene, and people desire a multi-scene variant of NeRF, the work would be much more impactful if it provides significant compression while achieving the same quality in such setups. Unfortunately, Fig. 5 shows having separate NeRF's have higher rendering quality at comparable sizes as the compressed models;\n(3) it would be interesting to explain how this is related to meta-learning. One can imagine having a meta-NeRF that quickly adapts to different scenes. This also achieves similar compression effects, and may work better for the multi-scene setup. Maybe comparisons can be made to such meta-learning methods; and\n(4) the work heavily relies on prior works in the compression aspect, and it's unclear to me what the novelty is in that regard.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Description:
The paper presents a method for encoding a compressed version of an implicit 3D scene, from given images from arbitrary view points. This is achieved via a function, learning with a NeRF model, that maps spatial coordinates to a radiance vector field and is optimized for high compressibility and low reconstruction error. Results shows better compression, higher reconstruction quality and lower bitrates compared to other STOA.

Strengths:
- Method for significantly compressing NerF models, which is very useful since such models are often trained for every new scene
- Retain reconstruction quality after compression by an order of magnitude
 
Weaknesses:
- The need for decompressing the model before rendering can be done means reduced rendering speed. This also requires longer training times.
- Experiments against other scene compression + neural rendering technique will have further strengthened the papers’s claims 
- The techniques used are well established, and thus there is not as much technical novelty.

Decision: Reject