ROAD: Learning an Implicit Recursive Octree Auto-Decoder to Efficiently Encode 3D Shapes
Authors: Sergey Zakharov
Rares Andrei Ambrus
Katherine Liu
Adrien Gaidon
Venue: CoRL 2022 Poster (2022)
Keywords: Implicit shape representations
Reconstruction
Data compression

Abstract: Compact and accurate representations of 3D shapes are central to many perception and robotics tasks. State-of-the-art learning-based methods can reconstruct single objects but scale poorly to large datasets. We present a novel recursive implicit representation to efficiently and accurately encode large datasets of complex 3D shapes by recursively traversing an implicit octree in latent space. Our implicit Recursive Octree Auto-Decoder (ROAD) learns a hierarchically structured latent space enabling state-of-the-art reconstruction results at a compression ratio above 99%. We also propose an efficient curriculum learning scheme that naturally exploits the coarse-to-fine properties of the underlying octree spatial representation. We explore the scaling law relating latent space dimension, dataset size, and reconstruction accuracy, showing that increasing the latent space dimension is enough to scale to large shape datasets. Finally, we show that our learned latent space encodes a coarse-to-fine hierarchical structure yielding reusable latents across different levels of details, and we provide qualitative evidence of generalization to novel shapes outside the training set.


ROAD:
Learning an Implicit Recursive Octree Auto-Decoder to Efficiently Encode 3D Shapes
Sergey Zakharov, Rareş Ambruş, Katherine Liu, Adrien Gaidon
Toyota Research Institute
Abstract:
Compact and accurate representations of 3D shapes are central to many pereep- tion and robotics tasks. State-of-the-ar learning-based methods can reconstruct single objects but scale poorly to large datasets. We present novel recursive im- plicit representation efficiently and accurately encode large datasets of complex 3D shapes by recursively traversing an implicit octree latent space. Our im- plicit Recursive Octree Auto-Decoder (ROAD) learns hierarchically structured latent space enabling sfate-of-the-al reconstruction results at a compression ratio above 99%. We also propose an efficient curriculum learning scheme that natu- rally exploits the coarse-to- to-feee properties of the underlying octree spatial repre- sentation. We explore the scaling law relating latent space dimension, dataset size, and reconstruction accuracy, showing that increasing the latent space dimension is enough to scale to large shape datasets. Finally, show that learned la- tent space encodes a coarse-to-fie across different levels of details, and we provide qualitative evidence of general- ization to novel shapes outside the training set.
hierarchical structure yielding reusable latents
Keywords: Implicit shape representations, Reconstruction, Data compression
Introduction
Accurately and efficiently representing 3D geometry is comeestone capability in computer vi- sion and computer graphics, with many practical applications in robotics and artificial intelligence. Decades of research in this area have produced a myriad of approaches, from traditional explicit methods [1, 2, 3, 4, 5, 6, to learning-bass based implicit representations that encode shapes in the weights of neural networks and use various learning cues such as Signed Distance Fields [8, 9, 10], occupancy [11, 5], or radiance [12, 13, 14, 15]. Choosing one representation over another typically involves various tradeoffs between accuracy, scalability and generalization [16. 17, 18). Related methods have either focused on modeling single shapes with increasing levels of accuracy at higher costs in terms of memory or time [12, 19, 20, 21] or on modeling classes of shapes, typically with single MLPs, which results in the ability to generalize to novel shapes as well as adapt to test-time data via differentiability but at the expense of high high-frrquency details [8, 22, 23, 24].
In this paper we address these key challenges through a novel neural network capable of simultane- ously encoding large number of shapes to a higher level of accuracy than previously possible. We build on recent advances which use neural fields (i.e. neural network parameterizations of contin- uous functions defined on Euclidean space) to represent any topology with arbitrary precision [18] combined with continuous, generative latent spaces for 3D shape generation [8, 25]. We aim to recover compact function which learns to map complex topologies to an implicit space of shapes, without sacrificing reconstruction accuracy or differentiability. Our key insight based on the fact that the neural field paradigm maps spatial coordinates to encoding of the surface and thus re- quires thousands of evaluations to extract the underlying surface; moreover, the underlying implicit function needs to accurately model space outside the target geometry [26, 27, 28]. Alternatively, some methods explicitly define the relationship between an underlying explicit data structure and the latent space [22, 20], thus partitioning the implicit function.
6th Conference on Robot Learning (CoRI. 2022), Auckland, New Zealand.

---

AccuCities
Thingi
ShapeNet
Google Scanned Objects
Our implicit Recursive Octree Auto-Decoder ROAD, can represent many diverse shapes accurately and precisely with small memory footprint thanks to recursive decoding of hierarchi- cally structured learned implicit shape representation.
Figure I:
To achieve high compression while still being able reconstruct high-free frequencyynyyyy details, we propose an implicit Recursive Octree Auto-Decoder (ROAD) formulation that operates entirely in the latent space and guided by an octree partitioning of the space. The octree data structure provides simple yet elegant solution for increasing surface detail while traversing down the tree [29, 20, 22], as well an intuitive setup for curriculum learning schedule where learning progresses according to coarse-to- to-nee approach. Our method uses single neural network to map latent vector to eight other latent vectors corresponding to its eight children in 3-dimensional space; in turn, each of the predicted latent vectors can be fed back to the network for further subdivision. To extract surface, we simply traverse down the tree starting from a single root latent vector, expanding nodes needed based predicted occupancy, until the desired level of resolution is reached. The output of a forward pass of our network is the actual surface, and can be obtained milliseconds, unlike seconds minutes for related methods requiring complex operations.
Our formulation leads to latent space that captures shape similarity and hierarchy in manner that is conducive to high compression well generalization to shapes outside of the training set. Our efficient implementation leads to a reduction of up to 99% terms of storage space compared the original mesh models; this includes network weights as well as any other learned features needed for reconstructing the meshes. Finally, we explore the relationship between 3D modeling power (as measured by the Chamfer distance), latent space dimension and dataset size, and show how to tune our model's 's capacity with a single hyperparameter - the dimension of the latent space.
Our contributions are summarized as follows:
A novel implicit representation parameterized by a recursive function that encodes an arbitrary number of 3D shapes in shared latent space, while retaining high reconstruction fidelity and requiring up to 99% less storage space compared to the input mesh models;
A curriculum learning method that naturally exploits the octree spatial data structure through coarse-to-fime optimization scheme;
An analysis of the scaling law that correlates latent space dimension, dataset size and 3D recon- struction accuracy and qualitative analysis of the learned latent space indicating a coarse-to- fine hierarchical structure resulting in reusable latents.

---

2 Related Work
Neural Fields are continuous coordinate-basse neural networks that encode underlying prop- erty of scene. The popularity of these representations has increased dramatically as recent results have shown that with enough modeling power coordinate-based networks can be used to encode underlying physical quantities with arbitrary levels of precision [18, 8, 12]. Applications of these techniques include modeling 3D shape [11, 5, 8. 9]. appearance / radiance [12, 13, 14, 15], geome- try [30]. semantic information [31], material properties [32, 33], human shape and appearance [34], and robotics [35, 36, 37, 38, 39, 40J. Neural Fields have been used in robotics to represent 3D ge- ometry and appearance with applications in grasping [41, 42], trajectory planning [36], object pose estimation and refinement [10, 43, 24, 44], object and surface reconstruction from sparse and noisy data [5, 45], modti perception [46], localization [47] and SLAM [48, 49]. For an overview of recent methods and applications please consult [18].
Neural Fields for Shape Representation. These methods represent shapes in the weights of the neural networks, and vary depending on the underlying signal used encode the 3D space, c.g. occupancy [11, 5], Signed Distance Functions [8. 9, 10], density and radiance [12, 15]. A second distinction comes from the target domain, with some methods overfitting to a single scene/object [12, 20, 21], and other methods learning a generalizable prior over entire categories of shapes, e.g. as generalizable latent space of Signed Distance Functions [8, 9, 10], as a con- volutional prior over grid cells [11, 5] or image pixels [13]. as weights of kernel learned from data [50, 23, 51], as an object centric shape [24] and/or appearance prior [52]. The modeling power of these methods can be further improved by modulating the input coordinates with period func- tions [53, 54], while rendering speed, training time and networks size can be improved by fac- torizing the scene tensor into multiple low-rank components [55], by utilizing multiple small size MLPs [56]. by training on Sparse Voxel Fields [57] or via multiresolution hash input encoding [21]. A number of methods employ an octree datastructure to guide learning towards occupied areas of space [29, 20, 22]. Closest to our method, NGLOD [20] also uses an octree to adaptively fit shapes to multiple eevels-of-Detail (LoDs), however unlike [20] our method can represent multiple shapes. Additionally, thanks to our recursive scheme, we only need to store root level latents and not the entire grid as is the case for [20]. Recursive parameterizations have also been employed in the con- text of radiance fields [58] or for 3D shape representation [22]. Unlike [22], we use a lightweight decoder-only recursive architecture represent high quality shapes dense oriented point clouds through which we can extract the encoded surface in real-time Its capacity can be easily extended by simply modifying the size of the input latent vector and it is capable of storing additional attributes (such as material information) at minimal cost.
Differentiable Rendering refers the ability to render image and back- sock-propagate training signal from the image back to the underlying representation; for an overview of latest methods please refer to [59, 60]. This allows 3D representations of scenes to be learned using only 2D supervision [61], generative models of objects [62], compositionality [63, 64, 65], learning from data in the wild [66. 67, 68] or test-time adaptation [10]. However, in the context of 3D shape representation, extracting the underlying surface from the implicit field typically involves expensive operations such as volume rendering [26], sphere tracing [27] or ray marching [28]. Our method maintains differentiabilty with respect to the input and can thus be optimized given partial 3D data as well as 2D images and additionally we output the underlying surface by design, foregoing the need for expensive operations for surface extraction.
3 Methodology
Preliminaries. Our approach takes as input a set of shapes S = {Si..... SK } and learns a space of implicit surfaces that represents the input. Each shape Sx is represented by an oriented point cloud consisting of points {D.e7 13 131 [Sxl and associated normals {n.T e | where |Skl
denotes the number of points in Sx-
Formulation. Our method represents each shape with a D-dimensional latent vector denoting the root node of an octree which is traversed recursively, up to predefined Level-of- (LoD) M. We seek to regress three functions to recursively reconstruct 3D geometry, parameterized by neural networks: Φ RRD RD&D for latent subdivision; 1h RDD -> R for mapping the latent space to
3

---

LoD1
LoD2
LoD3
ROND
Row
Our method extracts object surfaces by performing octree traversal. Starting from la- tent vector the parent cell ROAD extracts latent vectors for all children cells together with their occupancy, local SDF and surface normal estimates. It efficiently extends to large datasets while retaining high surface reconstruction quality.
Figure 2:
surface geometry (occupancy, signed distance to surface, surface normal); and & IR? + R for zero iso-surface projection. The hyperparameter M, the dimension of the latent space, is linked to the capacity of our representation as a function of the dataset size. Given a latent vector zm E R.D, where 0 < < M denotes the LoD of the latent vector, we define:
<(=")
{,, mm+++1 J
(1)
Ji=1
as the function that performs traversal of the latent space given input latent 2th and outputs latent vector 2, m++ [e) for each of the 8 possible children, where [Ø] denotes the ith output of d.
By definition, projects back to the latent space of dimension D, i.c. 2;n+1 E RD, forcing the resulting latent to encode both high and low level information. This formulation allows ROAD to simply pass back the resulting latent propagating coarser, higher-level information along the latent space. A latent 2 at the lowest LoD therefore encodes the geometry of entire object. Unlike other setree-based methods [20] that store all the octree latents, our formulation allows us to record only the root level latents for each input shape, i.c. - = { k= We provide ablations for other formulations in Section 4.
recursive function, i.e. through d(zm+1), while
We employ db to map any latent vector to underlying surface geometry as follows:
(1=²") to" 8 n"
The output of &i consists of Tm E (0 1)- - the occupancy estimate denoting whether to continue the constrained signed distance value from the center
expanding this cell further, 8 (-1,1) of the cell to the surface of the object; and n e R - the surface normal vector. To extract the surface information at particular LoD m starting from a root latent z0 we perform a tree traversal as follows:
Vl (o(="))(())) fo sh n
(3)
times
Note that for clarity we omitted the notation [.] in Eq.3, however after each subdivision the appropriate child la- tent is selected, according to the desired branch of the octree be expanded. We highlight that unlike other SDF based methods that implicitly encode surfaces [8], our method does not take as input Euclidean coordinates, and the mapping between different LoD levels is achieved entirely via the learned latents zm - m which are forced to encode both local structure well as global shape information. Moreover, unlike grid-based repre- sentations [11, 5] we fully exploit the sparse nature of the underlying octree representation, using the occupancy o; to recursively expand only occupied cells.
LoD4
LoD6
Figure 3: ROAD enables surface points be directly extracted different lev- els of detail.

---

Figure 5: Latent space structure. use principle component analysis to visualize the encoded geometries of the ShapeNet1 dataset in two dimensions. Object color is related to object instance, with objects of the same class having similar colors, and carried through each LoD. For visualiza- tion purposes, grey objects in LoD 0 are not propagated to the higher LoDs. Similar latent vectors encode similar geometries resulting in a clear class separation at LoD O. Similar areas of the pro- jected latent space become increasingly shared by the different classes at higher LoDs, suggesting that our approach efficiently encodes object geometry by learning common geometrie primitives.
Shape Recovery. To recover the final shape, we perform the gero-isosurface projection (see Fig. 4), using distance 8t, surface normal estimates n4, and voxel cell center co- ordinates 2; at the desired LoD. The voxel centers are determined and tracked whenever we subdivide a cell. To extract the object surface LoD m, use the follow- ing equation: p." = [(x)" s rx?")
= x aa" sT,
where " is value that scales s In our experiments we use a scaling factor equal to the voxel size at LoD m,
Figure Zero-isosurface projection.
i.. a = 2/(m).. Once the projection is performed for all the query points at particular LoD, we a dense surface point cloud that is differentiable back to the imput latent vector z This property allows us perform optimization complete partial shapes based on the prior encapsulated in the network. An overview of our complete pipeline is provided in Algorithm
Architecture and Training. We parameterize the functions é, te with a single MLP with parameters 0, choosing SIREN-based [69] network periodic activation functions have been shown to be more capable at representing fine details. Our MLP uses a single layer encoder and multiple 2- layer decoder heads output occupancy SDF s, surface normals n. All hidden layers are 512- dimensional. We train our method by optimizing both the latent vectors and the parameters 0 of the MLP using the Adam solver with learning rate of 6 10
Algorithm 1:
Octree-based recursive surface extraction
Input: M maximum recursion depth, " {=0} object latent vector
Output: p E TR3 surface points,
/* Recursively subdivide voxels until desired LoD reached
for m E [I....,M} do
2m
0
for
x mm
in 2 1- do
m
Ji ROAD(2
// recursive subdivision
Z m t 2
1f 0}:
// add occupied latents
end
7 end
/* Extract object shape for th t do
P (
10 end
return
5

---

ShapeNet150
Thingi32
Method
Storage (MB) (1)
gloU (f)
Chamfer (.)
Storage (MB) (1)
gloU (1)
Chamfer (L)
DeepSDF
1052.6
86.9
0.316
224.6
96.8
0.053
FFN [54]
301.6
88.5
0.077
64.3
97.7
0.033
SIREN [69]
151.3
78.4
0.381
32.3
95.1
0.077
Neural Implicits [74]
4.4
82.2
0.500
0.9
96.0
0.092
NGLOD [20]
185.4
91.7
0,062
39.6
99.4
0.027
Ours LoD6
86.3
0.175
96.4
0.138
Ours LoD7
94.2
0.067
98.4
0.045
Ours LoD8
3.8
94.9
0.041
3.2
98.7
0.022
Ours LoD9
94.9
0.036
98.7
0.017
Table 1: Shape Reconstruction. This table shows reconstruction and compression comparisons against two datasets. As opposed the baselines, our method trains a single model for the entire dataset, while still outperforming them in terms of reconstruction quality.
To supervise training, we define losses for each of the decoder levels at every LoD. Occupancy loss Lo binary cross entropy, whereas SDF L. and surface normals L, losses minimize the l2 distance between respective predictions and ground truth values. The final loss formulated as:
L
Σ MEM
woDo + ug L + Wn
where w. wg is function returning an inverse voxel radius for level m, and w, 0.1.
Curriculum Learning. When it comes to training on large datasets or on datasets with high res- olution models requiring high LoDs. the vanilla training procedure requires much more time to converge as opposed to training on simpler datasets. To alleviate this problem we introduce cur- riculum training procedure. Instead of initiating training from the desired final LoD, we change the first LoD to be lower (we use LoD 3 in our experiments) and keep track of the mean of predicted occupancy confidences at given level m. Lower LoDs are faster train on due a smaller number of latents to optimize. Moreover, it makes it faster to sample random training points at each iteration, thus further accelerating the training procedure. To compute our confidence score, we first take softmax over two occupancy values for all estimated voxels at a given LoD, and then take a max value between all the pairs. our average confidence 6 is high enough in either occupancy or non occupancy. jump to the next LoD and repeat this procedure until the final LoD is reached. In our experiments, we show that this simple technique helps to accelerate the training process especially when training large and high resolution datasets. We set 0 = 0.95 in experiments.
4 Experiments
To demonstrate the 3D reconstruction and compression capabilities of our approach we run a num- ber of experiments on the ShapeNet [70], Thingi10K [71], Google Scanned Objects [72], and the AccuCities [73] datasets. We report reconstruction metrics as measured by the Chamfer distance (multiplied by 103) well as interesection over union over points uniformly sampled in the bound- ing volume of the ground truth shape.
Reconstruction. We follow the protocol of [20] and train on a subset of 150 objects from ShapeNet [70] and subset of 32 objects from Thingi10K [71]. We report results Table 1. We note that the baselines compare against train one model for each shape in the dataset (i.e. [20] trains 32 networks for each object in Thingi32). Owing the efficient recur- sive decoding scheme implemented by our method, we can train single model for each dataset and still be competitive in terms of network size and recon- struction accuracy. We set the latent size to for Thingi32 and to 96 for ShapeNet150 respectively. Indeed, our network achieves state-of-the-ar recon- struction results of 0.036 Chamfer distance and an
0.35
0.34
Qunltr dibare
0.12
0.3
0.08
0.06
0.04
0.52
32
128
256
532
1030
Figure 6: Chamfer distance vs data quantity for different latent vector sizes.

---

Query
Nearest Neighbors
Query
Neighbors
Local
local
LoD 2
LoD
Olobal Coneett
Glebal Contect
Figure 7: Nearest neighbors. We visualize the geometry encoded by example latents, as well as the geometry encoded by the nearest neighbors within the same LoD across all objects in the Thingi32 dataset, determined by the Euclidean distance metric on the latent vector space (top row). We color the points the octree cell corresponding the latent, showing that our approach enables latents with similar shapes to be used at different global positions (bottom row).
IoU of more than 94% on ShapeNet150 with a network of size 3.8MB: this amounts to a com- pression of more than 99%, as the original mesh dataset measures 630MB. We report similar compression ratio on Thingi32 (473MB) while achieving state-of-the-ar reconstruction accuracy of 0.017 Chamfer distance and a competitive IoU of 98.7%.
We further explore the reconstruction capabilities of our method by encoding a model from the Ac- cuCities [73] dataset: neighborhood from London consisting of 1.9 million triangles and requiring 252MB of disk space. We set the latent size D to 512. Qualitative results are shown in Fig. with additional images in the supplementary. Quantitatively
achieve a Chamfer distance of 0.04 when comparing against the ground truth model, while using a network of size 11 MB.
Scaling to larger datasets. For this experiment we use the entire Google Scanned Objects [72] dataset consisting of a total of 1030 object models. We introduce training splits of different sizes (32, 128, 256, 512, 1030) study the relation between the dataset complexity and the latent vector size, novel hyperparameter specific to our formulation. We restrict our analysis into the scaling properties of our network to the dimension of the latent space, and mention that other methods that are generally applicable to machine learning models (i.e. number of layers, training schedule, etc.) can used further tweak the performance of our method. Our results are recorded in Fig, 6: for each split train networks with increasing latent vector sizes (64, 128. 256, 512) and record the resulting Chamfer distance by comparing the reconstructed models with the ground truth models of that specific split. We note strong correlation between dataset complexity and latent size. Specifically, our results indicate that our method can efficiently scale to an increasing number of shapes by only modifying the latent vector size while keeping the network parameterization intact.
Latent Space Analysis. To qualitatively analyze the properties of the learned latent space, we project the latent space specific LoDs into two dimensions via principle component analysis to visualize the encoded geometries of the ShapeNet dataset, as shown in Fig. 5. We observe that objects of the same class are spa- tially close in the projected space at the top level LoD, demonstrating that similar latent vectors en- code similar geometries. Furthermore, at higher LoDs, similar areas of the projected latent space are increasingly shared by the different classes, suggest- ing that our approach efficiently encodes object ge- ometry by learning geometric primitives common in the dataset.
33
B32
6/8
110
We also visualize the nearest neighbors of specific latent vectors from the network trained on Thingi32 to LoD 9 in Fig. 7. We show that similar latent vec- tors can represent the local geometry different ob- ject coordinates, without explicit positional encod- ing. Increasing the LoD also intuitively reduces the geometric complexity represented by the latent, as seen by the edge feature from LoD and the ori- ented patch feature from LoD 4.
Figure 8: Generalization. We demonstrate our method's increasing generalization abil- ity when trained on many objects.
7

---

Superposition
Storage (MB3)
gloU
Chamier
Surface density
Low
Medium
High
Direct
3.2
98.7
0.017
Sphere tracing
min 0.1 s
min 0.9% s
10 min
Additionn
3.2
96.9
0.039
Marching cubes Ours
6s
Concutenation
17.9
99.4
0.013
11 ms
13
17 ms
Table 2: Latent Fusion. Here we compare dif- ferent ways to propagate latent vectors to the next LoDs.
Table 3: Inference time. Our method extracts object surfaces in real time, significantly outper- forming the state of the art.
Generalization. In this experiment we demonstrate the generalization capabilities of our method. We take four networks from the data compression experiment each trained on a different split of the Google Scanned Objects dataset (32, 128, 256, and 512 objects) and optimize latent vectors to fit unseen models from the same as well as other datasets (Thingi32, ShapeNet150) while keeping network weights frozen. The results are shown in Fig. 8. Reconstruction quality plotted in terms of Chamfer distance shows increasing generalization capability for networks trained more models.
Surface Extraction. By design, our method differentiably extracts object surfaces in real time with minimal memory overheads. In comparison, NGLOD [20] or other SDF-based [8] methods require either an expensive sphere ray tracing or -ineeefferet marching cubes to extract surface. Ta- ble 3 compares inference times when extracting the object surface at different levels of density. For our method we perform inference up to LoDs 6, 7 and 8 respectively, which corresponds to approxi- mately 20000, 80000 and 300000 surface points, respectively for the Thingi32 models. We compare against marching cubes baseline from [8] and sphere tracing baseline from [20], and we iterate until the desired number of surface points is sampled. For a fair comparison with our method we extract 20000, 80000 and 300000 using both sphere tracing and marching cubes. We note that method extracts the object surface up to orders of magnitude faster that the sphere tracing base- line; our method requires total of 1-2 seconds to extract object surfaces for the entire ShapeNet150 dataset. The marching cubes baseline, while faster than sphere tracing, is still not real-time capable and not differentiable. Finally, observe that the results for this experiment were obtained single A6000 GPU, without any optimization.
Latent Vector Fusion. As described in Section and in Fig. 2, our method propagates information through the latent space via the recursive function & (see Eq. Here we explore different forms of latent subdivision: addition and concatenation, with the complete definitions provided in the supplementary. While addition does not change the dimension of the latent vector D, it explicitly defines how information propagated from parent child latent (i.e. via addition). Conversely, concatenation makes D increase with each recursion level, and information is directly copied as we traverse the latent space. This introduces significant modifications to the underlying neural network architecture, requiring specialized networks at each LoD. The results of this ablative analysis are summarized in Table As expected, contcatenation serves as an upper bound for performance and it achieves the highest performance but requires 5x more storage space, and more complicated formulation. Although similar in formuation to direct regression, the addition version of our method achieves poor results, which we attribute to the artificial constraint imposed on how information is propagated through the latent space.
5 Discussion
Limitations and Future Work. Our representation currently only supports 3D geometry. For future work, would like to explore its extension to other modalities (object color and material properties) as well as representations (images and radiance fields [12]). Another interesting direction that could be explored is combination of our pipeline with different downstream tasks (object detection and pose estimation). Our representaton fully differentiable and thus allows the propagation of useful 3D) gradients for shape optimization given partial information.
We presented novel recursive implicit representation to effectively represent and compress 3D geometry by framing as the traversal of implicit octree in a learned latent space. extracts geometry in real-time and scales to large datasets while retaining high reconstruction quality. As result, we outperform state-of tfite--f-thh-eee the- reconstruction results on the ShapeNet1 and Thingi32 datasets, even when compared to methods training single network per model. Our analysis of the representation explores the structure of the latent space and presents scaling law defining a relationship between latent space dimension, dataset size and reconstruction accuracy.
Conclusion.
8

---

References
[1] R. B. Rusu and S. Cousins. 3d is here: Point cloud library (pcl). In ICRA, 2011.
[2] C. R. Qi, H. Su, K. Mo, and L. J. Guibas. PointNet: Deep learning on point sets for 3d classification and segmentation. In CVPR, 2017.
[3] B. Curless and M. Levoy. volumetric method for building complex models from range images. In SIGGRAPH, 1996.
[4] A. Hornung, K. M. Wurm, M. Bennewitz, C. Stachniss, and W. Burgard. Octomap: An efficient probabilistic 3d mapping framework based on octrees. Autonomous robots, 2013.
[5] S. Peng, M. Niemeyer, L. Mescheder, M. Pollefeys, and A. Geiger. Convolutional occupancy networks. In ECCV, 2020.
[6] Whelan, Leutenegger, R. Salas-Moreno, B. Glocker, and A. Davison. Elasticfusion: Dense slam without pose graph. Robotics: Science and Systems, 2015.
[7] R. Hanocka, A. Hertz, N. Fish, R. Giryes, S. Fleishman, and D. Cohen-Or. Meshenn: network with an edge. TOG, 2019.
[8] J. J. Park, P. Florence, J. Straub, R. Newcombe, and S. Lovegrove. Deepsdf: Learning contin- uous signed distance functions for shape representation. In CVPR, 2019.
[9] V. Sitzmann, E. Chan, Tucker, N. Snavely, and G. Wetzstein. Metasdf: Meta-learning signed distance functions. NeurlPS, 2020.
[10] S. Zakharov, W. Kehl, A. Bhargava, and A. Gaidon. Autolabeling 3d objects with differentiable rendering of sdf shape priors. In CVPR, 2020.
[11] L. Mescheder, M. Oechsle, M. Niemeyer, S. Nowozin, and A. Geiger. Occupancy networks: Learning 3d reconstruction in function space. In CVPR, 2019.
[12] B. Mildenhall, Representing scenes as neural radiance fields for view synthesis. In ECCV, 2020.
Srinivasan, M. Tancik, T. Barron, R. Ramamoorthi, and Ng. Nerf:
[13] A. Yu, V. Ye, M. Tancik, and A. Kanazawa. pixelnerf: Neural radiance fields from one few images. In CVPR, 2021.
[14] V. Sitzmann, S. Rezchikov, B. Freeman, J. Tenenbaum, and F. Durand. Light field networks: Neural scene representations with single-evaluation rendering. NeurIPS, 2021.
[15] T. Barron, Mildenhall, M. Tancik, Hedman, R. Martin Bartin-Brualla, and P. Srinivasan. Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields. In ICCV, 2021.
[16] J. W. Tangelder and R. C. Veltkamp. A survey of content based 3d shape retrieval methods. Multimedia tools and applications, 2008
[17] E. Ahmed, A. Saint, A. E. R. Shabayek, K. Cherenkova, R. Das, G. Gusev, D. Aouada, and B. Ottersten. A survey on deep learning advances on different 3d data representations. arXiv, 2018.
[18] Y. Xie, T. Takikawa, S. Saito, O. Litany, S. Yan, N. Khan, F. Tombari, J. Tompkin, V. Sitzmann, and S. Sridhar. Neural fields in visual computing and beyond. arXiv, 2021.
[19] W. Yifan, L. Rahmann, and O. Sorkine-Homung. Geometry-consisten neural shape represen- tation with implicit displacement fields. arXiv, 2021.
[20] Т. Takikawa, Litalien, K. Yin. K. Kreis, C. Loop, D. Nowrouzezahraii Jacobson, M. McGuire, and S. Fidler. Neural geometric level of detail: Real-time rendering with im- plicit 3d shapes. In CVPR, 2021.
[21] Müller, A. Evans, C. Schied, and Keller. Instant neural graphics primitives with mul- tiresolution hash encoding. arXiv, 2022.

---

[22] J.-H. Tang, W. Chen, J. Yang, B. Wang, S. Liu, B. Yang, and L. Gao. Octfiel:: Hierarchical implicit functions for 3d modeling. arXiv, 2021.
[23] Williams, Z. Gojcic, S. Khamis, D. Zorin, J. Bruna, S. Fidler, and O. Litany, Neural fields as learnable kernels for 3d reconstruction. arXiv, 2021.
[24] S. Zakharov, R. A. Ambrus, V. C. Guizilini, D. Park, W. Kehl, F. Durand, J. B. Tenenbaum, V. Sitzmann, J. Wu, and A. Gaidon. Single-shot scene reconstruction. In CoRL, 2021.
[25] Z. Chen and H. Zhang, Learning implicit fields for generative shape modeling. In CVPR, 2019.
[26] S. Lombardi, T. Simon, J. Saragih, G. Schwartz, A. Lehrmann, and Y. Sheikh. Neural volumes: Learning dynamic renderable volumes from images. arXiv, 2019.
[27] S. Liu, Y. Zhang, S. Peng, B. Shi, M. Pollefeys, and Z. Cui. Dist: Rendering deep implicit signed distance function with differentiable sphere tracing. In CVPR, 2020.
[28] M. Niemeyer, L. Mescheder, M. Oechsle, and A. Geiger. Differentiable volumetric rendering: Learning implicit 3d representations without 3d supervision. In CVPR, 2020.
[29] A. Yu, R. Li, M. Tancik, H. Li, R. Ng. and A. Kanazawa. Plenoctrees for real-time rendering of neural radiance fields. In ICCV, 2021.
[30] G. Yang, S. Belongie, B. Hariharan, and Koltun. Geometry processing with neural fields. NeurIPS, 2021.
[31] A. Kundu, K. Genova, X. Yin, A. Fathi, C. Pantofaru, L. Guibas, A. Tagliasacchi, F. Dellaert, and T. Funkhouser. Panoptic neural fields: A semantic object-aware neural scene representa- tion. arXiv, 2022.
[32] M. Boss, R. Braun, Jampani, J. T. Barron, C. Liu, and H. Lensch. Nerd: Neural reflectance decomposition from image collections. In ICCV, 2021.
[33] S. Hadadan, S. Chen, and M. Zwicker. Neural radiosity. TOG, 2021.
[34] W. Jiang. K. M. Yi, G. Samei, O. Tuzel, and A. Ranjan. Neuman: Neural human radiance field from a single video. arXiv, 2022.
[35] J. Ortiz, A. Clegg, J. Dong, E. Sucar, D. Novotny, M. Zollhoefer, and M. Mukadam. isdf: Real-time neural signed distance fields for robot perception. arXiv, 2022.
[36] M. Adamkiewicz, T. Chen, A. Caccavale, R. Gardner, P. Culbertson, J. Bohg, and M. Schwa- ger. Vision-only robot navigation in neural radiance world. RA-L, 2022.
[37] L. Yen-Chen. P. Florence, J. T. Barron, T.-Y. Lin, A. Rodriguez, and P. Isola. Nerf-supervision: Learning dense object descriptors from neural radiance fields. arXiv, 2022.
[38] L. Yen-Chen, P. Florence, J. T. Barron, A. Rodriguez, P. Isola, and T.-Y. Lin. inerf: Inverting neural radiance fields for pose estimation. In IROS, 2021.
[39] K. Chitta, A. Prakash, and A. Geiger. Neat: Neural attention fields for end-to-end autonomous driving. In ICCV, 2021.
[40] A. Simeonov, Y. Du, A. Tagliasacchi, J. B. Tenenbaum, A. Rodriguez, P. Agrawal, and V. Sitz- mann. Neural descriptor fields: Se (3)-equivariant object representations for manipulation. In ICRA, 2022.
[41] M. Breyer, J. J. Chung, L. Ott, S. Roland, and N. Volumetric grasping network: Real-time 6 dof grasp detection in clutter. In CoRL, 2020.
[42] J. Ichnowski, Y. Avigal, J. Kerr, and K. Goldberg. Dex-nerf: Using a neural radiance field to grasp transparent objects. In CoRL, 2021.
[43] M. Irshad, S. Zakharov, R. Ambrus, Kollar, Z. Kira, and Gaidon. Shapo: Implicit representations for multi-object shape, appearance, and pose optimization. In ECCV, 2022.
10

---

[44] L. Huang, T. Hodan, L. Ma, L. Zhang, L. Tran, C. Twigg, P.-C. Wu, J. Yuan. C. Keskin, and R. Wang. Neural correspondence field for object pose estimation. ECCV, 2022.
[45] Williams, Z. Gojcic, S. Khamis, D. Zorin, J. Bruna, S. Fidler, and O. Litany, Neural fields as learnable kernels for 3d reconstruction. In CVPR, 2022.
[46] R. Gao, Y.-Y. Chang. S. Mall, L. Fei-Fei, and J. Wu. Objectfolder: A dataset of objects with implicit visual, auditory, and tactile representations. In CoRL, 2021.
[47] A. Moreau, N. Piasco, D. Tsishkou, B. Stanciulescu, and A. de La Fortelle. Lens: Localization enhanced by nerf synthesis. CoRL, 2022.
[48] E. Sucar, S. Liu, J. Ortiz, and A. Davison. imap: Implicit mapping and positioning in real- time. In ICCV, 2021.
[49] Z. Zhu, S. Peng, V. Larsson, W. Xu, H. Bao, Z. Cui, M. R. Oswald, and M. Pollefeys. Nice- slam: Neural implicit scalable encoding for slam. In CVPR, 2022.
[50] F. Williams, M. Trager, J. Bruna, and D. Zorin. Neural splines: Fitting 3d surfaces with infinitely-wide neural networks. In CVPR, 2021.
[51] H.-T. D. Liu, Williams, A. Jacobson, S. Fidler, and O. Litany. Learning smooth neural functions via lipschitz regularization. arXiv, 2022.
[52] W. Jang and L. Agapito. Codenerf: Disentangled neural radiance fields for object categories. In ICCV, 2021.
[53] V. Sitzmann, J. Martel, A. Bergman, D. Lindell, and G. Wetzstein. Implicit neural representa- tions with periodic activation functions. NeurlPS, 2020.
[54] M. Tancik, Srinivasan, Mildenhall, S. Fridovich-Kad Keil, N. Raghavan, U. Singhal, R. Ra- mamoorthi, J. Barron, and R. Ng. Fourier features let networks learn high frequency functions in low dimensional domains, NeurlPS, 2020.
[55] A. Chen, Z. Xu, A. Geiger, J. Yu, and H. Su. Tensorf: Tensorial radiance fields. arXiv, 2022.
[56] C. Reiser, S. Peng. Y. Liao, and A. Geiger. Kilonerf: Speeding up neural radiance fields with thousands of tiny mips. In ICCV, 2021.
[57] L. Liu, Gu, K. Z. Lin, T.-S. Chua, and Theobalt. Neural sparse voxel fields. NeurIPS, 2020.
[58] G.-W. Yang, W.Y.. Zhou, H.-Y. Peng, D. Liang. T.-J. Mu, and S.-M. Hu. Recursive-nerf: An efficient and dynamically growing nerf. arXiv, 2021.
[59] H. Kato, D. Beker, M. Morariu, T. Ando, T. Matsuoka, W. Kchl, and A. Gaidon, Diffcrentiable rendering: A survey. arXiv, 2020.
[60] A. Tewari, J. Thies, B. Mildenhall, P. Srinivasan, E. Tretschk, Y. Wang, C. Lassner, V. Sitz- mann, R. Martin-Brualla, S. Lombardi, et al. Advances in neural rendering. arXiv, 2021.
[61] V. Sitzmann, M. Zollhoefer, and G. Wetzstein. Scene representation networks: Continuous Rd-struutture-waree neural scene representations. NewrlPS, 2019.
[62] K. Mustikovela, S. De Mello, A. Prakash, U. Iqbal, S. Liu, Nguyen-Phuoc, C. Rother, and J. Kautz. Self-supervised object detection via generative image synthesis. In ICCV, 2021.
[63] M. Niemeyer and A. Geiger. Giraffe: Representing scenes as compositional generative neural feature fields. In CVPR, 2021.
[64] J. Ost, F. Mannan, N. Thuerey, J. Knodt, and F. Heide. Neural scene graphs for dynamic scenes. In CVPR, 2021.
[65] B. Yang, Zhang, Y. Xu, Y. Li, H. Zhou, H. Bao, G. Zhang, and Z. Cui. Learning object- compositional neural radiance field for editable scene rendering. In ICCV, 2021.
11

---

[66] P. Henzler, J. Reizenstein, P. Labatut, R. Shapovalov, T. Ritschel, A. Vedaldi, and D. Novotny. Unsupervised learning of 3d object categories from videos in the wild. CVPR, 2021.
[67] N. Müller, A. Simonelli, L. Porzi, R. Bulò, M. NieBner, and P. Kontschieder. Autorf: Learn- ing 3d object radiance fields from single view observations. arXiv, 2022.
[68] J. Reizenstein, R. Shapovalov, P. Henzler, L. Shordone, P. Labatut, and D. Novotny. Common objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction. In ICCV, 2021.
[69] V. Sitzmann, J. N. Martel, A. W. Bergman, Lindell, and G. Wetzstein. Implicit neural representations with periodic activation functions. In Proc. NeurIPS, 2020.
[70] A. X. Chang, T. Funkhouser. L. Guibas, P. Hanrahan, Q. Huang, Z. Li. S. Savarese, M. Savva, Song, Su, et al. ShapeNet: An information-rich 3d model repository. arXiv, 2015.
[71] Q Zhou and A. Jacobson. Thingi dataset of 10,000 3d-printing models. arXiv, 2016.
[72] L. Downs, A. Francis, N. Koenig, B. Kinman, R. Hickman, K. Reymann, B. McHugh, and
V. Vanhoucke. Google scanned objects: A high-quality dataset of 3d scanned household items. arXiv, 2022.
[73] 3d model of london and city models. tttp:://ww.eccuciiess.coo// Accessed: 2022-05-
18.
[74] Davies, D. Nowrouzezahrai, and A. Jacobson. Overfit neural networks as compact shape representation. 2020.
12


Reviews:
[{"review_id": "tynSlnov8Uq", "reviewer": "Reviewer_dXT1", "summary": "This paper proposes an efficient implicit representation to encode large scale datasets in real time: implicit recursive octree auto-decoder iROAD. The idea is directly learning latent code for each object in a hierarchy manner. During the inference, given the initial latent code $\\mathbf{z}_0$, we can call iROAD recursively to get latent code for different level of details (LoD). To speed up training, this paper introduces a curriculum procedure to first train low LoD and gradually jump to higher LoD until the desired/final LoD reached. Experiments on synthetic datasets (ShapeNet and Thingi32) show that the proposed method outperforms existing approaches (e.g., DeepSDF, NGLOD) with a small memory footprint.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n\n[S1] The paper is well motivated - compact and accurate representation of 3D shapes are always critial for robotics tasks. However, only a few approaches are proposed (NGLOD) to handle this problem. The idea of iROAD is simple yet very effective -> better than more complicated NGLOD.\n\n[S2] The performance is great - iROAD outperforms to methods training a single network per model significantly. This inspires us that jointly training will encode more data priors and achieve superior performance. I like the generalization experiment which shows that the latent space is able to generalize on unseen objects (seen class).\n\n**Weakness:**\n\n[W1] Training time comparison for iROAD and other baselines (DeepSDF, NGLOD) for encoding the whole dataset. I am concerned that training auto-decoder in hierarchy way would significantly increase the computation load. More concise comparison with baselines especially NGLOD would be highly appreciated.\n\n[W2] Quanlitative results?  It would be great to have more visual comparisons with existing approaches and see the difference in recovered surface (vs DeepSDF or NGLOD).  That would make the results more convincing.\n\n[W3] More experiments on the robustness of iROAD would be more interesting. Currently, all results are conducted on synthetic datasets with dense oriented points. In practice, the points might be sparse and contain noise, it would be very interesting to see whether the joint learning can help overcome the noise and sparisty issue with priors from other objects.\n\n**Others:** \n- why not reporting the storage for other LoDs in Table 1. Could we show the results of NGLOD with different LoD level?\n- In Table 3, it would be great to be more explicit - showing the performance of NGLOD and iROAD for each level (low, medium and high) and the inference time.\n- Good to have: Training time vs number  of objects\n- Include more implementation details for baselines ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The overall idea for this paper is simple but the performance is great. It outperform existing SOTA approach NGLOD and also achieves a large margin than some methods training a single network per model. The inference can be conducted in real time and the model storage is very efficient.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Relevant but unlikely to deploy to hardware in near future", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "issues": "See weakness. Primarly more comparison with baselines on the training time.", "reviewer_expertise": "3: The reviewer is fairly confident that the evaluation is correct"}, "extra_text": {}}, {"review_id": "ZJiNQ-hmfcx", "reviewer": "Reviewer_SYPp", "summary": "The paper proposes an octree-based implicit representation of 3D shapes. Explicit representations of such data (point clouds, meshes) present a high memory footprint that increases with resolution, while the proposed approach represents a shape with a single latent vector and a decoding neural network able to generalize to multiple shapes, resulting in a very efficient compression ratio while preserving a high accuracy.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n* Quantitative results show a clear improvement over prior work, considering the trade off between reconstruction accuracy, storage requirements and inference time. The high compression ratio, coupled with a fast decoding time make the contribution a very appealing solution for embedded robotics downstream tasks.\n* The proposed method does not simply overfit a neural network on each 3D shape but can be trained with multiple complex shapes simultaneously. When the training dataset is sufficiently diverse, the approach exhibits good generalization properties to unseen shapes, showing that the network does not only memorize training shapes but learns to combine geometric primitives at different level of details. This claim is supported by several convincing experiments.\n* The recursive decoding scheme based on an octree data structure is novel and relevant: it enables to limit the number of network parameters because decoding functions are shared between different resolutions. At inference time, the user can choose the desired level of detail, depending on throughput and accuracy requirements.\n\n**Weaknesses:**\n* Details are missing concerning how ground truth labels (occupancy, signed distance function and normal vectors of each cell) are defined. The paper states that input shapes are represented as an oriented point cloud (L112), but datasets used in experiments seems to provide meshes (L204). For the sake of reproducibility, I think that the data format of the input dataset should be clarified. While the preprocessing from point cloud to surface might be done with classical approaches, more details would be useful for an uninformed reader.\n* No experiments are conducted on downstream robotics tasks (stated in the limitations section). The differentiability of the resulting surface w.r.t the input latent vector is an interesting property which might enable to learn the shapes from weaker (2D) supervision signals.  This is suggested in the related work section (L106), but not supported by empirical experiments.\n\n**Questions/Suggestions:**\n* As a conclusion of the proposed experiments, I understand that the decoding network embeds a mapping from latent keys to the geometrical primitives present in the training set, whereas the input vector encodes these latent keys for different resolutions in a compact way. As a result, a new shape can be integrated in the framework without modifying the network weights (\"generalization\" experiment), as long as all geometric primitives present in the shape have been observed in other shapes during training. Is that correct ?\n* One drawback of implicit representations is the offline training time. How long training takes to fit a single shape ? A large dataset ? How fast is it to optimize only the input vector of a new shape in a trained network, compared with retraining everything ?\n* I would expect that simple augmentations during training (random 3D rotations and crops of the shapes) should help to improve the generalization ability of the decoding network.\n* Do you think that the proposed method could be applied on 2D signals using a quadtree data structure ?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "I recommend \"Strong Accept\" for this paper. I believe that the contribution is quite strong, with a clear writing and thorough experiments. I believe that the paper is suitable for publication in its current form, even if some minor details should be added for reproducibility.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "3: The reviewer is fairly confident that the evaluation is correct"}, "extra_text": {"issues": "As explained above, I believe that details are lacking concerning the input data: oriented 3D point cloud, meshes ? In particular, more explanation about how SDF labels are computed from a sparse point cloud would make the reproducibility easier.\n\nI am quite confused about the \"curriculum learning\" description. L178 \"Instead of initiating from the desired final LoD, we change the first LoD to be lower\". Does it mean that only the LoD at lower resolutions are trained in the first place, and then finer details are introduced ? L182 \"Once we reach a desired probability threshold theta, ...\" This notation is used for network weights earlier in the paper and it is not clear to me which probability is mentioned. I think the writing of this section can be improved.\n\nI am curious about answers and comments about my remarks in the Questions/Suggestions section, however I am not asking for additional experiments concerning them during the rebuttal."}}, {"review_id": "OjPJLxk5pW0", "reviewer": "Reviewer_XRB2", "summary": "The paper proposes a hierarchical, recursive latent space to learn an object shape representation.\nStarting from a root latent vector, a neural network recursively maps this latent vector into 8 more latent vectors refining the initial one. This way, the authors achieve higher surface reconstruction capabilities while keeping computation and memory requirements low.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\nThe paper is well written and I enjoyed reading it.\nThe idea, to my knowledge, of having a hierarchical latent space for representing an object shape is novel.\n\nWeaknesses:\nHowever, I have two concerns:\n1) The biggest concern about this paper is the relation to robotics. I have no idea why this was submitted to CoRL.\n2) The root latent vector has to contain all information about the object. Does this not counteract the idea of having an hierarchical representation of an object?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "Unfortunately, there is no relation to robotics in this paper. I have chosen weak accept, as I generally think the paper deserves publication, but I highly doubt that CoRL is the right venue for this work.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Irrelevant to robotics", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "2: Unsure, difficult to assess.", "issues": "Please clarify why it is sufficient to have a single root latent vector to represent fine details and why this is not possible with a monolithic model.", "reviewer_expertise": "4: The reviewer is confident but not absolutely certain that the evaluation is correct"}, "extra_text": {}}, {"review_id": "7w3SYkNTnLC", "reviewer": "Reviewer_NizH", "summary": "This paper presents a highly-efficient data structure for encoding multiple 3D shapes in a low dimensional latent space. The key idea is to use a hybrid neural network / octree structure to store signed distance function (SDF) values. The latent vector defines both the Octree structure and, via a small MLP, the SDF values. The results show that the method exhibits excellent compression ratios and reconstruction accuracy. ", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n=========\n- ”Neural fields” models are known to be quite limited in terms of the scale of scenes they can represent. Leveraging Octrees therefore has great potential to get neural fields models to scale to large scenes, but a significant challenge lies in how to get the Octree structure to generalize across shapes. This paper presents a solution to this problem by encoding the Octree shape with the latent code.\n- The architecture of the model is well designed and elegant. I especially like how the latent code recursively defines the Octree structure via splitting, much like you would construct a traditional octree. \n- The paper is very clearly written and nicely illustrated. A lot of effort has been put into communicating and explaining the idea.\n\nWeaknesses\n==========\n- There is not much discussion in the paper of the tradeoffs or weaknesses of the approach. How long does recursive decoding of the Octree take, for example?\n- The genrealization experiments are quite limited. How many sample points of the target object are used during the optimization and how is this affected by sampling fewer points? Can the latent vector be predicted from an image?\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "Overall, this is a good paper with a strong contribution in terms of a new hybrid neural+Octree data structure that can have applications in many areas in robotics and computer vision.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Relevant but unlikely to deploy to hardware in near future", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "issues": "Please address the questions in weaknesses above.", "reviewer_expertise": "4: The reviewer is confident but not absolutely certain that the evaluation is correct"}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)