DENSE RGB SLAM WITH NEURAL IMPLICIT MAPS
Authors: Heng Li
Xiaodong Gu
Weihao Yuan
luwei yang
Zilong Dong
Ping Tan
Venue: ICLR 2023 poster (2023)
Keywords: dense RGB SLAM
implict funciton
RGB VO

Abstract: There is an emerging trend of using neural implicit functions for map representation in Simultaneous Localization and Mapping (SLAM). Some pioneer works have achieved encouraging results on RGB-D SLAM. In this paper, we present a dense RGB SLAM method with neural implicit map representation. To reach this challenging goal without depth input, we introduce a hierarchical feature volume to facilitate the implicit map decoder. This design effectively fuses shape cues across different scales to facilitate map reconstruction. Our method simultaneously solves the camera motion and the neural implicit map by matching the rendered and input video frames. To facilitate optimization, we further propose a photometric warping loss in the spirit of multi-view stereo to better constrain the camera pose and scene geometry. We evaluate our method on commonly used benchmarks and compare it with modern RGB and RGB-D SLAM systems. Our method achieves favorable results than previous methods and even surpasses some recent RGB-D SLAM methods.The code is at poptree.github.io/DIM-SLAM/.

Published as a conference paper at ICLR 2023
DENSE RGB SLAM WITH NEURAL IMPLICIT MAPS
Heng Lii23+, Xiaodong Gu Weihao Yuan?, Luwei Yang', Zilong Dong', Ping Tan 123 'Hong Kong University of Science and Technology.
2 Allibaba Group, Simon Fraser University
1h. heng. ii@connect.. ust. hk, Juweiy8sfu. ca, pingtaness........m. .hk hkk fgianmu. ywh, dadong. gxd, list. d21 (ett.dd2]aalbbbaale aalllaaabbbbaiieeeeeaaaeea com
ABSTRACT
There an emerging trend of using neural implicit functions for map represen- tation in Simultaneous Localization and Mapping (SLAM). Some pioneer works have achieved encouraging results on RGB-D SLAM. this paper, we present a dense RGB SLAM method with neural implicit map representation. To reach this challenging goal without depth input, we introduce hierarchical feature vol- ume to facilitate the implicit map decoder. This design effectively fuses shape cues across different scales to facilitate map reconstruction. Our method simul- taneously solves the camera motion and the neural implicit map by matching the rendered and input video frames. To facilitate optimization, we further propose a photometric warping loss in the spirit of multi-view stereo to better constrain the camera pose and scene geometry. We evaluate our method on commonly used benchmarks and compare it with modern RGB and RGB-D SLAM systems. Our method achieves favorable results than previous methods and even surpasses some recent RGB-D SLAM methods. The code is at poptree gwt.gb.io/DD--SSA.....
INTRODUCTION
Visual SLAM fundamental task in 3D computer vision with many applications in AR/VR and robotics. The goal of visual SLAM is to estimate the camera poses and build a 3D map of the environment simultaneously from visual inputs. Visual SLAM methods can be primarily divided into sparse or dense according to their reconstructed 3D maps. Sparse methods (Mur-Artal & Tardós, 2017 Engel et al. 2017) focus on recovering camera motion with a set sparse or semi-dense 3D points. Dense works (Newcombe et al. (201 seek to recover the depth of every observed pixel and are often more desirable for many downstream applications such as occlusion in AR/VR or obstacle detection in robotics. Earlier methods (Newcombe et al. 2011 la Whelan et al. 2012) often resort to RGB-D cameras for dense map reconstruction. However, RGB-D cameras are more suitable to indoor scenes and
expensive because of the specialized sensors.
Another important problem in visual SLAM is map representation. Sparse SLAM methods (Mur- 2017] typically use point clouds for map representation, while 201 a) usually adopt triangle meshes. As observed in many
Artal & Tardós 2017 Engel et al.
dense methods (Necombe et al.] recent geometry processing works (Mescheder et al. 2019 Park al. 2019 Chen & Zhang 2019) neural implicit function offers a promising presentation for 3D data processing. The pioneer work, iMAP (Sucar al., 2021), introduces an implicit map representation for dense visual SLAMM This map representation is more compact, continuous, and allowing for prediction of unobserved areas, which could potentially benefit applications like path planning (Shrestha et al. 2019) and object ma- nipulation (Sucar al.) | (022)) However, observed in NICE-SLAM (Zhu et al. (0022), iMAP (Su-) car et al. 2021) is limited to room-scale scenes due to the restricted representation power MLPs. NICE-SLAM (Zhu et al. 2022) introduces hierarchical feature volume to facilitate the map recon- struction and generalize the implicit map to larger scenes. However, both iMAP (Sucar et al.) 2021) and NICE-SIAM (Zhu al. 2022) limited RGB-I cameras.
This paper presents a novel dense visual SLAM method with regular RGB cameras based on the implicit map representation. We also adopt hierarchical feature volume like NICE-SLAM to deal
"qqull contribution

---

Published as a conference paper at ICLR 2023
with larger scenes. But our formulation is more suitable for visual SLAM. Firstly, the decoders in NICE-SLAM (Zhu et al. 2022) are pretrained, which might cause problems when generalizing to different scenes (Yang et al. 202I, while method learns the scene features and decoders together on the fly to avoid generalization problem Secondly, NICE-SLAM (Zhu et al. 2022) com- putes the occupancy each point from features at different scales respectively and then sums these occupancies together, while we fuse features from all scales to compute the occupancy at once. In this way, our optimization becomes much faster and thus can afford to use more pixels and itera- tions, enabling our framework to work on the RGB setting. In experiments, we find the number of feature hierarchy important to enhance the system accuracy and robustness. Intuitively, features from fine volumes capture geometry details, while features from coarse volumes enforce geometry regularity like smoothness or planarity. While NICE-SLAM (Zhu et al. 2022) only optimizes two feature volumes with voxel sizes of 32cm and 16cm, our method solves six feature volumes from Scm to 64cm. Our fusion of features across many different scales leads to more robust and accurate tracking and mapping as demonstrated in experiments.
Another challenge in our setting is that there are no input depth observations. Therefore, we design sophisticated warping loss to further constrain the camera motion and scene map in the same spirit of multi-view stereo (Zheng et al., 2014] Newcombe et al.] [011 Wang et al.) (021b] Yu al. 2021). Specifically, we warp one frame to other nearby frames according the estimated scene map and camera poses and optimize the solution to minimize the warping loss. However, this warping loss subject to view-dependent intensity changes such as specular reflections. To address this problem, we carefully sample image pixels visible in multiple video frames and evaluate the structural similarity of their surrounding patches to build a robust system.
We perform extensive evaluations on three different datasets and achieve state-of-the-art perfor- mance on both mapping and camera tracking. Our method even surpasses recent RGB-D based methods like iMAP (Sucar et al. (02TT) and NICE-SLANE (Zhu et al. (022) on camera tracking.
Our contributions can be summarized as the following:
We design the first dense RGB SLAM with neural implicit map representation,
We introduce a hierarchical feature volume for better occupancy evaluation and a multiscale patch- based warping loss to boost system performance with only RGB inputs,
We achieve strong results on benchmark datasets and even surpass some recent RGB-D methods.
2 RELATED WORK
Visual SLAM Many visual SLAM algorithms and systems have been developed the last two decades. We only quickly review some of the most relevant works, and more comprehensive sur- veys can be found at (Cadena et al., 2016 Macario Barros et al. 2022). Sparse visual SLAM algorithms (Klein & Murray |007] Mur-Arta Artal & Tardos (017) focus solving accurate camera poses and only recover a sparse set of 3D landmarks serving for camera tracking. Semi-dense meth- ods like LSDDSLAM (Engel al., 2014) and DSO (Engel al. 2017) achieve more robust tracking in textureless scenes by reconstructing the semi-dense pixels with strong image gradients. In com- parison, dense visual SLAM algorithms (Newcombe et al. (01 aim to solve the depth of every observed pixel, which is very challenging, especially in featureless regions.
In the past, dense visual SLAM is often solved with RGB-D sensors. KinectFusion (Newcombe et al. 201 Ia) and the follow-up works (Whelan et al. 2012 2021 Dai et al. 2017) register the input sequence of depth images through Truncated Signed Distance Functions (TSDFs) track camera motion and recover a clean scene model at the same time. Most recently, iMAP and NICE-SLAM, introduce neural implicit functions as the map presentation and achieve better scene completeness, especially for unobserved regions. All these methods are limited to RGB-D cameras.
More recently, deep neural networks are employed to solve dense depth from regular RGB cameras. Earlier methods (Zhou et al. 2017 Ummenhofer et al.) 2017 Zhou et al. 2018) directly regress camera ego-motion and scene depth from input images. Later, multi view geometry constraints are enforced in (Bloesch et al. 2018 Tang & Tan 2019 Teed & Deng 2020 Wei et al. 2020; Teed & Deng 2021) for better generalization and accuracy. But they all only recover depth maps instead of complete scene models. Our method also employs deep neural networks solve the challenging
2

---

Published as a conference paper at ICLR 2023
V. Volume Varieble
D
Randered
Warping
Global KayFrames Frames
Depth
Loss
Concelte
Ray Sampli
Boloct
keyframe
keyframe !!
Keyframes
Sampled F
c Render
inpui steamm
P
Camera Pose
Variable
I
Rendered
Photomeinc
Loss
Trame H-
Trame
Trame 11-2
Add
Figure I: Pipeline of our framework. Given a camera pose, our method sample the multi-scale features along view rays and pass the concatenated features through an MLP decoder to compute the depth and color at cach pixel. In this way, we can solve the camera motion and a 3D scene map (captured by the feature volume and MLP decoder) simultaneously by matching the rendered images with observed ones. We further include warping loss across different views to enhance the system performance.
dense visual SLAM problem. Unlike these previous methods that only recover depth map per frame, we follow iMAP (Sucar et al. 202T) and NICE-SLAM (Zhu al. 2022) compute a neural implicit function for map representation.
Neural Implicit Representations Neural implicit functions have been widely adopted for many different tasks in recent years. It has been introduced for 3D shape modeling (Mescheder et al,, al.,a.22.11 2019 Chen & Zhang 2019), novel view synthesis (Mildenhall al. 2020 202
2019
Park et al.,
ct al.
2020| Maalin-Balllllll et [021 Barron et al. 2021, |araa Fridovich- Keil and Alex Yu al.
2002
Müller et al. (2022). clothed human reconstruction (Saito et al. 2019 2020), scene reconstruc-
tion
Murez et al.
2020 Sun et al., 202I). and object reconstruction Yarrv al. 2020 Niemeyer
ct al.
2020; Wang et al.
2021a). etc. But all these works typically require precisely known camera
poses. There only few works (Jeong et al. 2021 Zhang et al.) [002 Lin et al.
2021 trying to
deal with uncalibrated cameras. Furthermore, all these methods require a long optimization process, and hence, are unsuitable for real-time applications like visual SLAMM.
Recently, the two pioneer works, iMAP (Sucar et al. 2021) and NICE-SLAM (Zhu al. 2022). both adopt neural implicit functions represent the scene map in visual SLAM. These works can solve camera poses and 3D scene maps in real-time by enforcing the rendered depth and color images to match with the input ones. However, they both require RGB-D sequences as input, which requires specialized sensors. In comparison, our visual SLAM method works with regular RGB cameras and solves camera poses and the 3D scene simultaneously real-time.
APPROACH
The overall pipeline of our framework shown in Figure [ Given an RGB video as input, our method aims to recover a 3D scene map and the camera motions simultaneously. We represent the scene map by a neural implicit function with a learnable multi-resolution feature volume. By sampling features from the volume grid along view rays and querying the sampled feature with the MLP decoder, we can render the depth and color of each pixel given the estimated camera parameters. Since this rendering process is differentiable, we can simultaneously optimize the neural implicit map as well as the camera poses by minimizing an objective function defined on photometric rendering loss and warping loss.
3.1 IMPLICIT MAP REPRESENTATION
This section introduces the formulation of our implicit map representation that combines a learnable multi-resolution volume encoding {Vi) and an MI.P decoder & for depth and color prediction.
Multi-resolution Volume Encoding Directly representing the scene map with MLPs, which maps a 3D point to its occupancy and color, confronts a forgetting problem because the MLP is globally updated for any frame (Sucar et al. 2021). To address this, equip the MLP with multi-resolution volumes [Vi]- which are updated locally on seen regions of each Trame(Sara Fridovich Keil and Alex Yu al.) 2022; Müller et al. 2022). The input point is encoded by the feature F sampled from the volumes {V), which could also explicitly store the geometric information. We adopt a combination of J. volumes, whose resolution arrange from the voxel size of Umar to the voxel size of
3

---

Published as a conference paper at ICLR 2023
Umin This hierarchical structure works better than single-scale volume because the gradient-based based camera optimization on high-resolution volume is susceptible to a suboptimal solution without a good initialization. In contrast, in coarse-to-fine architecture, the low-resolution volume could enforce the smoothness in 3D space in early registration, while the high-resolution volume could encode the shape details. To compute this encoding of given 31) point p (a, y. 2),, interpolate feature (p) at the point p by trilinear sampling. Then the 3D feature encoding F(p) is obtained by concatenating the features from all levels. Notably, the feature channel of each volume is set to 1. The small feature channel reduces memory consumption without losing performance, which is demonstrated in the ablation study.
first
Color and Depth Prediction The MLP decoder & consists of three hidden layers with feature channel of 32 and two heads that output the color cp and density Op as,
(1) However, extracting scene geometry from the computed density requires careful tuning of the den- sity threshold and leads to artifacts due to the ambiguity present in the density field. To address this problem, following (Oechsle et al. (021), a sigmoid function is added on the output of Op to regularize it to [0, 1], in which case fixed level set of 0.5 could be used to extract the mesh. Also, the ray termination probability of point p, is now computed as.
(op, Cp) = (FF())).
i-1
W: = Op. II 0p,)
j=0
During volume rendering, hierarchical sampling strategy similar NeuS (Wang et al.] 2021a) is adopted. We first uniformly sample 32 points on view ray and then subsequently conducts importance sampling of additional 16 points for times based on the previously estimated weights. N 96 points are sampled in total, For each point, we compute depth Î and color I as,
N
D D= 2.2 i- W.C,
(3)
where 2; and C; are the depth and color of the point p; along the viewing ray. Notice that since the rendering equation is differentiable, the camera pose, multi-resolution volume encoding, and the parameters of the MLP decoder could be optimized together from the gradient back-propagation.
3.2 JOINT OPTIMIZATION
In this section, we present the objective function for optimization. The implicit scene represen- tation and the camera poses are jointly optimized in set of frames W with corresponding poses P (P& [R,, tu., k E W). where R and t are the rotation and translation camera parameters respectively. The frame selection strategy is explained later in section 3.3
Photometric Rendering Loss To optimize the scene map and the camera poses, we compute the L1 loss between the rendered and observed color for a set of randomly sampled pixels M:
Leenter
1
Σ 11 - QEM
(4)
LM
Here, |M| denotes the number of sampled pixels. This rendering loss alone is insufficient to deter- mine the scene geometry and could lead to ambiguous solutions, where the rendered image matches the observation well but the depth estimation is completely wrong. To overcome this problem, introduce a photometric warping loss to further constrain the optimization problem.
Photometric Warping Loss We define the photometric warping loss in similar spirit multi-view stereo (Zheng et al., 2014) to enforce the geometry consistency. Let qı denotes a 2D pixel in frame k. we first lift it into 3D space and then project it to another frame as:
9 K,R," KR '
homo Dd + tx tz), homo
(5)
where q
home (u, v, 1) is the homogeneous coordinate of qu. and Dap denotes its estimated depth. We minimize the warping loss defined as the following,
Twarpingg
Σ Σ
[q.. Iq--ill1-
(6)
M
4

---

Published as a conference paper at ICLR 2023
However, minimizing the warping loss single pixel does not model the view riew-dependent intensity changes, e.g., specularities, and could create artifacts in the scene map. We thus define the warping loss over image patches instead of individual pixels to address this issue. For image patch N centered on pixel qe and with the size of 8 X 6, we assume all pixels on the patch have the same depth value as qı- Similar the case of a single pixel, we project all pixels in this patch to another frame and obtain No
Thus, we extend the pixel-wise warping loss to patch-wise as,
9h-41'
1 E
Lwarping
SSIM q. qu ..,;
N.G.
where Bq denotes a visibility mask which will be explained later, and S = {1, 5, 111 denotes the patch size. In practice, we select the structure similarity loss (SSIM) (Wang al. 2004) as it achieves better results than other robust functions, e.g. (Park et al. 2017). Note that this formulation degenerates pixel-wise when a patch of size is used. Further, this warping assumes front-parallel image patches, which might be improved with a perspective warping based on surface normals extracted from the scene map. Here, we choose the front-parallel warping for its better computation efficiency.
Visibility Mask We define a binary visibility mask g to filter pixels with low visibility. For any pixel 9и- we count the number of its visible frames by projecting it to all other frames in the set W. The projection counted as valid if the projected pixel is within the frame boundary. Only pixels with more than 5 valid projections is regarded as visible, whose visibility value is set to 1. Note that we do not model occlusion here for better efficiency.
Regularization Loss To deal with textureless regions, we enforce an edge-preserving regularization term to smooth the rendered depth 1 as,
E -
Lmmooth
(8)
keW
where vi, vD are the first-order gradients of the computed color and depth images, respectively. Thus, the final objective for mapping and tracking in single thread is obtained as,
arg min wwarpinggwwrrpin + Grender + 4ssoooth k.W
where ar denotes the weighting factors, and @ denotes the parameters of the implicit scene represen- tation, including the multi-resolution volumes and the MLP decoder.
3.3 SYSTEM
In this section, we introduce several crucial components to build our complete visual SLAM system.
Initialization The initialization is performed when a small set of frames (13 frames in all our exper- iments) are collected. The pose of the first frame is set to the identity matrix. Then the poses of the remaining frames are optimized from the objective function for N; iterations. After the initializa- tion, the first frame is added to the global keyframe set and fixed. The parameters of MLP decoders are also fixed after initialization.
Window Optimization During camera tracking, our method always maintains window of active frames (21 frames in all our experiments). The active frames include two different types: local window frames and global keyframes. For frame k, we regard the frames with index from k- to k + 5 as the local window frames. We further sample 10 keyframes from the global keyframe set. Specifically, we randomly draw 100 pixels on frame and project them to all keyframes using estimated camera poses P and depths D. We then evaluate the view overlap ratio by counting the percentage of the pixels projected inside the boundary of those keyframes. Afterward, we randomly select 10 keyframes from those with an overlap ratio over 70% to form the global keyframe set. Then, all these active frames are used to optimize the implicit scene representation and the poses for No iterations. After the window optimization, the oldest local frame new incoming frame of l 6 with constant velocity motion model is added to the set of local window frames. The camera poses of all global keyframes are fixed during tracking. Please refer to the appendix about windows optimization for mapping in two threads cases.
is removed, and a
Keyframe Selection We follow the simple keyframe selection mechanism similar to iMAP and NICE-SLAM, while more sophisticated method might be designed based on image retrieval and
5

---

Published as conference paper at ICLR 2023
0-0
0-1
0-2
0-3
0-4
r-0
r-1
r-2
Avg.
NICE-SLAM ORB- SLAM2(RGB)
1.02
3.91
1.76
6.10
11.9
4.37
2.76
3.54
4.4
0,43
0.30
12.2
0.39
11.4
0.30
0.42
0.25
3.21
DROID-SLAN SLAM
ATE RMSE[cm]J
0.25
0.45
0.32
0.44
0.37
0.35
0.33
0.28
0.35
Ours(two threads)
0.89
0.72
1.11
0.75
0.92
0.84
1.22
0.66
0.89
Ours(one thread)
0.67
0.37
0.36
0.33
0.36
0.48
0.78
0.35
0.46
Acc.[cm]
5.87
3.71
4.81
4.27
4.83
3.58
3.69
4.68
4.43
IMAP
Comp.[cm]4
6.11
5,26
5.65
5,45
6.59
5.06
4.87
5.51
5,56
Comp. Ratiol 5cm%)1
77.71
79.64
77.22
77.34
77.63
83,91
83,45
75.53
79.06
Acc.[em]L
2.26
2.50
3.82
3.50
2.77
2.73
2.58
2.65
2.85
NICE-SLAM
Comp.[cm]]
2.02
2.36
3.57
3.83
3.84
2.87
2.47
3.00
3.00
Comp. Ratio S 5cm%)1
94.93
92.61
85.20
82.98
86.14
90.93
92.80
89.07
89.33
Acc.[em]l
70.56
1.42
2.11
2.11
2.02
1.79
49.00
26.17
19.40
DI-Fuison
Comp.[cm]]
3.58
2.20
4.83
4.71
5.84
3.57
39.40
17.35
10.19
Comp. Ratiol 5cm%
87.17
91.85
80.13
79.94
80.21
87.77
32.01
45.61
72.96
Acc.[cm]
3.26
2.92
4.90
4.47
5.35
4.01
4.53
4.40
4.23
DROID-SLAM
Comp.[cm]]
4.22
5.54
7.97
7.77
6.80
7.10
5.72
9.43
6.82
Comp. Ratiol 5cm%
77.57
72.95
59.85
62.30
66.50
68.35
71.68
68.76
68.50
Acc./em|
2.81
2.33
5.02
5.61
4.83
3.61
4.12
6.06
4.29
Ours(two threads)
Comp.[cm]
2.71
3.25
6.63
6.33
6.11
5.97
5.31
7.43
5.46
Comp. Ratio S 5cm%)1
88.56
84.7
71.40
72.54
73.23
81.12
76.94
70.85
77.41
Acc.[em]L Comp.[em]|
2.60
2.02
4.50
5.43
4.57
3.68
3.64
5.84
4.03
Ours(one thread)
2.65
3.31
6.09
5.98
5.81
5.32
4.72
5.70
4.20
Comp. Ratiol 5cm%J1
89.57
84.9
75.3
73.48
76.62
82.2
80.4
74.44
79.6
Table 1: Quantitative results the Replica dataset. Here, 'o-x' and "r-x" denote office-x and room-x sequence respectively. The top part shows the results tracking, and lower part shows results of mapping. DROID-SLAN and our method work with the RGB inputs, while others are all hased on RGB-D data. The best results in RGB and RGB-D SLAMs are bolded respectively.
visual localization methods like (Arandjelovic et al. 2016 Brachmann et al. 2017). Specifically, new keyframe is added if the field of view changes substantially. This view change is measured by the mean square f of the optical flow between the last keyframe and the current frame. A new keyframe is added if Tef, where Tky- 10 pixels in all our experiments.
EXPERIMENTS
Datasets We evaluate our method on three widely used datasets: TUM RGB-D (Sturm et al., 2012a) EuRoC (Burri al., 2016), and Replica (Straub et al. 2019) dataset. The TMU RGB-D and EuRoC dataset are captured in small to medium-scale indoor environments. Each scene in the TUM dataset contains a video stream from handheld RGB-D camera associated with the ground-truth trajec- tory. The EuRoC dataset captures stereo images using a drone, and their ground-trute truth 6D poses provided with Vicon System. We only the monoeular setting in the EuRoC dataset. The Replica dataset contains high-quality 3D scene models. We use the camera trajectories provided by iMAP (Sucar al. 202I) render the input RGB sequences.
Implementation Details Our method is implemented with PyTorch (Paszke et al. 2017 and runs on server with two NVIDIA 2080Ti GPUs. Our single-thread implementation, i.e. (one thread), and two-thread implementation, i.e. ours (two thread), need one and two GPUs respectively. The voxel sizes of the multi-resolution volumes for ours (one thread) are set {64, 48, 32, 24, 16, 8),m, where Umaz = 64cm and Umin = Scm. For ours (two threads). we replace the last Sem volume with 16em volume. In all our experiments, we fix the iteration number of initialization N; to 1500, the number of sampling pixels [M| to 3000, the size of the window [W| to 21. The iteration number of window optimization for ours(one thread) N., is 100. In our two-thread version, we change N.,, to 20 for tracking. We follow NICE-SLAM and perform mapping once every 5 frames. Please refer to the Appendix A.5 for more details. We use the Adam (Kingma & Ba, 2014) optimizer to optimize both the implicit map representation and the camera poses, with learning rates of 0.001 and 0.01, respectively. The wwamingg Orenders ammoah is 0.5, 0.1, 0.01, respectively. We apply a post-optimization the end of the sequence to improve the quality of the mesh. Note similar optimization included

---

Published as a conference paper at ICLR 2023
DROID SLAM
iMAP
Oars
Ground Truth
office0
office2
Figure Visual comparison of reconstructed meshes Replica datasets. Our results are more complete and have sharper textures, which indicate more precise surface shapes. More examples are in the appendix.
Method
frl /dsk
fr2 fr2/xyz fr / of ffice
Mean
iMAP (Sucar et al. 2021)
4.9
2.0
5.8
4.2
iMAP Sucar et al. 2021
7.2
2.1
9.0
6.1
J-FusionnHuang et al. 2021
4.4
2.3
15.6
7.4
NICE-SLANE Zhu et al. 2022
2.7
1.8
3.0
2.5
BAD-SLAM Schöps al. 2019
1.7
1.1
1.7
1.5
Kintinuous Whelan et al. 2012
3.7
2.9
3.0
3.2
ORB-SLAMZ(RGBD)
1.6
0.4
1.0
1.0
ORB- SB--LLLM(RGGBB Mur-Artal & Tardós 2017
1.9
0.6
2.4
1.6
DROID-SI -L.M Teed & Deng 2021)
1.8
0.5
2.8
1.7
Ours (one thread)
2.0
0.6
2.3
1.6
Table 2: Camera tracking results the TUM-RGBD dataset. The upper part shows RGB-D SLAM methods, while the lower part shows RGB SLAMM methods. "x" denotes the version reproduced by NICF-SLAMM. The best results are bolded. metric unit is [cm].
iMAP (Suaar al. 202T) and NICE-SLAM (Zhu al. 2022). All metrics reported in this paper are an average of the results for runs.
4.1 EVALUATION OF MAPPING AND TRACKING
Mapping To compare the quality of the reconstructed scene maps, we first evaluate the RGB-D methods the Replica dataset, including NICE-SLAM (Zhu et al.) 2022). iMAP (Sucar et al. 2021), and DI-Fusion (Huang et al.) 2021). Note all these methods use additional depth image inputs. We further compare with the RGB-only method, DROID-SLAM (Teed & Deng 2021) where the TSDF-Fusion (Curless Levoy. (996) is employed to compute mesh model of the scene map with voxel grid resolution of 256 using the estimated depth maps.
We use three metrics to evaluate the recovered scene map on Replica with 200, 000 sampled points from both the ground-truth and the reconstructed mesh. (i) Accuracy (cm), the mean Fuclidean

---

Published as a conference paper at ICLR 2023
Method
v101
V1 0202
V103
V201
V202
V203
Tartan VO Wang et al. 2020)
0.389
0.622
0.433
0.749
1.152
0.680
SVO al. 2014)
0.070
0.210
X
0.110
0.110
1.080
(SO(r et al. 2017
0.089
0.107
0.903
0.044
0.132
1.152
DROID-VO [eed & Deng 2021
0.103
0.165
0.158
0.102
0.115
0.204
DPVO et al. 2022)
0.053
0.158
0.095
0.095
0.063
0.310
Ours (one thread)
0.068
0.079
X
0.053
0.178
X
Table 3: Camera Tracking Results on EuRoC. The hest results are bolded. The failure cases are marked as X. The metric unit is [m].
distance between sampled points from the reconstructed mesh to their nearest points on the GT mesh. (ii) Completion (cm), the mean Euclidean distance between sampled points from the GT mesh to their nearest points on the reconstructed mesh. (iii) Completion Ratio (%), the percentage of points in the reconstructed mesh whose nearest points the GT mesh is within cm. The unseen region is removed it is not inside any camera viewing frustum.
As shown in the lower part of Table our method outperforms iMAP and DROID-SLAM map- ping. Although our results are slightly inferior to those of NICE-SLAM and DI-Fusion, this is reasonable since they have additional depth images input. A visual comparison with iMAP and DROID-SLAM is provided in Figure 2 From the zoomed regions, we can tell that our method generates more complete and accurate models than iMAP and DROID-SLAM. Additional visual comparisons are provided in the appendix.
Tracking To evaluate the quality of camera tracking, we compare our method with many recent visual SLAM methods with RGB or RGB-D inputs. For a fair comparison, we use the official implementation of DROID-SLAM to produce its result and eite other results from NICE-SI.AM and DPVO (Teed et al. 2022). As for the evaluation metric, we adopt the RMSE of Absolute Trajectory Error (ATE RMSE) (Sturm al. 2012b). The estimated trajectory is aligned with the ground truth with scale correction before evaluation.
The upper part of Table T shows the tracking results on the Replica dataset. Our method clearly outperforms NICE-SLAM, though it has additional depth input. Our tracking performance is similar to DROID- SLAM on this dataset.
Table Table shows results on the TMU RGB-D - dataset. We report results the same three examples as iMAP. As shown in the table, our method outperforms all previous learning-based methods, in- cluding iMAP (Sucar et al. 2021), NICE-SLAM (Zhu et al. 2022). DI-Fusion (Huang al.) [021] and has comparable performance with robust conventional methods like BAD-SLAM (Schöps et al., 2019), Kintinuous (Whelan et al., 2012), and ORB-SLAME (Mur-Artal & Tardós 2017). Further- more, our method does not include global bundle adjustment or loop closure, which is common in the conventional SLAM system. Our method achieves reasonable tracking performance while enjoying the benefits of neural implicit map representation, which can produce watertight scene model or predict unobserved regions.
Table 3 shows experiment results on the EuRoC dataset. We compare the tracking results with sev- eral strong baselines including Tartan VO (Wang et al. 2020), SVO (Forster al.. 2014), DSO (Engel et al., 2017), DROID-VO VO (Teed & Deng 2021). and DPVO (Teed et al. (022). Our method gen- erates comparable results successful cases. Note there are two failure cases, where our method loses camera tracking due to the fast camera motion which leads to small view overlap between neighboring frames.
4.2 ABLATION STUDY
To inspect the effectiveness of the components in our framework, we perform the ablation studies on the office-O C sequence the Replica dataset.
Hierarchical Feature Volume To evaluate the effectiveness of our hierarchical feature volume, we experiment with different levels of hierarchies. We vary the number of hierarchies L and the number of feature channels C' to keep the total number of learnable parameters similar. Table 4 shows the results of different settings. Clearly, the system performs worse when there are less hierarchies of feature volumes, even though longer feature is learned to compromise it. Note the system fails
8

---

Published as a conference paper at ICLR 2023
(L,
Contigrations
w/o MLP
(6,1) (4,4)
(2,8)
(1,16)
(6,2)
(6,4)
(6,8)
(6,16)
MEAN
0.67
0.76
2.27
X
0.63
0.69
0.61
0.66
8.92
ATE RMSE(em))
STD
0.14
0.18
0.25
X
0.12
0.08
0.11
0.09
1.49
MEAN
2.60
3.39
6.78
X
2.86
2.35
2.96
2.48
11.43
Accuacry(em)/
STD
0.56
0.91
0.95
X
0.49 0.33
0.40
0.12
3.17
Table 4: Ablation study of different configurations of the feature volume. The parameters L and C are the numbers of hierarchies and the number of the feature channel, respectively. We test on office-0 for times and calculate the mean and standard deviation of the camera tracking error and map accuracy.
Method
Memory(MB)) FLOPs(x10>| Mapping(ms)) Tr Tracking(ms))
iMAP
1.02
443.91
448(1000, 10)
101 6)
NICE-SLAM
12.0( 16cm)
104.16
130(1000, 10)
4200, 6)
Ours (two threads)
9.1(16cm)
18.76
330(3000, 100)
72(3000, 20)
Ours (one thread)
29.1
18.76
335 100)
Table 5:
Memory, computation, and running time. We report the runtime in time (pixel, iter) format for fair comparison. Please refer to appendix for more information about our two-threads version.
to track camera motion at textureless regions when is set to This experiment demonstrates the effectiveness of fusing features from different scales to better capture scene geometry.
Number of Feature Channel We further vary the number of feature channel with fixed at 6. Table 4 shows similar system performance is achieved for C = 2, 4, 8, 16. Therefore, to keep our method simple, we select C in experiments to reduce computation cost.
MLP Decoder To study the impact of the MLP decoder, also experiment without the MLP decoder, where the 1-channel feature can be regarded as the occupancy directly. At a 3D point, we sum the occupancies from all hierarchies for the final result. The color is computed similarly with another 6-level volume of 3 channels. As shown in the right of Table the system performance drops significantly without the MLP decoder.
4.3 MEMORY AND COMPUTATION ANALYSIS
Table analyzes the memory and computation expenses. The memory cost is evaluated on the office-0 sequence of the Replica dataset. We use the same bounding box size of the feature volume in both NICE-SLAM and our method. iMAP is the most memory efficient with only 1.02MB map size. However, as discussed in NICE-SLAM, it cannot deal with larger scenes. Our one thread version, i.e. ours (one thread), recovers the scene map at higher resolution of 8cm and requires a larger map of 29.1 MB, while NICE-SIAM works at 16cm map resolution and takes only 12 MB map. Our two thread version, i.c. ours (two threads), uses the same scene map resolution of 16cm, and is more memory efficient with map size of 9.1 MB.
Table 5) also reports the number of FLOPs for computing the color and occupancy at a single 3D point. Our method requires much less FLOPS, benefiting from our short features and single shallow MLP decoder. Table 5 further reports the computation time of processing one input frame. We report the results time [pixel, iter) format for comparison. Without depth input, our method samples more pixels and takes more optimization iterations, while our method is still faster than iMAP. Our running time reported with one or two RTX 2080TT GPUs, while the running time of NICE-SLAM and iMAP are cited from their papers with an RTX 3090 GPU.
CONCLUSION
This paper introduces dense visual SLAM method working with regular RGB cameras with neural implicit map representation. The scene map is encoded with a hierarchical feature volume together with an MLP decoder, which are both optimized with the camera poses simultaneously through back-pron bock-propagation. To deal with regular RGB cameras, a sophisticated warping loss is designed to enforce multi-view geometry consistency during the optimization. The proposed method compares favorably with the widely used benchmark datasets, and even surpasses some recent methods with RGB-D inputs.

---

Published as a conference paper at ICLR 2023
REFERENCES
Relja Arandjelovic, Petr Gronat, Akihiko Torii, Tomas Pajdla, and Josef Sivic. Netvlad: Con ar- chitecture for weakly supervised place recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5297-5307, 2016.
Jonathan T. Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, and Pratul P. Srinivasan. Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields. In Proceedings of the International Conference on Computer Vision (ICCV), 2021.
Michael Bloesch, Jan Czarnowski, Ronald Clark. Stefan Leutenegger, and Andrew J. Davison. Codeslam learning a compact, optimisable representation for dense visual slam. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), 2018.
Eric Brachmann, Alexander Krull, Sebastian Nowozin, Jamie Shotton, Frank Michel, Stefan Gumhold, and Carsten Rother. Dsac-differentiable ransae for camera localization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6684-6692, 2017.
Michael Burri, Janosch Nikolic, Pascal Gohl, Thomas Schneider, Joern Rehder, Sammy Omari, Markus W Achtelik, and Roland Siegwart. The euroc micro acrial vehicle datasets. The Interna- tional Journal of Robotics Research. 2016. URL URL/attp://1/ sagepub. com/ content/ abstrrctt.ttttt..........
ear
Cesar Cadena, Luca Carlone, Henry Carrillo, Yasir Latif, Davide Scaramuzza, Jose Neira, lan Reid, and John J. Leonard. Past, present, and future of simultaneous localization and mapping: Toward the robust-peroeption age. IEEE Transactions on Robotics (TRO), 32: 1-33009-1332 2016.
Zhiqin Chen and Hao Zhang. Learning implicit fields for generative shape modeling. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), June 2019.
Brian Curless and Marc Levoy. A volumetric method for building complex models from range images. In Proceedings the 23rd Annual Conference Computer Graphics and Interactive Techniques, SIGGRAPH '96, pp. 103-312, 1996.
Angela Dai, Matthias Nießner, Michael Zollhofer, Shahram Izadi, and Christian Theobalt. Bundle- fusion: Real-time globally consistent 3d reconstruction using online surface re-integration. ACM Transactions on Graphics (TOG), 2017.
Jakob Engel, Thomas Schops, and Daniel Cremers. Lsd-slam: Large-scale direct monocular slam. In Proceedings of the European Conference on Computer Vision (ECCV), 2014.
Jakob Engel, Vladlen Koltun, and Daniel Cremers. Direct sparse odometry. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 40(3):611 625, 2017.
Christian Forster, Matia Pizzoli, and Davide Scaramuzza. Svo: Fast semi-dired monocular visual odometry. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 15-22. IEEE, 2014.
Jiahui Huang, Shi-Sheng Huang, Haoxuan Song, and Shi-Min Hu. Di-fusion: Online implicit 3d reconstruction with deep priors. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), 2021.
Yoonwoo Jeong, Seokjun Ahn, Christopher Choy, Anima Anandkumar, Minsu Cho, and Jaesik Park. Self-calibrating neural radiance fields. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5846-5854, 2021.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 11X.::142..6980, 2014.
Georg Klein and David Murray. Parallel tracking and mapping for small ar workspaces. In Proceed- ings of the International Symposium Mixed and Augmented Reality (ISMAR), 2007.
10

---

Published as a conference paper at ICLR 2023
Chen-Hsuan Lin, Wei-Chiu Ma, Antonio Torralba, and Simon Lucey. Barf: Bundle-adjusting neural radiance fields. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recog- nition (CVPR). pp. 5741-5751, 2021.
William Lorensen and Harvey E Cline. Marching cubes: A high resolution 3d surface construction algorithm. ACM siggraph computer graphics. 21(4):163-163 1987.
Andréa Macario Barros, Maugan Michel, Yoann Moline, Gwenolé Corre, and Frédérick Carrel. A comprehensive survey visual slam algorithms. Robotics, 11(1), 2022. ISSN 2218-6581 URL
htttps:/wwwi mmpi. com 2218 0tt.//222866681111111 6581/11/1/24
Ricardo Martin- Noha Radwan, Mehdi S. M. Sajjadi, Jonathan T. Barron, Alexey Dosovit- skiy, and Daniel Duckworth. Nerf in the wild: Neural radiance fields for unconstrained photo collections. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Occupancy networks: Learning 3d reconstruction in function space. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. In Proceedings of the European Conference on Computer Vision (ECCV), 2020.
Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics prim- itives with multiresolution hash encoding. ACM Trans. Graph., 41(4):102:1-002:15 July 2022. doi: 10.11455/5282233333117,
URL
3530127
Raúl Mur-Artal and Juan D. Tardós. Orb-slam2: An open-source slam system for monocular, stereo, and rgb-d cameras. IEEE Transactions on Robotics (TRO), 3365):2255-1262, 2017.
Zak Murez, Tarrence As, James Bartolozzi, Ayan Sinha, Vijay Badrinarayanan, and Andrew Rabinovich. Atlas: End-to-end 3d scene reconstruction from posed images. In Proceedings of the European Conference on Computer Vision (ECCV), 2020.
Richard A. Newcombe, Shahram Izadi, Otmar Hilliges, David Molyneaux, David Kim, and An- drew J. Davison. Kinectfusion: Real-time dense surface mapping and tracking. In Proceedings of the International Symposium on Mixed and Augmented Reality (ISMAR), 201
Richard A. Newcombe, Steven J. Lovegrove, and Andrew J. Davison. Dtam: Dense tracking and mapping in real-time. In Proceedings of the International Conference on Computer Vision (ICCV), 201
Michael Niemeyer, Lars Mescheder, Michael Oechsle, and Andreas Geiger. Differentiable volumet- ric rendering: Learning implicit 3d representations without 3d supervision. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), 2020.
Michael Oechsle, Songyou Peng, and Andreas Geiger. Unisurf: Unifying neural implicit surfaces and radiance fields for multi-view reconstruction. In Proceedings the on Computer Vision (ICCV), 2021.
the International Conference
Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. Deepsdff: Learning continuous signed distance functions for shape representation. In Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
Seonwook Park, Thomas Schöps, and Mare Pollefeys. Illmiiatioon change robustness in direct visual slam. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 4523-4530. IEEF, 2017.
Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017.
11

---

Published as a conference paper at ICLR 2023
Shunsuke Saito. Zeng Huang, Ryota Natsume, Shigeo Morishima, Angjoo Kanazawa, and Hao Li. Pifu: Pixel-aligned implicit function for high-resolution clothed human digitization. Proceed- ings of the International Conference Computer Vision (ICCV), 2019.
Shunsuke Saito, Tomas Simon, Jason Saragih, and Hanbyul Joo. Pifuhd: Multi-level pixel-aligned implicit function for high-resolution 3d human digitization. In Proceedings of the IEEE Confer- ence on Computer Vision and Pattern Recognition (CVPR), 2020.
Sara Fridovich-Kei Keil and Alex Yu, Matthew Tancik, Qinhong Chen, Benjamin Recht, and Angjoo Kanazawa. Plenoxels: Radiance fields without neural networks. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), 2022.
Thomas Schöps, Torsten Sattler, and Mare Pollefeys. Bad slam: Bundle adjusted direct rgb-d slam. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp- 134-144, 2019.
Rakesh Shrestha, Fei-Peng Tian, Wei Feng, Ping Tan, and Richard Vaughan. Learned map prediction for enhanced mobile robot exploration. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2019.
Rakesh Shrestha, Siqi Hu, Minghao Gou, Ziyuan Liu, and Ping Tan. real world dataset for multi- view 3d reconstruction. In Proceedings of the European Conference Computer Vision (ECCV), 2022.
Julian Straub, Thomas Whelan, Lingni Ma. Yufan Chen, Erik Wijmans, Simon Green, Jakob J. Engel, Raul Mur-Artal, Carl Ren, Shobhit Verma, Anton Clarkson, Mingfei Yan, Brian Budge, Yajie Yan, Xiaqing Pan, June Yon, Yuyang Zou, Kimberly Leon, Nigel Carter, Jesus Briales, Tyler Gillingham, Elias Mueggler, Luis Pesqueira, Manolis Savva, Dhruy Batra, Hauke M. Strasdat, Renzo De Nardi, Michael Goesele, Steven Lovegrove, and Richard Newcombe. The Replica dataset: A digital replica of indoor spaces. 2019.
Sturm, N. Engelhard, Endres, W. Burgard, and D. Cremers. A benchmark for the evaluation of rgb-d slam systems. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Oct. 2012a.
Jürgen Sturm, Nikolas Engelhard, Felix Endres, Wolfram Burgard, and Daniel Cremers. A bench- mark for the evaluation of rgb-d slam systems. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 573-580, 2012b.
Edgar Sucar, Kentaro Wada, and Andrew Davison. Nodeslam: Neural object descriptors for multi- view shape reconstruction. In Proceedings of the International Conference on 3D Vision (3DV), 2020.
Edgar Sucar, Shikun Liu, Joseph Ortiz, and Andrew Davison. iMAP: Implicit mapping and posi- tioning in real-time. In Proceedings of the International Conference on Computer Vision (ICCV), 2021.
Jiaming Sun, Yiming Xie, Linghao Chen, Xiaowei Zhou, and Hujun Bao. Neuralrecon: Real-time coherent 3d reconstruction from monocular video. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
Chengzhou Tang and Ping Tan. Ba-net: Dense bundle adjustment networks. In Proceedings of the International Conference on Learning Representations (ICLR), 2019.
Zachary Teed and Jia Deng. Deepv2d: Video to depth with differentiable structure from motion. In Proceedings of the International Conference on Learning Representations (ICLR), 2020.
Zachary Teed and Jia Deng. DROID-SLAM: Deep Visual SL.AM for Monocular, Stereo, and RGB- D Cameras. 2021.
Zachary Teed, Lahav Lipson, and Jia Deng. Deep patch visual odometry. arXiv preprint arXiv:2208. 04726, 2022.
12

---

Published as a conference paper at ICLR 2023
Benjamin Ummenhofer, Huizhong Zhou, Jonas Uhrig, Nikolaus Mayer, Eddy Ilg. Alexey Dosovit- skiy, and Thomas Brox. Demon: Depth and motion network for learning monocular stereo. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
Peng Wang. Lingji Liu, Yuan Liu, Christian Theobalt, Taku Komura, and Wenping Wang. Neus: Learning neural implicit surfaces by volume rendering for multi-viww reconstruction. Advances in Neural Information Processing Systems (NeurIPS), 34, 2021a.
Qianqian Wang, Zhicheng Wang, Kyle Genova, Pratul Srinivasan, Howard Zhou, Jonathan Bar- ron, Ricardo Martin-Brualla Brualla, Noah Snavely, and Thomas Funkhouser. Ibrnet: Learning multi- view image- based rendering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021b.
Wenshan Wang, Yaoyu Hu, and Sebastian Scherer. Tartanvo: A generalizable learning-basee vo, In Proceedings of the Conference on Robot Learning (CORL). 2020.
Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600-612, 2004. doi: 10.1 1..11///TP.00388198666.
Xingkui Wei, Yinda Zhang, Zhuwen Li, Yanwei Fu, and Xiangyang Xue. Deepsfm: Structure from motion via deep bundle adjustment. In Proceedings of the European Conference on Computer Vision (ECCV), 2020.
Whelan, J. McDonald, Michael Kaess, Maurice Fallon, Hordur Johannsson, and John J. Leonard. Kintinuous: Spatially extended kinectfusion. In RSS Workshop RGB-D: D: Advanced Reasoning with Depth Cameras, July 2012.
Thomas Whelan, Stefan Leutenegger, Renato F. Salas-Moreno, Ben Glocker, and Andrew J. Davi- son. Elasticfusion: Dense slam without a pose graph. In Proceedings of Robotics: Science and Systems (RSS), 2021.
Xingrui Yang, Hai Li, Hongjia Zhai, Yuhang Ming, Yuqian Liu, and Guofeng Zhang. Vox-Fusion: Dense tracking and mapping with voxel-based neural implicit representation. In Proceedings of the International Symposium Mixed and Augmented Reality (ISMAR), pp. 80 89, 2021.
Lior Yariv, Yoni Kasten, Dror Moran, Meirav Galun, Matan Atzmon, Basri Ronen, and Yaron Lip- man. Multiview neural surface reconstruction by disentangling geometry and appearance. Ad- vances in Neural Information Processing Systems (NeurIPS), 33, 2020.
Lin Yen-Chen, Pete Florence, Jonathan T. Barron, Alberto Rodriguez, Phillip Isola, and Tsung- Lin. iNeRF: Inverting neural radiance fields for pose estimation. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2021.
Alex Yu, Vickie Ye, Matthew Tancik, and Angjoo Kanazawa. PixeINeRF: Neural radiance fields from one or few images. In Proceedings the the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
Jiahui Zhang, Fangneng Zhan, Rongliang Wu, Yingchen Yu, Wenqing Zhang, Bai Song, Xiao- qin Zhang, and Shijian Lu. Vmrf: View matching neural radiance fields. arXiv preprint arXiv:2207.02621, 2022.
Kai Zhang, Gernot Riegler, Noah Snavely, and Vladlen Koltun. Nerf++: Analyzing and improving neural radiance fields. arXiv:2010.07492, 2020.
Enliang Zheng, Enrique Dunn, Vladimir Jojic, and Jan-Michael Frahm. Patch match based joint view selection and depth map estimation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1510-1517, 2014.
Huizhong Zhou, Benjamin Ummenhofer, and Thomas Brox. Deeptam: Deep tracking and mapping. In Proceedings of the European Conference Computer Vision (ECCV), 2018.
13

---

Published as a conference paper at ICLR 2023
Tinghui Zhou, Matthew Brown, Noah Snavely, and David Lowe. Unsupervised learning of depth and ego-motion from video. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
Zihan Zhu, Songyou Peng, Viktor Larsson, Weiwei Xu, Hujun Bao, Zhaopeng Cui, Martin R. Os- wald, and Marc Pollefeys. Nice-slam: Neural implicit scalable encoding for slam. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2022.
14

---

ablation study Published as a conference paper at ICLR 2023
A APPENDIX
In the appendix, we present the following:
A.1 FEATURE VOLUME INITIALIZATION
We initialize the parameters of all feature volumes under normal distribution, while the mean and standard deviation of the distribution are 0 and 0.001, respectively.
A.2 POST OPTIMIZATION IN OURS(ONE THREAD)
To enhance the geometry and color consistency throughout the entire scene, also introduce post-optimization module similar to one NICE-SLAM (Zhu et al. 2022) and iMAP (Sucar et al. 2021). This module is optional at the end of processing an imput video clip improve the final mesh quality without heavy computation. This post-optimization module optimizes the implicit scene representation with the camera poses fixed using all keyframes in alternative way. In each iteration, we sample a random keyframe & and select 20 keyframes from all other keyframes with least 70% overlap with frame k. We optimize the implicit function using all keyframes 1, 000 iterations.
A.3 ABLATION ON WARPING LOss
To the impact of utilizing multi-scale patch warping photometric loss,
on different settings of the warping loss. We evaluate different configurations during the initializa- tion stage. We mark our default configuration
the baseline: using multi-scale patch warping loss defined as Equation 7 where S {1.5. 11). We then compare it with two different patch size configurations, where grades to the pixel-wise warping loss Equation 5] As shown in Table 6 our default configuration achieves 1.85em in terms of the average depth error. The setting of (1, 113 slightly inferior to the bascline, while the setting of pixel-wise warping fails recover accurate depth. This is also illustrated in the qualitative comparison Figure]
{1} and S [1] respectively. Note that the setting of S {1} de-
A.4 MORE ABLATION ON FEATURE CHANNEL
We further verify the number of feature channels with the initialization stage. As shown in the right of Table/6] increasing the number features does not improve the quality of the initialized depth. Note that while the setting (L c 16) still initialize the reconstruction, the camera tracking soon fails in textureless regions. This ablation further justifies our choice C
deth
images
(b)
Figure 3: The visualization of the initialized depth map during optimization iterations. (a) The default config- uration with multiple patch-wise warping loss, S (1, 5, 11). (b) The pixel-wise warping loss, (1).
15

---

Published as a conference paper at ICLR 2023
S
(I,C)
Depth Error(em),
{1,5, 11)"
{1, [,]
[1]
(6,1)*
(6,4)
(6) (1,
MEAN
1.85
3.73
35
1.85
1.72
1.93
1.83
STD
0.15
0.85
3.65
0.15
0.09
0.16
0.19
Table 6: Mean depth errors of initialized map for different settings in the ablation study. The configuration of our method marked with report the mean and of depth error on runs of office-0 on Replica.
Replica
TUM-RGBD
Method
Mapping
Tracking
Mapping Tracking
NICE-SLAM
(1000,60)
(200,10) (5000,60) (5000.200)
Ours(two threads)
(3000,100)
(3000,20)
Ours(one thread)
(3000,100)
Table 7: (Pixel, Iteration) used different datasets. NICE-SIAAM needs more pixels and iterations on TUM-
RGBD, while ours(one thread) keeps the same configuration on all datasets.
A.5 TWO THREADS FOR TRACKING AND MAPPING, OURS(TWO THREADS)
We provide more details about our configuration which runs tracking and mapping on two threads. The thread tracking is similar to ours(one thread), while we only optimize the camera poses:
arg min "warping/wwrppng + Orenderrrendde PeW
(10)
where W is the window of the frames defined in Section 3.3 and we sample high gradient pixels
following the strategy in iNeRF (Yen-Chen al. 202T). We sample 3, 000 pixels and optimize 20 iterations for each frame.
In the mapping thread, we optimize both cameras poses and the parameters of the scene simultane- ously:
arg min e.Pa.kEW Wanquiu Otwarping wwapping + Osgender Lrender + Omooth L
(11)
where Wmapping is the keyframe window mentioned A.2, and we random sample 3, 000 pixels for mapping.
The major difference between the post-optinization in A.2 and the mapping thread is that we only pots-optimization the end of the sequence. Furthermore, do not optimize the camera pose during post-optimization in ours(one thread).
The memory and time cost of both ours(one thread) and ours(two threads) are listed in Table 5 To fairly compare the memory consumption between our method and NICE-SLAM (Zhu et al. 2022) we replace the volume with size 8em by 16cm. NICE-SLAM (Zuu et 2022 and ours have the same resolution in this configuration.
We report our performance of ours(two threads) and ours(one thread) on the Replica Dataset in TT The result of ours(two threads) is slightly inferior compared with ours(one thread), while it still outperforms another RGB method.
In Table 7 we report the number of pixels and iterations used in different datasets. Our method keeps the same setting for all datasets, while NICE-SLAM requires more pixels and iterations on the real-world dataset.
A.6 MESH EXTRACTION
Our implicit map representation contains a hierarchical feature volume and an MLP decoder. We can compute occupancy for each point in the 3D space from implicit map representation. Our method can easily predict the occupancy for unseen points. We use the marching cubes algorithm (Lorensen & Cline 1987) to create mesh for visualization and evaluation.
16

---

Published as a conference paper at ICLR 2023
A.7 MORE VISUALIZATION RESULT
To see more shape details on Replica (Straub et al. 2019), we also render these extracted meshes at close viewpoint and compare our results with those from iMAP (Sucar al. 2021 and DROID-
SLAM (TTed & Deng, 202T) in Figure It is clear that our method recovers more shape details and can fill in unobserved regions with the neural implicit map.
We provides visualization of all reconstructed scenes in Replica (Straub et al. 2019 in Figure and 6
In Figure we show two mesh reconstructions on the TUM RGB-D dataset. Our method could recover high-quality mesh due accurate camera poses and the smaller voxel size.
In Figure g we run our method on with different voxel size. We show the results when the smallest volume size 2, 0.5cm, re- spectively. Our method could recover fine detail of the small object if we increase the resolution of volume.
in (Shrestha et al. 2022)
17

---

Published as a conference paper at ICLR 2023
DROID-SLAN SLAM
iMAP
Ours
Ground Ground-Truth Truth
(a) Hole-Filling
(b) Fine- Line-Geometry
Figure 4: Rendering the reconstructed mesh from close viewpoint. Our results are more complete and accurate than those from iMAP and PROID-SLAM,
18

---

Published as a conference paper at ICLR 2023
office-1 0
office-1
office-3
office-4 4
Ours
Ground Truth
Figure 5: Visual comparison of the reconstructed meshes the Replica datasets.
19

---

Published as a conference paper at ICLR 2023
office-4 q
FOOH
Foom-1
room- 2
Ours
Ground Truth
Figure 6: Visual comparison of the reconstructed meshes the Replica datasets.
20

---

Published as a conference paper at ICLR 2023
fr1 .edsk
fi2 .yz
Ours
NICE-SLAM
Figure 7: The reconstructed mesh TUM-RGBD dataset.
age y
8cm
2cm
0.5cm
KU
BKK
Reconstruction Mesh
Input Image
Figure 8: The reconstructed mesh of pi
MeshMVS Shrestha al.
2022 dataset using different size voxel.
21


Reviews:
[{"review_id": "lEWG0_3a7_", "reviewer": "Reviewer_W3wZ", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper attempts to tackle an important topic of monocular dense mapping using neural implicit maps. Inspired by iMAP and NICE-SLAM, differential neural rendering and multi-scale feature volumes are adopted to serve as the driving force of optimisation. On important contribution claimed is the multi-scale patch-based photometric loss applied to the reference view and its overlapping views. Experiment results on three public available datasets shows the effectiveness of the proposed method in terms of tracking and mapping.", "summary_of_the_review": "I think this is a good paper which tries to taclke an interesting and important topic. \n\nHowever, I am not fully convincing about its real timeness (i.e., a SLAM) and would like to hear more from authors.\nI would like to adjust my rating after my concerns are properly addressed and more information could be provided.\n\n\n--------------------------------------------------------------------------------------------------------------------------------------------------------\nPost Rebuttal\n\nI have carefull read all the reviews and authors' feedbacks.\nMost of my concerns have been properly addressed by the authors, and I think a monocular dense neural filed SLAM like this would be beneficial to the area. There have also been some other NeRF-SLAM system recently available, it would be good if authors could add some discussion towards these recent works.\n\nOverall, I keep my positive rating towards this paper.", "strength_and_weaknesses": "Strengths:\n\nMonocular dense mapping of indoor scenes is an interesting and important research direction. This paper builds upon previous work on RGB-D neural mapping and extends them to monocular setup with proposed multiscale scene encoding together with warping loss as well as window optimisation. Though the idea of each component is not novel and has been adopted in related works, there is good system contribution by extending neural dense mapping to monocular cases.\nAdequate quantitative results on three public datasets show the performance gain compared to other baselines.\n\n\nWeaknesses:\n\n\n- As mentioned in the strength part, the unique technical novelty of this paper is a bit limited and is not as large as its system contribution.\n\n- To my own understanding, incorporating depths not only provide direct supervision of geometry, more importantly, these geometric cues would able to largely accelerate the learning process. I am still not fully clear whether the proposed system would able to convergence in real-time as a SLAM system. More explanation from authors are appreciated. If it is real-time, I would not call it a SLAM system but a SfM/reconstruction system.\n\n- Related to last point, in the main text, though the run time is provided, it is more desirable to show the practical tracking and mapping time efficiency (including necessary ). For example, iMAP works at 10Hz tracking and 2Hz mapping. What about the proposed system? I\n\n\n- One interesting question to analysis or discuss is whether batch optimization of MLP and poses from scratch are ale to converge using proposed patch-based photometric losses. Works like BARF and NeRF-- struggles in indoor scenes. If not, could you provide more insights why incremental manner would work?\n\n- Reconstruction results on real-world TUM, Euroc datasets or self-captured scenes are expected as only reconstructed meshes on Replica are shown. It would be interesting to see these real-world scenes captured in an less idea conditions. Some video demos would be great to see.\n\n- The MLP decoder is fixed after intialisation stage. What is the main point of dosing so. Will there be clear performance drop enabling optimisation of MLP decoder? I am wonder whether fixing it would somehow affect the system performance to regions which are largely different to these during initialsation.\n\n\n- Like iMAP, the reconstruction are mainly shown on bouded single room scenes (like Replica). Would the monocular tracking perform well on ScanNet datasets where the image capturing conditions are more challenging and the scene volume is also larger.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "E7QRXrgH_V", "reviewer": "Reviewer_K6me", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In this paper, the author proposes a RGB SLAM model with neural implicit map representation. In contrast to previous competitors, e.g., iMap and NICE-SLAM requiring depth maps as input, the proposed approach takes only RGB image sequences as input and leverages the hierarchical feature volumes with more levels to boost the performance. Experiments show that the proposed method achieves more accurate poses than NICE-SLAM and better depth maps than DROID-SLAM, but at the cost of higher memory and running time.", "summary_of_the_review": "As this paper is the first dense RGB SLAM with neural implicit map representation, I tend to accept this paper.", "strength_and_weaknesses": "strength:  \n\n(1)\tThe main contribution of the paper is the proposed dense SLAM system with implicit map representation, which allows SLAM system with implicit map representation to take only RGB images as input. \n\n(2)\tExperiments demonstrate that the proposed method gives competitive pose and depth accuracy to previous both classic and learning-based methods.  \n\nWeakness:\n\n(1)\tHigher memory and computational cost. The proposed method adopts hierarchical feature volumes with more levels to make the system work, which inevitably brings additional memory and time cost.\n\n(2)\tMissing explanations of some details.\n\na.\tOne thread for joint pose and map optimization. In many classic VO/SLAM systems (e.g., ORB-SLAM) and learning-based models (e.g., NICE-SLAM), two or more threads are used for tracking and mapping respectively to speed up. Is it possible to use a thread for tracking by taking only important pixels into consideration and another slower thread for mapping in this system? \n\nb.\tIn Eq (4), the paper uses randomly sampled M pixels for photometric rendering loss computation. In fact, because of the noise or the dynamic objects, not all pixels can provide the equal contribution to the constraint, so it might be better to use more stable pixels as in Eq (7).\n\n(3)\tDiscussion of related works of NeRFs without pose as input. \nThis work is also related to works of NeRFs which do not require poses as input, such as R1. \n\nR1: Barf: Bundle-adjusting neural radiance fields, Lin et al., ICCV 2021.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "6lB4U5G57B", "reviewer": "Reviewer_pLyh", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a SLAM system based on neural implicit representation as the underlying 3D map representation. Inspired by NICE-SLAM, the proposed approach uses a multi-resolution feature grid to represent the scene and jointly updates the camera poses and the map. The main difference to prior work is that the proposed approach operates on RGB instead of RGB-D images. This is achieved by integrating a multi-view stereo loss to enforce consistent depth and color predictions between close-by images. The proposed approach is evaluated on both synthetic and real data and the paper provides an ablation study.", "summary_of_the_review": "Overall, this is a solid paper that I think is of interest to the community. Most of the concerns raised above can be addressed quite easily. I am happy to increase my ranking if they are addressed. I did not give a higher ranking as I feel that the concerns have to be addressed first.\n\n------- After discussion -----------\n\nDuring the discussion phase, the authors successfully addressed my concerns. I thus increase my score and recommend to accept the paper.", "strength_and_weaknesses": "**Strength**:\n\nS1) The proposed approach is technically sound and its description in the paper is easy to follow.\n\nS2) Each part of the proposed approach is well-motivated.\n\nS3) The experimental results show that the proposed approach performs well on multiple datasets and is competitive with RGB-D-based methods.\n\n**Weaknesses**:\n\nW1) There are multiple statements that need to be clarified and / or revised (see below).\n\nW2) Eq. 5 is wrong. It should be $q_{k \\rightarrow l} = K_l \\tilde{R}_l^T \\left(\\tilde{R}_k K_k^{-1} q_k^\\text{homo} \\tilde{D}_q + \\tilde{t}_k - \\tilde{t}_l \\right)$.\n\nW3) Some design choices are not fully motivated: For the photometric warping loss, why not use the fact that the neural implicit representation also provides normal information to do a perspective warp of the patch instead of using a fronto-parallel warp? Why use SSIM and not any other robust loss (e.g., see [Park et al., Illumination change robustness in direct visual SLAM, ICRA 2017])?\n\nW4) Many components described in Sec. 3.1 are based on or inspired by prior work but are missing references. E.g., there is prior work on using multi-resolution volumes in combination with a MLP, e.g., NICE-SLAM, and work on explicitly storing the required information in the voxel volume (thus foregoing the use of a MLP), e.g., [Fridovich-Keil et al., Plenoxels Radiance Fields without Neural Networks, CVPR 2022] It would be good to acknowledge this prior work here. Similarly, using a photometric warping loss is common when training neural networks for monocular depth prediction, defining the photometric warping loss over patches is standard in the multi-view stereo literature, and depth map regulation is a standard loss term in networks that predict monocular depth. I don't think it is an issue that the paper builds upon existing work, but existing ideas should be clearly acknowledged.\n\nW5) It is unclear to me how the weights for Eq. 9 are chosen and how sensitive the proposed approach is to their choice.\n\nW6) Given that [Schoeps et al., CVPR 2019] showed that RGB-D-based methods perform worse on TUM than RGB-based methods, attributed due to the rolling shutter sensors used to capture the datasets, why not chose a dataset that enables a better comparison, e.g., the one from [Schoeps et al., CVPR 2019]? Why not compare against ORB-SLAM2 (or better ORB-SLAM3) in RGB- instead of RGB-D-mode to compare against a state-of-the-art feature-based approach?\n\nW7) The paper states that \"Our method is implemented with PyTorch (Paszke et al., 2017) and runs on a server with two NVIDIA 2080Ti GPUs.\" and that \"Our running time is reported with an RTX 2080TI GPU\". It is unclear to me whether one or two GPUs are used when reporting the running times in Tab. 4. This is important information required to understand the results.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "3w4LSiED-6", "reviewer": "Reviewer_bVvS", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents a dense SLAM method using neural implicit model as map representation. Compared to previous methods, the proposed method does not need depth maps as input. A hierarchical feature volume is used to facilitate map decoding. A photometric warping loss is proposed to be combined with the photometric rendering loss for optimizing the scene geoemtry and camera poses. Evaluation on both synthetic and real datasets show comparable or better results compared to previous methods. ", "summary_of_the_review": "Despite only moderate modifications and enhancement to NICE-SLAM, the presented work delivers surprisingly good results in some of the evaluations. I feel those parts need to be better supported by further experiments such as more dedicated ablation studies. Compared to the existing works, the overall novelty of this work is marginal. I therefore tend to reject the paper. ", "strength_and_weaknesses": "Streangth\n- The photometric warping loss is novel and seems to be the main source of the boosted performance compared to NICE-SLAM.\n- The evaluations are thorough and consider both map and camera pose accuracies and cover both synthetic and real datasets.\n\n\nWeakness\n- Compared to NICE-SLAM, the main differences are (1) using different number of levels for hierarchical feature volume (2) adding the photometric warping loss. However, the camera pose accuracies in Table 1 are ~10 times better than NICE-SLAM. This is quite surprising, especially considering that the proposed method does not consume depth maps. It might be helpful to add more discussions accordingly.\n- In Eq (6), why are the warped images comapred to the rendered images instead of to the real images? This looks a bit strange and more explaination should be added.\n- The way of getting the visibility mask on page 5 is a bit strange. The used method does not handle occlusion at all.\n- In 3.3, the naive way of finding the global keyframes are not scalable as it needs to traverse all the previous keyframes for each new frame. This will never work in a real SLAM system.\n- Please add units to Table 2 and Table 3.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster