SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Method	Part	Apartment-1	Apartment-2	Apartment-0	Office-0-C
(Single Room)
RMSE[cm]↓ / Mean [cm]↓ / Median [cm]↓
CCM-SLAM [33]	2.12/1.94/1.74	0.51/0.45/0.40	-/-/-	9.84/8.23/6.41
ORB-SLAM3 [4]	4.93/4.65/5.01	1.35/1.05/0.65	0.67/0.58/0.47	0.66/0.62/0.62
Swarm-SLAM [17]	Part 1	4.62/4.17/3.90	2.69/2.48/2.34	1.61/1.33/1.09	1.07/0.96/0.98
Ours (w/o)	1.15/0.99/0.88	1.45/1.34/1.36	0.70/0.48/0.27	0.71/0.62/0.67
Ours (w/)	1.11/0.95/0.81	1.41/1.30/1.36	0.62/0.47/0.30	0.50/0.46/0.55
CCM-SLAM	9.31/6.36/5.57	0.48/0.43/0.38	-/-/-	0.76/0.36/0.16
ORB-SLAM3	4.93/4.04/3.80	1.36/1.24/1.11	1.46/1.11/0.79	0.54/0.49/0.47
Swarm-SLAM	Part 2	6.50/5.27/4.39	8.53/7.59/7.10	1.98/1.48/0.94	1.76/1.55/1.83
Ours (w/o)	2.12/2.05/2.23	2.54/2.45/2.60	1.61/1.55/1.70	1.02/1.03/0.99
Ours (w/)	1.72/1.61/1.46	2.41/2.33/2.44	1.28/1.17/1.37	0.79/0.74/0.70
CCM-SLAM	5.71/4.15/3.66	0.49/0.44/0.39	-/-/-	5.30/4.29/3.29
ORB-SLAM3	4.93/4.35/4.41	1.36/1.15/0.88	1.07/0.85/0.63	0.60/0.56/0.55
Swarm-SLAM	Average	5.56/4.72/4.15	5.61/5.04/4.72	1.80/1.41/1.02	1.42/1.26/1.41
Ours (w/o)	1.64/1.52/1.56	2.00/1.90/1.98	1.16/1.02/0.99	0.86/0.81/0.83
Ours (w/)	1.42/1.28/1.14	1.91/1.82/1.90	0.95/0.82/0.84	0.65/0.60/0.63
Table 1: Two-agent Tracking Performance. ATE RMSE(↓), Mean(↓) and Median(↓) are used as evaluation metrics. We quantitatively evaluated respective trajectories (part 1 and part 2) and average results of the two agents. Comparison between ours(w/o) and ours(w/) reveals the importance of global pose graph optimization for collaborative tracking. "-" indicates invalid results due to the failure of CCM-SLAM.
Figure 5: Two-agent Trajectories on 4 Scenes. CCM-SLAM relies too much on 2D geometric information so that it has a large drift. In contrast, our neural implicit system has a robust performance.
by complex environments in multi-room sequences, the proposed system generally maintains better performance than other methods. It is worth noting that ORB-SLAM3 is not a collaborative SLAM system. It lacks the capability to process multiple sequences at the same time, thus unable to overcome the efficiency bottleneck in scene exploration. Specifically, we concatenate multiple image sequences and feed them into ORB-SLAM3, leveraging its "atlas" strategy for multi-sequence processing. Also, it can be seen that CCM-SLAM failed in some scenes because traditional RGB-based methods are prone to feature mismatching especially in textureless environments. Fig. 5 depicts the trajectories of each agent. Once two sub-graphs are fused, only a low-cost fine-tuning is required to adjust two neural fields and corresponding MLPs into a shared domain. Afterward, these two agents can reuse each other's previous observations and continue accurate tracking. Shared MLPs, neural fields, and following global pose graph optimization make CP-SLAM system a tightly collaborative system.
4.2 Single Agent with Loop
In Table. 2, we illustrate the localization performance of our system operating in single-agent mode on 4 loop closure datasets. Our method exhibits a notable superiority over recent methods, including NeRF-based and traditional ones, primarily attributed to the integration of concurrent front-end and back-end processing, as well as the incorporation of neural point representation. Qualitatively, we present trajectories of Room-0-loop and Office-3-loop from NeRF-based methods in Fig. 6. The
Figure 6: Single-agent Trajectories on 4 Scenes. In comparison with frequent jitters in the trajectories of the other two methods, our trajectory is much smoother.
Method	Metric	Room-0-loop	Room-1-loop	Office-0-loop	Office-3-loop	Average
NICE-SLAM [50]	RMSE [cm] ↓	1.27	1.74	2.27	3.19	2.12
Mean [cm] ↓	1.15	1.61	1.91	2.77	1.86
Median [cm] ↓	1.09	1.66	1.82	2.28	1.71
Vox-Fusion [44]	RMSE [cm] ↓	0.82	1.35	0.99	0.82	0.99
Mean [cm] ↓	0.77	1.30	0.94	0.74	0.94
Median [cm] ↓	0.78	1.25	0.95	0.73	0.93
ORB-SLAM3 [4]	RMSE [cm] ↓	0.54	0.21	0.58	0.89	0.56
Mean [cm] ↓	0.52	0.19	0.51	0.80	0.51
Median [cm] ↓	0.53	0.19	0.52	0.84	0.52
Ours(w/o)	RMSE [cm] ↓	0.61	0.51	0.67	0.38	0.54
Mean [cm] ↓	0.56	0.48	0.63	0.32	0.50
Median [cm] ↓	0.54	0.52	0.67	0.27	0.50
Ours (w/)	RMSE [cm] ↓	0.48	0.44	0.56	0.37	0.46
Mean [cm] ↓	0.44	0.40	0.53	0.31	0.42
Median [cm] ↓	0.43	0.46	0.56	0.27	0.43
Table 2: Single-agent Tracking Performance. Our system consistently yields better results compared with existing single-agent neural and traditional SLAM. In the fourth and fifth rows, we compare the accuracy of our system without and with the pose graph optimization module. Results are obtained in an origin-aligned manner with the EVO [9] toolbox.
experimental results demonstrate that the concentration level of density energy around sampling points is critical for neural SLAM. NICE-SLAM [50] uses dense voxel grids, which contain a large number of empty spaces, and sampling points in these empty spaces have almost no contribution to the gradient propagation. Vox-Fusion [44] has incorporated an important modification, implementing a sparse grid that is tailored to the specific scene instead of a dense grid. However, grid nodes can only be roughly placed near objects, which is unfavorable for tracking. In our point-based method, we ensure that neural field fits the real scene well. Feature embeddings can accurately include scene information. Concentratively distributed sample points and neural point based representation provide more exact gradients for pose backpropagation, which enables us to surpass NICE-SLAM and Vox-Fusion at lower resolution and memory usage. Moreover, we extend experiments on the TUM-RGBD real-world dataset, comparing with Co-SLAM [39] and ESLAM [12]. The results in Table. 3 illustrate that our method has also achieved state-of-the-art performance in the real-world setting, and the loop detection and pose graph optimization are equally effective for the real-world scene.
4.3 Map Reconstruction
The results in Table. 4 present a quantitative analysis of the geometric reconstruction produced by our proposed system in comparison to NICE-SLAM [50] and Vox-Fusion [44]. In our approach, we render depth maps and color maps every 10 frames throughout the entire predicted trajectories and utilize TSDF-Fusion (built-in function in Open3D [49] library) to construct mesh map. In
Method	fr1-desk (w/o loop)	fr2-xyz (w/o loop)	fr3-office (w/ loop)	Average
Co-SLAM [39]	7.10/6.83/6.79	4.05/3.76/3.47	5.58/5.06/4.57	5.58/5.22/4.95
ESLAM [12]	6.81/6.56/6.88	Fail/Fail/Fail	4.23/3.91/3.73	-
Ours	7.84/7.34/7.12	3.93/3.50/3.29	3.84/3.47/3.39	5.20/4.77/4.60
Table 3: Real-world Tracking Performance. In this real-world experiment, we can find that CP-SLAM still performs the best. Besides, ESLAM fails in the fr2-xyz because of OOM (out of memory). '-' indicates that metrics cannot be evaluated due to ESLAM failures.
Method	Metric	Room-0-loop	Room-1-loop	Office-0-loop	Office-3-loop	Average
NICE-SLAM [50]	Depth L1 [cm] ↓	1.54	1.00	0.93	2.06	1.38
Acc. [cm] ↓	3.30	3.19	2.88	3.95	3.33
Vox-Fusion [44]	Depth L1 [cm] ↓	0.77	1.30	0.94	0.74	0.94
Acc. [cm] ↓	2.25	1.67	1.68	2.31	1.98
Ours	Depth L1 [cm] ↓	0.32	0.23	0.22	0.76	0.38
Acc. [cm] ↓	1.53	1.20	1.21	1.7	1.41
Table 4: Reconstruction Results. Our system has a more powerful geometric reconstruction capability than existing methods.
Figure 7: Reconstruction. Collaborative reconstruction results of Apartment-1 (a) and Apartment-0 (b) show good consistency. (c) It can be seen that our system also achieves more detailed geometry reconstruction in four single-agent datasets, e.g., note folds of the quilt and the kettle in red boxes. Holes in the mesh reconstruction indicates unseen area in our datasets.
NICE-SLAM, several metrics are employed for mapping evaluation, namely depth L1 loss, mesh accuracy, completion, and completion ratio. In our loop datasets, scenes are not completely scanned, which leads to holes in mesh reconstruction. Therefore, our comparative experiments mainly focus on depth L1 loss and mesh accuracy. Fig. 7 qualitatively compares the single-agent reconstruction results of three methods and shows our collaborative mapping results. Evidently, our method achieves more detailed geometry across all datasets.
In this section, we examine some modules and designs in our system to prove their importance and the rationality of our pipeline.
Pose Graph Optimization. In this section, we conducted ablation experiments on the PGO module. Table. 1 and Table. 2 report results of ours(w/o PGO) and ours(w/ PGO) in single-agent and two-agent cases respectively. With the help of PGO module, the average positioning accuracy in the single-agent experiment decreases by  $10\%$ , while that in the two-agent experiment decreases by  $13\%$ .
Map Refinement. As shown in Fig. 8(a), we qualitatively illustrate the neural point field layout before and after map refinement in MeshLab. We can observe that a refined neural point cloud fits the ground truth mesh better.
Sampling Concentration. The density concentration near sampling points is a keypoint that determines the performance of neural SLAM. We design a set of experiments with different sampling interval lengths on Replica Room-0-Loop sequence. As shown in Fig. 8(b), tracking accuracy and depth L1 loss consistently drop to  $1.0\mathrm{cm}$  and  $0.6\mathrm{cm}$  as sampling points gradually diverge. This experiment fully verifies the theory in Section 4.2.
Neural Point Density. In CP-SLAM, we employed a fixed-size cubic cell within the filtering strategy
Metric	ρ=10cm	ρ=14cm	ρ=18cm	ρ=22cm
Ours (w/)	RMSE [cm] ↓	0.83	0.65	0.86	1.12
Mean [cm] ↓	0.77	0.58	0.76	1.01
Median [cm] ↓	0.83	0.51	0.71	0.94
Table 5: Neural Point Density Analysis. Results indicate that the point cloud density should be at an appropriate level, neither too high nor too low. Empirically, we have found that our system achieves the best performance when the cubic cell is set to  $\rho = 14cm$ .
Method	Tracking/Frame	Mapping/Frame	MLP Size	Feature Size
NICE-SLAM [50]	1.77s	11.26s	0.58×10^5	238.88MB
Vox-Fusion [44]	0.36s	0.83s	2.73×10^5	0.15MB
Ours	0.30s	10.10s	1.36×10^5	0.62MB
Table 6: Runtime and Memory Analysis.
(a)
(b)
Figure 8: Map Refinement and Sampling Concentration Ablation. (a) From a bottom view of Apartment-1, we can observe that there is an offset in the unrefined neural point cloud(blue), while the refined neural point cloud(pink) fits well. (b) As the sampling points diverge, the error rises.
to adjust the neural points, i.e., only the neural point closest to the center of a cubic cell is retained. To further explore the impact of point cloud density on tracking accuracy, we compared performance in the original Replica Room0 scene across various sizes of the cubic grid. The ablation study results in Table. 5 demonstrate that when the number of neural points is small, they are not enough to encode detailed scene geometry and color due to inadequate scene representation. Conversely, an excessive number of neural points obviously extends the learning time to converge. We empirically found that the setting  $\rho = 14cm$  worked consistently well on all the test scenes with different complexities in our experiment.
Memory and Runtime Analysis. We evaluated the runtime and memory consumption of our system on Replica Office-0-loop scene compared to NICE-SLAM and Vox-Fusion. We report single-frame tracking and mapping time, the size of MLPs and memory footprint of the whole neural field in Table. 6. The huge feature size in NICE-SLAM is due to its dense hierarchical feature grid.
We have proposed CP-SLAM, the first dense collaborative neural SLAM framework based on a novel neural point based representation, which maintains complete front-end and back-end modules like traditional SLAM systems. The comprehensive pipeline enables our system to outperform the state-of-the-art methods in both localization and reconstruction. One limitation of our method is its requirement for considerable GPU resources to operate multiple image sequences. Also, our system has slightly weaker hole-filling ability in unobserved regions than feature grid-based methods, which arises from the fact that neural points are distributed around the surfaces of observed objects, encoding surrounding scene information within a fixed-radius sphere. Moreover, the relative pose computation in the loop closure relies on the existing rendering-based optimization, which may be inaccurate for large viewpoint changes thus leading to drifting of map fusion. Hence, it is interesting to design a lightweight system with a coarse-to-fine pose estimation for future work.
Acknowledgment: This work was partially supported by the NSFC (No. 62102356).
[49] Qian-Yi Zhou, Jaesik Park, and Vladlen Koltun. Open3D: A modern library for 3D data processing. arXiv:1801.09847, 2018.
[50] Zihan Zhu, Songyou Peng, Viktor Larsson, Weiwei Xu, Hujun Bao, Zhaopeng Cui, Martin R Oswald, and Marc Pollefeys. Nice-slam: Neural implicit scalable encoding for slam. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12786-12796, 2022.