Co-SLAM：Joint Coordinate and Sparse Parametric Encodings for
Neural Real-Time SLAMIRU)DVW+LJK)LGHOLW\5*%'6XUIDFH5HFRQVWUXFWLRQ
Hengyi Wang*

Jingwen Wang*Lourdes Agapito* equal contribution

Paper, code, and additional results can be found athttps://hengyiwang.github.io/projects/CoSLAM.html

University College London

RQ

HUYLHZ
1. Introduction

3. Pipeline

5HVXOWVPSOH SRLQW DORQJ D UD\ HDFK JULG LV
RGB-D based Neural SLAMWULOLQHDU LQWHUSRODWLRQ 0XOWLOHYHO
Input dataModel
FRQFDWHQDWHG DQG GHFRGHG LQWR 6')
Scene
FRPSXWH WKH VDPSOH ZHLJKWrepresentation&RORU LV
DUDWHO\IURPWKHILQHVWJULG

5. Performance Analysis
1. Tracking: Update camera pose only

1

Output2. Mapping: Update camera pose and41) Reconstruction (Better, Faster)
Datasetmodel parameters by sampling rays fromall keyframe rays (NeuralRGBD[CVPR’22])

3. Model: HashGrid+OneBlob for SDF,Replica

feature+OneBlob for color

Keyframe
databasea) RGB-D sequence

3avoid free space noise (hash collision)b) Tracking + Mapping

Parametric encoding + Tiny MLP

HQHUJ\WHUPVZKLFKFRXOGEHJURXSHGNICE-SLAM
Scene
HV representation

RVVHV

/RVVHV

4. Results

1[CVPR’22]

Coordinate + sparse
parametric encoding + Tiny MLP

w/ smooth term

Replica

Memory efficient
Hole filling

Mapping (ms)MemoryATE#

One grid

9.9⇥207.8⇥2025.4⇥1020.2⇥10

1.7M
0.8M

8.69
8.75

MethodNeural RGBD
GO-SurfOurs

2

Acc. #0.0151
0.01580.0149Comp. #0.01970.0195
0.0179C-`1 #

0.01740.01770.0164NC "0.9316
0.93170.9292F-score "0.96350.9591
0.9629Run time
10-25h15-45min100-500s

Table 7. Quantitative results of the reconstruction on 10 synthetic
scenes [1]. The evaluation metrics and protocol follow Neural
RGBD [1] and GO-Surf [6]. We achieve on-par performance but
our training is significantly faster.

Hash lookup

NICE-SLAM[CVPR’22]

Tracking (ms)Table 6. Performance analysis of modeling the geometry and appearance using one/two grids. The ATE is similar while using
one grid require less computational cost. Run-time is reported inms/iter ⇥ #iter format.

2

Co-SLAM[OURS]iMAP[ICCV’21]

MethodScanNetTwo grids

2

1w/o smooth term

Figure 3. Reconstruction result w/ and w/o our smoothness term.
Smoothness term could effectively remove hash collision artefacts.

Fast
Scalable

1

Name

Co-SLAM[OURS]

w/o BA
LBA
GBA-10
GBA‡
GBA

Keyframe
Q/RVVHVdatabase

KF selection
Local
X

Global

#KF
0X

X
X
X

10X
X

All

X
X

Poseoptim.
X
XX

ATE (cm)

Std. (cm)

16.81
9.69
9.54
9.72
8.75

1.69
1.38
0.67
0.53
0.33

Table 8. Ablation using different BA strategies on Co-SLAM:(LBA) BA with rays from 10 local keyframes; (GBA-10) BA withrays from 10 randomly selected keyframes; (GBA‡ ): BA with raysfrom all keyframes w/o pose optimization; (GBA) BA with raysfrom all keyframes and pose optimization (our full method). Allmethods sample a total of 2048 rays per iteration.

Information gain -> keyframe
Active sampling

Every k frame -> keyframe
Visual overlapping

KHFNRXWRXUSURMHFWSDJHIRUPRUH

Every k frame -> keyframe rays
Global samplig

3.62

4.93

80.51

9.9

0.3

NICE-SLAM

2.37

2.64

91.13

0.9

17.4

Co-SLAM

2.10

2.08

93.44

17.4

0.3

iMAP*

18.30

26.41

20.73

0.3

0.2

5.96

5.30

77.46

1.3

3.1

2.95

2.96

86.88

15.6

0.3

Synthetic
NICE-SLAMRGBD
Co-SLAM

ATE
DatasetMethodRMSE[1] Dejan Azinović, Ricardo Martin-Brualla, Dan B. Goldman,
Matthias Nießner, and Justus Thies. Neural RGB-D surface (cm)12iMAP

References

w/o smoothw/ smoothiMAP[ICCV’21]

Comp.
FPSRatio#Param.

2) Tracking

2Coordinate encoding + MLP

Comp(cm)

based method while GO-Surf [6] is a parametric encodingbased method. By using the proposed joint coordinate andparametric encoding, we achieve competitive(Better,reconstrucFaster)tion performance with significantly faster training speed.

c) Trajectory + Reconstruction

2. Motivation

Q2EMHFWLYHV

4. Smoothness: Regularize feature toAcc(cm)

MethodiMAPNICE-SLAM

Co-SLAMGT

iMAP*such sampling strategy can still significantly improve theNICE-SLAM performance Co-SLAM
GT
and the robustness of the pose estimation.

2.5. Batch-mode Optimisation
To validate the representation ability of the proposedreconstruction. In Proceedings of the IEEE/CVF Conference
on Computer Vision and PatterniMAP*Recognition, pages 6290– 36.7
ScanNet6301, 2022. 1,2, 3, 4
NICE-SLAM
9.6[2] Angela Dai,(6AngelX. Chang, Manolis Savva, Maciej Halber,scenes)
Thomas Funkhouser, and Matthias
Nießner. Scannet: Richly- 8.7
Co-SLAM
annotated 3d reconstructions of indoor scenes. In ProceediMAP Vision and Pattern 4.2
ings of the IEEE Conference on Computer
TUM
Recognition,pagesRGBD
5828–5839, 2017. 2NICE-SLAM
2.5[3] Angela Dai,(3MatthiasNießner, Michael Zollhöfer, Shahramscenes)
Izadi, and Christian Theobalt. Bundlefusion:
Real-time glob- 2.4
Co-SLAM
ally consistent 3d reconstruction using on-the-fly surface reintegration. ACM Transactions on Graphics (ToG), 36(4):1,2017. 4[4] Julian Straub, Thomas Whelan, Lingni Ma, Yufan Chen, Erik
Wijmans, Simon Green, Jakob J. Engel, Raul Mur-Artal, CarlRen, and Shobhit Verma. The Replica dataset: A digitalreplica of indoor spaces. arXiv preprint arXiv:1906.05797,2019. 2, 3
[5] Edgar Sucar, Shikun Liu, Joseph Ortiz, and Andrew J. Davison. iMAP: Implicit mapping and positioning in real-time.
In Proceedings of the IEEE/CVF International Conference on
Computer Vision, pages 6229–6238, 2021. 1, 2, 3, 5, 6, 7, 8
[6] Jingwen Wang, Tymoteusz Bleja, and Lourdes Agapito.
Go-surf: Neural feature grid optimization for fast, highfidelity rgb-d surface reconstruction.
arXiv preprint
arXiv:2206.14735, 2022. 1, 2, 4
[7] Zihan Zhu, Songyou Peng, Viktor Larsson, Weiwei Xu, Hujun Bao, Zhaopeng Cui, Martin R. Oswald, and Marc Pollefeys. Nice-slam: Neural implicit scalable encoding for slam.
In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12786–12796, 2022. 1,2, 3, 5, 6, 7, 8FPS

#Param.

0.4

0.2

0.7

101.6

12.8

1.6

9.9

0.2

0.1

3.1

13.3

1.63) Memory ablation

Joint encoding helps the memory compression