Adaptive Hyperparameter Tuning for Black-box LiDAR Odometry
Kenji Koide1 , Masashi Yokozuka1 , Shuji Oishi1 , and Atsuhiko Banno1

Abstract— This study proposes an adaptive data-driven hyperparameter tuning framework for black-box 3D LiDAR
odometry algorithms. The proposed framework comprises offline parameter-error function modeling and online adaptive
parameter selection. In the offline step, we run the odometry
estimation algorithm for tuning with different parameters andenvironments and evaluate the accuracy of the estimated trajectories to build a surrogate function that predicts the trajectory
estimation error for the given parameters and environments.
Subsequently, we select the parameter set that is expected toresult in good accuracy in the given environment based on
trajectory error prediction with the surrogate function. The
proposed framework does not require detailed informationon the inner working of the algorithm to be tuned, andimproves its accuracy by adaptively optimizing the parameter
set. We first demonstrate the role of the proposed framework in
improving the accuracy of odometry estimation across different
environments with a simulation-based toy example. Further, anevaluation on the public dataset KITTI shows that the proposed
framework can improve the accuracy of several odometry
estimation algorithms in practical situations.

I. INTRODUCTION
LiDAR odometry estimation is inevitable for autonomous
systems such as mobile service robots and autonomous
driving vehicles. A number of LiDAR odometry estimationand mapping algorithms have been proposed, and many
autonomous systems are built based on these robust andaccurate algorithms. However, these algorithms are often
surprisingly complex and involve many hyperparameters.
Their performance can be significantly affected by the choiceof hyperparameters; therefore, the hyperparameters must be
fine-tuned depending on the system and environment. Despite their significance, hyperparameters are manually tuned
in most studies, which results in considerable human effort
and suboptimal parameter sets. Although a few studies haveproposed automatic parameter tuning methods for odometry
estimation [1], [2], they are all dedicated to specific algorithms, sensors, and task configurations [3].
In our previous work, we proposed a general and automatic hyperparameter optimization approach for black-box
odometry estimation methods [4]. This approach repeats theparameter-sampling-and-trial loop with efficient parameter
space exploration based on sequential model-based optimization (SMBO). It uses only information on the input parameter
set and output trajectory of the algorithm to be tuned (i.e.,
black-box modeling) and facilitates the improvement of theperformance of any odometry estimation methods without
1 All the authors are with the Department of Information Technology
and Human Factors, the National Institute of Advanced Industrial Science and Technology, Umezono 1-1-1, Tsukuba, 3050061, Ibaraki, Japan,

k.koide@aist.go.jp

detailed information on their inner working. Although thisapproach is effective in improving the performance of odometry estimation methods on certain metrics and datasets, weobserved that it suffers considerably from overfitting to thetraining sequences. If the training set contains only easy
sequences, the SMBO tends to choose a very aggressive
parameter set that results in instability in other sequences.
Conversely, if there is a difficult sequence in the training set,
the optimized parameter set tends to be very conservative,
resulting in deteriorated trajectory accuracy.
In this work, to deal with overfitting and achieve abalance between robustness and accuracy, we propose a datadriven approach for adaptively selecting the hyperparametersof black-box odometry estimation methods. The proposed
framework automatically models the effects of the differences in hyperparameters on the behavior of the odometry
estimation algorithm, and then selects the best parameter set
for the current environment.
The proposed framework comprises offline parametererror function modeling and online adaptive parameter selection. In the offline step, we run the odometry estimation
algorithm F with different parameter samples xj and environments Ei and evaluate the trajectory estimation errors yij .
Subsequently, we build a surrogate function S that predicts
the trajectory estimation error of the algorithm for a given
parameter set and environment: S(xj , Ei ) ' yij . In the
online step, we identify the parameter set x̃t that minimizes
the predicted trajectory error for the current environment Ẽt :x̃t = arg min S(x, Ẽt ).
x

We first demonstrate the role of the proposed framework in
improving the accuracy of an odometry estimation algorithm
across different environments through a simulation-based
toy example. Further, an evaluation on the public dataset
KITTI shows that the proposed framework can be used inreal applications and improves the accuracy of odometry
estimation algorithms with completely different architectures(GICP odometry [5], LeGO-LOAM [6], and SuMa [7]).
II. RELATED WORKSLAM and odometry estimation algorithms are inherently
very complex and use many hyperparameters (e.g., point
cloud resolution, feature matching threshold, and keyframeinterval). It is known that, to achieve the best results, some
popular SLAM frameworks such as Google Cartographer [8]need to fine-tune many hyperparameters that affect each other1. This tuning process requires a deep understanding of the
1 https://google-cartographer-ros.readthedocs.io/
en/latest/tuning.html

Input Cloud

SMBO-based
Parameter Sampling

Online Parameter Selection
Param2

Offline Parameter-Error Function Modeling

Input Cloud

Param1

Environment
Descriptor

Odometry Estimation
Surrogate Function
Error Evaluation

Environment
Descriptor'

SMBO-based
Parameter Selection

Odometry Estimation

Fig. 1: Overview of the proposed adaptive parameter tuning framework.

inner working of the frameworks and considerable humaneffort of trial and error.
Despite the significance of hyperparameters, there are only
a few studies on the automatic parameter tuning of SLAMrelated algorithms. Nobili et al. automatically optimized
the outlier removal parameters of ICP algorithms for ahumanoid robot based on a metric of the overlap between
point cloud pairs [2]. Zheng used an exhaustive grid search
to optimize the parameters of a visual SLAM algorithm(e.g., number of features, patch size, and edge threshold)[9]. Similarly, Permeleau et al. investigated the changes in
the ICP registration accuracy depending on the parametervalues using exhaustive parameter-by-parameter evaluation[1]. Although several techniques have been proposed forthe fine-tuning of SLAM-related algorithms for specific use
scenarios [3], all the above works are dedicated to specificsensors, environments, and task configurations.
In the context of machine learning, hyperparameter tuning
has been recognized as an important step in maximizing
the performance of learning models [10]. When the numberof parameters to be tuned is small, fine-tuning is often
performed by exhaustive grid and random searches [11].
They are easy to implement and work well for classic models
that use only a few parameters (e.g., RBF SVM parameterselection [12]). However, the required number of evaluations
with these methods rapidly increases as the number ofparameters increases. Thus, they are not suitable for tuning
computationally expensive models with many parameters(e.g., deep neural networks).
In recent years, the sequential model-based optimization(SMBO) approach has been used to efficiently optimize
many parameters in computationally expensive models [13],[14]. This approach introduces a surrogate model that approximates the expensive model evaluation and samples
the parameter set that maximizes an acquisition criterion(e.g., expected improvement) on the surrogate model. Thisfacilitates the efficient exploration of the parameter spacewith fewer evaluations. Recent deep-learning-based methods,
which require the optimization of many hyperparameters foran expensive training model, have benefited from SMBO-

based automated hyperparameter tuning [15].
Although we can simply apply the SMBO approach to LiDAR odometry estimation such that the trajectory evaluation
metric is minimized on training sequences [4], we observedthat simple SMBO-based parameter optimization can considerably suffer from overfitting to the training sequences
for two reasons. First, we typically have fewer training
sequences in the odometry estimation problem compared tothe usual machine learning problems. Second, the corruptionof the trajectory estimation results in a catastrophic error in
the trajectory evaluation metrics (e.g., absolute and relative
trajectory errors [16]), which causes the SMBO to choosea very conservative parameter set. It is difficult for mostodometry estimation algorithms to deal with different environments using a single fixed parameter set; hence, we
argue that an adaptive mechanism to select an appropriate
parameter set depending on the environment is necessary
to further improve the trajectory estimation accuracy acrossdifferent environments.
Although there are a few works that adaptively optimize
the hyperparameters of specific algorithms depending on theenvironment (e.g., adaptive octree-resolution selection forvisual SLAM [17]), to the best of our knowledge, no work
has proposed a general and adaptive hyperparameter tuning
framework without a limitation on the algorithms to be tuned.
III. METHODOLOGY
Fig. 1 shows an overview of the proposed adaptive
hyperparameter tuning framework for black-box odometry
estimation algorithms. We first model a surrogate function S
offline that predicts the trajectory estimation error for a given
parameter set x and environment descriptor E. To achievethis, we run the odometry estimation algorithm multiple
times with different parameter samples xj and environments
Ei and evaluate the trajectory estimation errors yij . A knearest neighbor regressor-based surrogate function is thenfitted to the evaluation results: S(xj , Ei ) ' yij . To efficiently
explore the parameter space, we employ an SMBO-based parameter sampling technique. We then extract the environment
descriptor Ẽt for the input point cloud P̃t and identify theparameter set x̃t that minimizes the predicted error with thesurrogate function: x̃t = arg min S(Ẽt , x).
x

It should be noted that, while we assume uniform distributions for the parameters to be tuned for simplicity, the proposed framework can naturally be extended to accept otherparameter distribution classes (e.g., discrete and categorical
distributions).
A. Offline Parameter-Error Function Modeling
For each training sequence D, we run the odometry estimation algorithm F multiple times with different parameter
samples xj and obtain a set of estimated trajectories Tj =F(D, xj ). The accuracy of each trajectory is evaluated using
a metric G, such as relative trajectory errors (RTEs) [16]. We
use the notation G(F(D, x)) = G(x) for brevity. To efficiently explore the parameter space, we employ the SMBObased parameter sampling technique shown in Algorithm 1.
The SMBO fits a surrogate model Mj to the evaluation
history H and samples a parameter set that maximizes theacquisition criterion A on the surrogate model. We use theexpected improvement (EI) [18] as the acquisition criterion,
with the expectation that y negatively exceeds a thresholdy∗ :Z ∞EIy∗ :=max (y ∗ − y, 0) pMi (y|x)dy.
(1)

Fig. 2: Example of parameters sampled with SMBO. The
blue dashed line indicates the density of the sampled parameters. More parameters are sampled from the region where
the error is expected to be small.
Algorithm 1 Sequential Model-Based Optimization [13]1: H ← []2: for j ∈ [1, · · · , N ] do3:x∗ ← arg max A (Mj−1 (x))
x

4:
5:
6:

y ∗ ← G (x∗ )H ← H ∪ (x∗ , y ∗ )Fit Mj to H
return H−∞

The parameter set x∗ that maximizes EI can be determinedby introducing density functions l(x) and h(x), which are
fitted to the evaluation history where y < y ∗ and y >= y ∗ ,respectively, and identifying the parameter set that maximizes g(x)/l(x). y ∗ is a threshold chosen such that thequantile γ = p(y < y ∗ ). We refer the reader to [13] fora detailed derivation.
For parameter sampling, the SMBO is used to minimize
the average translational RTEs. We set the number of SMBO
iterations to 256, and thus obtain 256 parameter sets andcorresponding trajectories for each training sequence. Fig.
2 shows example parameter values of LeGO-LOAM [6]sampled with the SMBO. The blue line indicates the densityof the sampled parameters given by the kernel density
estimation. It can be observed that the SMBO efficiently
explores the parameter space by taking more samples fromthe region where the error is expected to be small.
For each frame i, we calculate the RTE yij of the subtrajectory starting from the frame with a parameter set
xj . The parameter values xj are then concatenated withthe environment descriptor Ei of the starting frame i tocompose a feature vector qij = [Ei , xj ], which describes the
combination of the environment and parameter values. We
then model a surrogate function S that predicts the estimation
error for a given parameter set and environment by fitting a knearest neighbor regressor (k = 5) such that S(Ei , xj ) ' yij .
For simplicity, we use the descriptor of the first frameof a sub-trajectory to predict the trajectory estimation error.
However, error prediction inherently requires informationon all the frames in the time period of the sub-trajectory.
Although this simplification can be justified by assumingthat consecutive frames have similar environment structuresin odometry estimation, extensions using a sequence ofdescriptors to capture drastic environmental changes andsensor motion can be considered in future work.
B. Online Parameter Selection
For the online parameter selection, we first extract theenvironment descriptor Ẽt for the input point cloud P̃t andfind the parameter set x̃t that minimizes the predicted error
with the surrogate function:x̃t = arg min S(Ẽt , x).

(2)xBecause the surrogate function S is a k-nearest neighbor
regressor that is nonlinear and non-convex, its minimization
is not trivial. Thus, we again use the SMBO parametersearch to determine the best parameter set x̃t for the current
environment. We perform parameter selection every second
and update the parameter set.
C. Environment Descriptor
As a descriptor to represent the environment structure,
we use simple NDT voxel classification-based features [19].
Following [19], we calculate the normal distribution voxels
for the input point cloud. Based on the eigenvalues of thecovariance matrices (λ1 > λ2 > λ3 ), we classify the voxels
into linear (λ2 /λ1 < te ), planar (λ3 /λ2 < te ), and spherical(other cases) distributions; te is a constant in (0, 1). Each
distribution class has three subclasses based on the line tilt
angle, plane tilt angle, and eigenvalue ratio (λ1 /λ2 ), and they
are invariant to yaw rotation. We count the number of each

A

C

B

C

A

B

Keyframe interval [m]CB

NDT resolution [m]ATime [Frame ID]

Fig. 4: Adaptively selected parameters through different
environments in the toy example.
Fig. 3: Simulation environment for the toy example. The
robot moves across cave (A) and outdoor (C) environments
through an open space (B). These environments have very
different structures and thus require different parameter settings.

subclass for several distance ranges and create a histogram(3.0 m × 10 bins × 9 sub-classes). We then apply PCA
dimensionality reduction (D = 10) to obtain the feature
vector E representing the environment structure.
It is worth mentioning that the proposed framework isagnostic to the environment descriptor, and other features(e.g., hand-crafted [20] as well as learned features [21]) canbe used with slight modifications.
IV. E XPERIMENTS
A. Simulation-based Toy Example
To demonstrate the process where the proposed framework
adaptively selects parameters and improves the accuracy ofthe odometry estimation across different environments, we
conducted experiments in a simulated environment, as shownin Fig. 3. The environment consists of a cave (A), open space(B), and outdoor street environments (C). They have very
different environmental structures and thus require different
parameter settings.
We used a simple keyframe-based NDT scan matchingodometry algorithm with only two parameters: NDT voxel
resolution and keyframe update interval. They provide a
trade-off between accuracy and stability (large resolution:better convergence and lower accuracy, large keyframe interval: low drift and worse stability) and need to be tuned
depending on the environment.
For offline parameter-error function modeling, werecorded four LiDAR data sequences and ground truth trajectories for each of the cave and outdoor environments.
For each sequence, we sampled 256 parameter sets andcalculated the translational RTEs of 25 m sub-trajectories

TABLE I: Translational RTEs [%] on the toy example
sequence with different parameter sets
Parameter setting
RTE
Baseline
0.767
NDT odometry
Tuned (Fixed)
0.759 (-0.008)
Tuned (Adaptive)
0.530 (-0.237)
Values in parentheses are with respect to the baseline
Methodto evaluate the estimated trajectories. For validation, werecorded a sequence across the cave, open space, and outdoor
environments.
We ran the NDT scan matching odometry algorithm on
the validation sequence with three parameter settings: 1)
manually tuned baseline parameter set, 2) fixed parameter
set tuned with SMBO for the training sequences, and 3)
adaptively selected parameter sets. Fig. 4 shows a plot of
the adaptively selected NDT resolution and keyframe interval
parameters. It can be observed that small keyframe intervalvalues were selected to prevent estimation corruption in the
cave environment (A), whereas large interval values wereselected after the sensor moved out from the cave to improve
the accuracy (B)(C). It can also be observed that large NDTresolution values tend to be selected in (B) and (C) to better
capture the structure of the open environments. It is worthemphasizing that the proposed framework automatically determines how the parameters should be tuned for different
environments using the data-driven approach; consequently,
it does not require any information on the inner working of
the algorithm to be tuned.
Table I shows the averaged translational RTEs for the different parameter settings. Although the fixed tuned parameterset exhibited slightly better accuracy (0.759 %) comparedto the manually tuned baseline (0.767 %), the improvement
was small. This is because it needed to select a conservative
parameter set for the cave environment, which did not
result in accuracy improvement in the outdoor environment.

(a) Environment (sequence 08)(b) Keyframe interval

Fig. 6: Example of adaptively optimized hyperparameters.

Fig. 5: Parameters of GICP scan matching odometry sampledwith SMBO for KITTI sequence 00 to 06. The color of thepoints indicates translational RTE [%].

In contrast, the adaptive parameter selection significantly
improved the estimation accuracy (0.530 %). This is because
it adaptively changed the parameter set from conservative toaggressive after leaving the cave environment.
This simple toy example illustrates the necessity of an
adaptive parameter selection mechanism to enhance thetrajectory estimation accuracy across different environments.
B. Evaluation on KITTI
Next, we conducted an evaluation on a public LiDAR
odometry dataset, KITTI, to demonstrate that the proposed
approach improves the accuracy of several odometry estimation algorithms in practical situations. The dataset contained
11 sequences recorded using Velodyne HDL-64E at a rateof 10 Hz. We used sequences 00-05 for the parameter
optimization, and sequences 06-10 for validation.
We evaluated three odometry estimation algorithms withdifferent architectures: keyframe-based GICP odometry [5],LeGO-LOAM [6], and SuMa [7] 2 . As the baseline, we
manually tuned the parameter sets of the GICP odometry
and LeGO-LOAM for sequence 00. For SuMa, we used
the parameter set optimized for all the sequences in [7].
We compared the trajectory errors of these algorithms with
1) the baseline parameter set, 2) a fixed parameter set
optimized with SMBO for the training sequences, and 3)
2 The list of tuned parameters of each algorithm is available at https://github.com/SMRT-AIST/adaptive_tuning/tree/devel

adaptively selected parameter sets. For the parameter-error
function modeling of the adaptive parameter selection, weused translational RTEs of 100 m sub-trajectories as theevaluation metric.
Fig. 5 shows the parameters of the GICP odometry (maximum correspondence distance and keyframe interval) for
each training sequence (sequence 00-05) sampled during theoffline parameter-error function modeling. The color of thepoints indicates the translational RTE. It can be observedthat different sequences require very different parameters. In
particular, sequence 01, which is the most difficult sequence
in KITTI owing to the feature-less environment structure,requires a very large maximum correspondence distanceparameter to prevent estimation corruption. It can also be
observed that the required keyframe interval is significantly
changed for each sequence. This result clearly demonstrates
the necessity of the adaptive parameter tuning mechanism.
If a conservative parameter set is chosen for the difficult
sequence (sequence 01), the estimation results of othersequences would deteriorate. Conversely, if a parameter set
for another sequence (e.g., sequence 00) is chosen, the estimation would be corrupted on the difficult sequence. Thereis no parameter set that works well for all the sequences, andthus the parameter set must be adaptively selected dependingon the environment.
Table II shows the translational RTEs averaged over 100
to 800 m trajectories for the odometry estimation algorithms
with different parameter settings. We used the KITTI official
evaluation code to calculate the RTEs.
Although the fixed parameter set optimized with SMBO
improved the RTEs of the GICP odometry for the training
sequences, the RTEs on the test set deteriorated comparedto the baseline. This is because the fixed parameter selection
strategy required to choose a conservative parameter set forthe difficult sequences to improve the RTEs on the training
set. The proposed approach significantly improved the RTEs
on both the training and test sets by adaptively selecting thebest parameter sets depending on the environment. Fig. 6
shows the adaptively selected keyframe interval values of theTABLE II: Average translational RTEs [%] for KITTI
MethodGICP

LeGO-LOAM

SuMa

Seq. ID# Frames
Baseline
Fixed
Adaptive
Baseline
Fixed
Adaptive
Baseline
Fixed
Adaptive

00
4541
1.30
1.21
1.21
1.73
1.87
1.47
0.89
0.84
0.90

01
1101
2.42
2.45
2.16
13.33
3.07
2.78
6.76
5.59
5.69

Training set
02
03
04
05
06
Avg.
4661
801
271
2761
1101
1.72
1.08
1.89
0.90
1.46
0.78
1.66
1.08
0.99
0.61
1.34 (-0.12)
0.55
1.48
0.97
0.78
0.64
1.26 (-0.20)
0.57
1.76
1.61
1.35
1.07
2.44
1.05
1.83
1.63
1.33
1.02
1.77 (-0.67)
1.09
1.53
1.22
1.21
0.79
1.45 (-0.99)
0.90
1.37
1.37
0.48
1.06
1.52
0.62
1.27
1.14
0.56
0.87
1.35 (-0.17)
0.60
1.24
1.19
0.58
1.00
1.39 (-0.13)
0.61
Values in parentheses are with respect to the baseline

GICP odometry for sequence 08. Small interval values wereselected on corners to prevent estimation corruption, whereas
large values were selected on several long straight paths toreduce the odometry drift and achieve better accuracy.
The results of LeGO-LOAM exhibited a similar trend tothat of the GICP odometry. Although the fixed parameter set
significantly improved the translational RTEs on the training
set, the RTEs on the test set deteriorated. The adaptive
parameter selection successfully improved the RTEs for boththe training and test sets.
Although both the fixed and adaptive parameter sets significantly improved the RTEs of SuMa on the training set,
the improvement on the test set was very small. This may be
because 1) the baseline parameter set was highly optimized
for the entire dataset, and 2) the modified parameters affected
the consistency of the surfel map of SuMa, resulting in better
accuracy in some test sequences (sequences 06, 07, 09, and10) but worse accuracy in a long sequence (sequence 08).
These results indicate the shortcomings of the current formof the proposed framework; it uses a simple environment
descriptor and regression model that may not capture thefine behavior of the odometry estimation algorithm. We inferthat this can be further improved by considering as input
environment descriptor sequences to better represent thechanging environmental structures and introducing a small
amount of prior information on the relationship between theparameters and odometry estimation algorithm behavior.
V. C ONCLUSIONSThis study proposed an automatic and adaptive hyperparameter tuning framework for black-box LiDAR odometry
estimation algorithms. The proposed method first runs theodometry estimation algorithm to be tuned with different
parameters and environments offline, and creates a surrogate
function to predict the estimation error for novel parameters and environments. For online parameter selection, it
chooses the parameter set that is expected to yield better accuracy in the given environment with the surrogate
function. We employed an SMBO technique for efficientoffline parameter space exploration and online surrogate
function minimization. The evaluation results showed that the
proposed approach improves the accuracy of the odometry
estimation algorithms without detailed information on theirinner working.

07
1101
1.30
1.35
0.92
0.97
1.02
1.02
0.70
0.66
0.65

08
4071
1.43
2.00
1.18
1.69
1.76
1.58
1.36
1.43
1.40

Test set
09
1591
1.17
1.06
1.22
1.33
1.29
1.28
1.06
0.86
0.94

10
1201
1.72
1.12
1.50
1.77
1.83
1.73
1.93
1.73
1.65

Avg.
1.33
1.54 (+0.21)
1.13 (-0.20)
1.52
1.56 (+0.04)
1.43 (-0.09)
1.23
1.21 (-0.02)
1.20 (-0.03)

ACKNOWLEDGMENT
This work was supported in part by a project commissioned by the New Energy and Industrial Technology
Development Organization (NEDO).
R EFERENCES[1] F. Pomerleau, S. Magnenat, F. Colas, M. Liu, and R. Siegwart,“Tracking a depth camera: Parameter exploration for fast ICP,” inIEEE/RSJ International Conference on Intelligent Robots and Systems.
IEEE, Sept. 2011.
[2] S. Nobili, R. Scona, M. Caravagna, and M. Fallon, “Overlap-based
ICP tuning for robust localization of a humanoid robot,” in IEEE
International Conference on Robotics and Automation. IEEE, May
2017.
[3] F. Pomerleau, F. Colas, and R. Siegwart, “A review of point cloud
registration algorithms for mobile robotics,” Foundations and Trendsin Robotics, vol. 4, no. 1, pp. 1–104, May 2015.
[4] K. Koide, M. Yokozuka, S. Oishi, and A. Banno, “Automatic hyperparameter tuning for black-box lidar odometry,” in IEEE International
Conference on Robotics and Automation. IEEE, May 2021.
[5] A. Segal, D. Haehnel, and S. Thrun, “Generalized-ICP,” in Robotics:
Science and Systems. Robotics: Science and Systems Foundation,
June 2009.
[6] T. Shan and B. Englot, “LeGO-LOAM: Lightweight and groundoptimized lidar odometry and mapping on variable terrain,” inIEEE/RSJ International Conference on Intelligent Robots and Systems.
IEEE, Oct. 2018.
[7] J. Behley and C. Stachniss, “Efficient surfel-based SLAM using 3d
laser range data in urban environments,” in Robotics: Science andSystems. Robotics: Science and Systems Foundation, June 2018.
[8] W. Hess, D. Kohler, H. Rapp, and D. Andor, “Real-time loop closurein 2d LIDAR SLAM,” in IEEE International Conference on Robotics
and Automation. IEEE, May 2016.
[9] Z. Zheng, “Feature based monocular visual odometry for autonomous
driving and hyperparameter tuning to improve trajectory estimation,”
Journal of Physics: Conference Series, vol. 1453, p. 012067, Jan. 2020.
[10] P. Probst, A.-L. Boulesteix, and B. Bischl, “Tunability: Importance ofhyperparameters of machine learning algorithms.” Journal of Machine
Learning Research, vol. 20, no. 53, pp. 1–32, Mar. 2019.
[11] J. Bergstra and Y. Bengio, “Random search for hyper-parameter
optimization,” Journal of Machine Learning Research, vol. 13, no.
null, p. 281–305, Feb. 2012.
[12] S. Han, Cao Qubo, and Han Meng, “Parameter selection in svm withrbf kernel function,” in World Automation Congress, June 2012, pp.
1–4.
[13] J. S. Bergstra, R. Bardenet, Y. Bengio, and B. Kégl, “Algorithms
for hyper-parameter optimization,” in Advances in Neural Information
Processing Systems, J. Shawe-Taylor, R. S. Zemel, P. L. Bartlett,F. Pereira, and K. Q. Weinberger, Eds., Dec. 2011, pp. 2546–2554.
[14] B. Shahriari, K. Swersky, Z. Wang, R. P. Adams, and N. de Freitas,“Taking the human out of the loop: A review of bayesian optimization,” Proceedings of the IEEE, vol. 104, no. 1, pp. 148–175, Jan.
2016.

[15] X. Liang, “Image-based post-disaster inspection of reinforced concrete
bridge systems using deep learning with bayesian optimization,”
Computer-Aided Civil and Infrastructure Engineering, vol. 34, no. 5,
pp. 415–430, Dec. 2018.
[16] Z. Zhang and D. Scaramuzza, “A tutorial on quantitative trajectory
evaluation for visual(-inertial) odometry,” in IEEE/RSJ International
Conference on Intelligent Robots and Systems. IEEE, Oct. 2018.
[17] E. Vespa, N. Funk, P. H. J. Kelly, and S. Leutenegger, “Adaptiveresolution octree-based volumetric SLAM,” in International Conference on 3D Vision. IEEE, Sept. 2019.
[18] D. Zhan and H. Xing, “Expected improvement for expensive optimization: a review,” Journal of Global Optimization, vol. 78, no. 3,

pp. 507–544, July 2020.
[19] M. Magnusson, H. Andreasson, A. Nuchter, and A. Lilienthal,“Appearance-based loop detection from 3d laser data using the normal
distributions transform,” in IEEE International Conference on Robotics
and Automation. IEEE, May 2009.
[20] R. B. Rusu, N. Blodow, and M. Beetz, “Fast point feature histograms(FPFH) for 3d registration,” in IEEE International Conference on
Robotics and Automation. IEEE, May 2009.
[21] Z. Gojcic, C. Zhou, J. D. Wegner, and A. Wieser, “The perfect match:
3d point cloud matching with smoothed densities,” in IEEE/CVF
Conference on Computer Vision and Pattern Recognition. IEEE,
June 2019.