Learning Inertial Odometry for Dynamic Legged Robot State Estimation
Authors: Russell Buchanan
Marco Camurri
Frank Dellaert
Maurice Fallon
Venue: CoRL2021 Poster (2021)
Keywords: Legged Robots
Inertial Navigation
Deep Neural Networks

Abstract: This paper introduces a novel proprioceptive state estimator for legged robots based on a learned displacement measurement from IMU data. Recent research in pedestrian tracking has shown that motion can be inferred from inertial data using convolutional neural networks. A learned inertial displacement measurement can improve state estimation in challenging scenarios where leg odometry is unreliable, such as slipping and compressible terrains. Our work learns to estimate a displacement measurement from IMU data which is then fused with traditional leg odometry. Our approach greatly reduces the drift of proprioceptive state estimation, which is critical for legged robots deployed in vision and lidar denied environments such as foggy sewers or dusty mines. We compared results from an EKF and an incremental fixed-lag factor graph estimator using data from several real robot experiments crossing challenging terrains. Our results show a reduction of relative pose error by 37% in challenging scenarios when compared to a traditional kinematic-inertial estimator without learned measurement. We also demonstrate a 22% reduction in error when used with vision systems in visually degraded environments such as an underground mine.

Learning Inertial Odometry for Dynamic Legged Robot State Estimation
Russell Buchanan
Marco Camurri Oxford Robotics Institute University of Oxford mcamurri@robots. ox. ac uk
Oxford Roboties Institute University of Oxford russell crosossrrbots... ox. ac uk
Frank Dellaert
Maurice Fallon
Institute for Roboties and Intelligent Machines Georgia Institute of Technology dellaert8cc, gatech. deelaert@cc.atechh. edu
Oxford Roboties Institute University of Oxford
mfal afflll mfallondrobots. uk
Abstract: This paper introduces novel proprioceptive state estimator for legged robots based on a learned displacement measurement from IMU data. Recent re- search in pedestrian tracking has shown that motion can be inferred from inertial data using convolutional neural networks. A learned inertial displacement mea- surement can improve state estimation in challenging scenarios where leg odom- etry is unreliable, such as slipping and compressible terrains. Our work learns to estimate displacement measurement from IMU data which is then fused with traditional leg odometry. Our approach greatly reduces the drift of proprioceptive state estimation, which critical for legged robots deployed in vision and lidar denied environments such as foggy sewers or dusty mines. We compared results from an EKF and an incremental fixed-lag factor graph estimator using data from several real robot experiments crossing challenging terrains. Our results show a reduction relative pose error by 37 % challenging scenarios when compared to traditional kinematic-inertial estimator without learned measurement. We also demonstrate a 22 % reduction error when used with vision systems in visually degraded environments such underground mine.
Keywords: Legged Robots, Inertial Navigation, Deep Neural Networks
Introduction
Recent advances in legged robot locomotion have motivated the use of quadruped robots in dull and dirty industrial applications. perform these tasks autonomously, accurate state estimation with limited drift over long periods of time is fundamental. However, operating in challenging environ- ments such mines and sewers, dust particles and vapor can severely degrade camera and lidar systems [1] (Fig. 1). Proprioceptive methods can work in these environments, they fuse informa- tion only from joint kinematics and Inertial Measurement Unit (IMU) data [2, 3, 4]. However, slip events or deformation of the robot's foot or terrain introduce non-Gaussian error that accumulates over time and leads to unbounded drift.
Separately, recent advances in pedestrian tracking using only IMU data with machine learning have shown promising results [5, 6]. The key insight of these works is that possible learn mo- tion prior from inertial data which could be robust to changing biases. We propose deep neural network capable of learning motion prior on the locomotion of legged robot for any terrain and in challenging conditions such slipping. The network takes buffer of IMU data and out- puts displacement measurement and covariance which is fused with kinematic information. We demonstrate the application of this measurement in both filtering and factor-grap optimization con- texts track the motion of an ANYbotics ANYmal quadrupedal robot and compare results to other learning- based inertial odometry systems [5].
5th Conference on Robot Learning (CoRI. 2021), London, UK.

---

Figure I: Left: The ANYmal C100 quadrupedal robot exploring a limestone mine. Right: Images from the on-board camera which demonstrate the vision challenges in such an environment.
The contributions can be listed follows: 1) First kinematics-inertial estimator based on learned IMU displacement measurements; 2) Integration with both a filter-based estimator and a factor graph, merging concepts from machine learning and model based methods; 3) Extensive testing on a quadruped robot, including application to field experiments of the robot navigating a mine.
2 Related Work
In this section, we briefly summarize relevant work on proprioceptive state estimation for legged robots (Section 2.1) and related research in learned inertial state estimation (Section 2.2).
2.1 Kinematic-Inerrin State Estimation in Legged Robots
Lin et al. [7] presented the first state estimator for legged robots which fused IMU and kinematic data in an Extended Kalman Filter (EKF). Their method was highly dependent on the assumption that their hexapedal robot had always three feet in contact with the ground. Bloesch et al. [2] introduced a new filtering approach called the Two-State Implicit Filter (TSIF). This method employs residual- based modeling of the available information, eliminating the need for explicit process model. The default state estimator for the ANYmal robot, which we use our experiments, is based on TSIF.
More recently, Pronto [8] has been presented as an EKF-bbsed state estimator with the capacity to fuse pose corrections from exteroceptive sensing. This done by maintaining a history of states, measurements and covariances, and applying corrections to the past trajectory asynchronously.
Factor graphs have also been used for legged robot state estimation. Hartley et al. [9] proposed hybrid preintegrated contact factor so that multiple foot steps or measurements could be combined into a single factor, reducing the number of nodes in the graph. Wisth al. [10] used factor graphs to fuse visual information with inertial and kinematics. In [11]. the same authors improved on the factor graph formulation by adding feature tracking- based lidar measurements.
2.2 Learned Inertial Navigation
Recent research in inertial navigation systems (INS) has applied machine learning to infer motion directly from IMU data typically focused tracking a handheld mobile phone. These methods have shown impressive performance when compared to traditional inertial-only pedestrian tracking methods. However, these methods are susceptible to failure if applied outside of the training domain (e.g., network trained with pedestrian motion would not generalize to a person on a bicycle).
Chen et al. [6] introduced IoNet, the first data driven INS which estimates odometry using deep recurrent neural networks (RNNs). The network was trained to relate buffered data from an IMU into 2D displacements and orientation changes. Yan et al. [12] proposed RoNIN, network architecture which similarly infers 2D velocity and orientation from raw IMU data.
The first approach estimating 3D motion was proposed by Liu et al. [5], who presented the TLIO filtering framework including learned motion estimates from batches of IMU signals. An EKF was used to filter the full 6 DoF state and process updates were performed by traditional IMU integration. They trained a network with 40 h of pedestrian data to output displacement and covariance from the buffered IMU data. These measurements were provided to the EKF as relative position measure- ments and allowed the filter to correct for biases online. Their approach showed impressive results for IMU only navigation but was unable to integrate other sensors such as kinematics or vision.

---

Currently, the application of inertial learning to robotics is limited. Brossard et al. [13] demonstrated IMU-only state estimation specifically for wheeled vehicles, heavily exploiting the constrained mo- tion model. Zhang et al. [14] trained network to learn the noise parameters of the IMU opposed to the motion of the subject. Their RNN generates rotation, velocity and acceleration terms with bias removed, however this makes integration with other frameworks such as filtering more difficult.
3 Problem Statement
Our objective is to estimate the state of legged robot equipped with an IMU, joint sensors (en- coders, torque sensors) and, optionally, exteroceptive sensors. Following standard conventions, we define fixed world frame V aligned with gravity, and two frames attached to the robot: the base frame and the IMU frame I. The transformations between B and I are known. The robot's 's state at the current time xtts)
3g is defined follows:
(1) and includes the rotation Rus E SO(3) in world coordinates, the position *P.h E R the linear velocity иa and the gyro and accelerometer IMU biases ,b², ,,b". The IMU measures rotational velocity 1 E I33 and the proper acceleration , E R3. These are corrupted by zero-mean Gaussian noise 17", n7 and by slowly changing biases:
K [R&, PR, Vk, bg. bj]
(t) (t) (t) + (t) ā(t) (,a(t)) wg) + bt)) + (t))
(2)
where ug is the gravity vector in world frame. Without constraining measurements, the biases are not observable and simple IMU integration will quickly lead to runaway error.
4 Learned Displacement Measurement
In this section, describe our learning method including network architecture and loss functions. Let Zij be a sequence of N consecutive IMU measurements collected from time 1 to Ij and gravity aligned. From Lij. we want to predict the robot's linear position increment d, R between t; and t,. To this end, we define a nonlinear function Fa that will be approximated by a neural network:
Fo : (Zzz) + (d, ú;)
where ú; (ūT, ai, (t)) contains the logarithms of standard deviations from which the covariance of the displacement Σ E p3x3 can be computed:
S = diag (exp(2)"".exx((2")) exxp(2")).xx((2(3)
(4)
Following an approach similar to TLIO [5], approximate Fg with ID version of the ResNet18 architecture [15]. We also make use of two loss functions, but contrast to [5] we use robust Huber loss kernel to account for outliers in the training set.
4.1 Loss Function
For the first 10 epochs, the loss function is defined by the Mean Square Error (MSE):
C(d, a)
1 72
Σ
n
de d
(5)
where is the number of training samples, à is the network output, and d is the ground truth displacement. Once the network has begun to converge, the loss function changed modification of the Gaussian Maximum Likelihood (GML), which allows the network to learn un- certainty (as done in [16]) but without being over influenced by outliers. If the MSE loss is not initially used we find the network does not converge. The negative log-loss of the GML is given by:
robust
C(d, Sd d)
log
exp H(d: de. Σ3)
8T det Sid)
v
log det (Eg) + H(ds - dg. EX + const.
(6)
2
k
3

---

IMU
R;
Iij
Neural Network
de
Pronto
Encoders,
Robot
Torque Sens.
IMU Buffer
(400 Hz)
9j 9j. 7
Figure 2: EKF-IKD pipeline: IMU data is integrated in Pronto which provides an estimate of the robot's orientation. The IMU buffer is aligned with gravity and provided as input to the neural network. Displacement measurements and covariance are fused back into Pronto.
where H(ck, Σg is the sum of the Huber loss h((;,) applied to each axis of c dd - -d::
+ liek, of + h(e of) h(e,,) = { 2 if |d| < 6,
H(Es, Σ =
(7)
||c| 3 otherwise
We treat each axis separately because, shown in later sections, state estimation errors along the z-axis are often quite different from the errors on the zy-plane. Our loss function resulted in 12% less error computing displacements in test experiments compared [5].
5 State Estimation
This network outputs a relative position measurement and uncertainty which is suitable for incorpo- ration into both filtering and smoothing frameworks, as described the following sections.
5.1 Filtering Estimator
We incorporate the learned displacement measurement into Pronto, an open-source modular legged robot estimation framework [8]. As Pronto maintains history of states, measurements and co- variances, it can easily incorporate relative position measurements. The full system architecture is shown in Fig. 2: the IMU measurements from time t, to tj are accumulated and rotated be gravity aligned at time f; using the orientation R, from the filter's 's internal state. They are then passed to the network to estimate displacement and uncertainty. These are fused, using the covariance in Eq. 4, into the filter as a relative position measurement:
2, p: +d
where position p, at time i is taken from the filter's 's history, while displacement is from the network.
We call our proposed filtering method EKF-IKD (EKF with inertial, kinematic, and learned dis- placement). For all training and experiments, we used a buffer size N 400 (1s of 400 Hz IMU data). Displacement measurements that this overlap compromises independence of measurements but have not included analysis of this effect due space limits.
estimated at 20 Hz, which are overlapping. We recognize
5.2 Factor Graph Estimator
In this section, we describe a novel factor graph architecture that incorporates the Iearned inertial factor. In contrast to filters, factor graph methods re-optimize the full or partial history of states, given all the measurements connected to them, allowing for more accurate and smoother estimates.
Fig. 3 shows the proposed factor graph architecture. When new camera keyframe is available, new node (white circle) created. Similarly to [17], we connect consecutive nodes æ& and #*+1 with preintegrated IMU and leg odometry factors (blue and orange factors), while visual features are modeled as individual landmarks (gray circles) and factors (yellow factors). In visually degraded environments, the visual landmarks might not be observable for long period of time (e.g., from node 23 the example), so important to have robust proprioceptive odometry to fall back on. To this end, we exploit the novel learned displacement factor to further constrain the robot motion

---

Prior
--MU
Llg Odom.
Visual
Learned
VISION BLACKOUT
lo
Eo
1
22
33
AC
IS
Figure 3: Proposed factor graph framework with IMU factor from [18), kinematics factor from [17] and our learned factor which specially helps in visual blackouts. The learned factor connections are dictated by the rates the other sensors, as explained in Sec. 5.2.
(magenta line of Fig. 3). We call the factor graph implementation of our method FG-VIKD (factor graph with vision, IMU, kinematics and learned displacement).
The learned displacement measurement d, estimates the position of the robot between the start and end times of the buffer. The learned factor residual ra can then be computed using ΣS from Eq. 4:
ra = RT - PP) a
Σ
Fixed-lag optimization is done over the last 5 of states each time a node is inserted to the graph. Since the IMU buffer is longer than the period between two keyframes, the nodes connected by the learned factor are not consecutive and instead skip several nodes. In the typical setup the camera creates nodes at 20 Hz and the IMU buffer length is 1 s meaning the learned factor would connect æn to 22n, 21 to 22 etc. (for simplicity, every two nodes are connected in Fig. 3). We also integrate IMU measurements between optimizations to provide 400 Hz signal state estimate.
6 Lab Experiments
We performed several experiments with our ANYbotics ANYmal C100 robot as shown Fig. 4. The robot was teleoperated over different terrains in the lab: flat ground, soft ground, slippery surface and an obstacle. Ground truth poses were measured using Vicon motion capture system.
Flat Ground First, show performance on rigid, flat terrain. In these conditions, kinematic state estimate is very accurate, however over long trajectories there is upward drift (the Z dimension).
Soft Ground Deformable terrain poses a major challenge to legged robot state estimation as the specific moment of contact ill-defined the foot continues move and compress the terrain after contact is detected. This additional downward motion results in a hallucinated upward Z drift. Some realistic examples of deformable terrain include sand, mud or tall grass. To reproduce this effect we placed several layers of foam padding on the floor.
Slipping When robot slips, such as on wet or loose terrain, certain kinematic assumptions are violated as the foot is in contact with the terrain but does not have zero velocity. This results in large spikes in odometry error. To create slip events in the lab, we placed a common office white board on the floor and covered it with lubricant (hand sanitizer).
Obstacle Terrain Blind climbing designed to show that our method can generalize to motions on non-plane non-planar surfaces.
uneven obstacle course with ramps. This experiment was
Figure 4: Lab experiments. Left: Robot foot com- pressing into soft terrain. Middle: Slip event while walking slippery ter- rain. Right: Robot climb- ing over obstacle.
5

---

Table Experimental Results for Filter-basee methods.
Absolute Pose Error (APE) [m]
5 Relative Pose Error (RPE) [m]
Data
TLIO
TSIF
EKF-IK
EKF-IKD
TLIO
TSIF
EKF-IK
EKF-IKD
FLAT
1.88
0.14
0.42
0.20
0.58
0.09
0.19
0.16
SLIP.
0.28
0.21
0.38
0.20
0.35
0.13
0.20
0.12
SOFT
0.31
0.35
0.53
0.24
0.31
0.15
0.24
0.16
OBST.
0.37
0.19
0.39
0.27
0.44
0.12
0.22
0.22
6.1 Training
For the lab experiments, we collected ten walking sequences for each terrain totaling 2 2h16 min in 40 sequences. Four sequences were selected test experiments and the remaining were split 75:25 into training and validation datasets. We trained single model from all the data and used this same model in all experiments. We used the Adam optimizer with an initial learning rate of 10-1 and Huher delta of 1.35. Training lasted 200 epochs with the model minimizing validation error used in all lab experiments. Training took 6 hours on laptop with Nvidia Quadro P2000 GPU.
We used the Vicon system to collect training data. First, we remove the biases from raw IMU signal and inject back random bias values. In order estimate the true IMU biases, we create factor graph where each node has a prior pose factor from a Vicon measurement. IMU factors connect the nodes allowing the biases to be estimated in batch optimization for all Vicon measurements.
6.2 Filtering Results
We present results comparing filtering--ased proprioceptive estimator Pronto (which we hereafter refer to as EKF-IK) which is open-source and uses IMU and kinematics and our proposed method which incorporates IMU, kinematics and the learned displacement measurement EKF-IKD, also show the trajectory provided by the default state estimator for the ANYmal robot which call TSIF and uses IMU and kinematics as well as the cearning-based estimator TLIO.
We used two metries: absolute pose error (APE) and relative pose error (RPE) as defined in [19]. APE measures the amount of drift over an entire experiment. RPE measures the drift every X m traveled we used m for lab experiments and 10 m for the mine experiment. In all experiments, adding the learned measurement improved the state estimation significantly. As shown in Table 1, in comparison with EKF-IK, APE was reduced by 46 % on average (51 % for most challenging terrains: slipping and soft) and RPE was reduced by 22 % on average (37 % on challenging terrains).
Our proposed method out performs TLIO for every experiment which is expected we use kine- matics. The TSIF estimator outperforms our method situations where the terrain is rigid and stable but is par when slipping or on soft terrain. We attribute this to the years of fine-tuning by the company on this estimator for this particular robot. such, we make no claims of beating the sfate-of-the-art,, however we point to the significant improvement gains to EKF-IK from the addition of the learned measurement.
Flat Ground The results for the filtering methods on flat ground are shown on the left of Fig. 5. There is clear upward Z drift for TSIF and EKF-IK which significantly reduced using the learned measurement in EKF-IKD
Soft Ground The results for the filtering methods soft ground shown on the right of Fig. 5. The periods where the robot walked on the soft terrain are highlighted in gray. The upward Z drift
Table Experimental Results for Factor- Factor-Graph Graph methods.
Absolute Pose Error (APE) [m]
5 m Relative Pose Error (RPE) [m]
Data
FG-IK
FG-IKD
FG-IK
FG-IKD
FLAT
0.36
0.22
0.25
0.24
SLIP.
0.23
0.20
0.18
0.17
SOFT
0.44
0.39
0.28
0.23
OBST.
0.23
0.21
0.18
0.18

---

Stspage
Flat Ground
Soft Ground
Slipping
Obstacle
6.25
4.30
2.30
2.5
9.99
0.0
190
R99 (m)
9.25
0.20
044
3.15
0.30
3.50
Glme Es]
time
time Dx]
Lime Es]
Figure 5: Top: Position in W estimate comparison for flat, soft, slippery, and obstacle grounds. Note the upward drift for TSIF and EKF-IK on the soft (gray area) and slippery (yellow area) terrains.
Bottom: m Relative Pose Error (no data for first travelled).
for TSIF and EKF-IK is more pronounced but this is reduced in EKF-IKD, Along the x axis the performance is slightly worse indicating scale error introduced by the learned measurement but overall error is greatly reduced as shown the RPE plot.
Slipping As shown in Fig. 5, periods when the robot is slipping have high error. The addition of the learned measurement in EKF-IKD significantly reduces spikes in error, particularly 60 Our method does slightly better than TSIF in this experiment.
Obstacle Terrain This experiment demonstrates that method can generalize to non-planar ter- rain. The upper right plot of Fig. shows that EKF-IKD reduces drift while preserving true changes in elevation the robot climbs over the terrain. Initially, found that if the training dataset did not include examples over elevated terrain, the network assumed constant planar elevation. The resulting state estimator then ignored elevation gain and loss from climbing terrain.
6.3 Factor Graph Results
We present results comparing two factor graph-based proprioceptive estimators: FG-IKD which is our proposed factor graph implementation without vision data (simulating camera failure) and FG-IK which the same factor graph without vision or the learned displacement. Since there is no vision in these cases, nodes are created 40 Hz fixed frequency in sync with the IMU.
We demonstrate that by adding the learned factor this factor graph without any vision, APE was reduced by 18 % and RPE by 7 %, as shown in Table 2. The eтгоr in these factor graph methods is higher than the filtering methods because the robot state intentionally estimated at lower rate to allow visual data to be incorporated. Additionally. without landmarks, FG-IK cannot take advantage of the graph structure. Nevertheless, FG-IKD shows an improvement in proprioceptive estimation implying that if a robot's vision system were to fail, our method would reduce drift.
7 Field Experiment: Underground Mine
While the learned displacement measurement can improve proprioceptive state estimation, most deployments of robots vision systems. However, performance of these systems will degrade in challenging environments such as underground mines. In these cases, the learned displacement mea- surement support the vision-based state estimation system. To demonstrate this, performed field experiment in which we telcoperated our ANYmal robot through a decommissioned limestone
Table 3: Experimental results for the Mine Experiment.
Absolute Pose Error (APE) [m]
10 m Relative Pose Error (RPE)[m]
Data
TL.O
EKF-IKD
FG-VIK
FG-VIKD
TL.O
EKF-IKD
FG-VIK
FG-VIKD
MINE
3.37
5.43
4.63
4.09
0.66
0.49
0.45
0.35

---

Ground
FG-VIK
FG-VIKD
Start
RPE [m)
1.0
time [s]
Figure 6: Results from the mine experiment. Left: Top-down perspective of VG- vision-based factor graph and method FG-VIKD in W. Right: 10m RPE for both trajectories. We show
images from three times where the error spikes in the baseline FG-VIK due to poor visibility.
mine with large rooms and narrow passageways as shown in Fig. 1. The floor is mostly hard with flat and sloped sections, but thick layers of dust and loose debris create slippery areas.
In the mine the controller from [20] was used instead of the default controller. Thus new model was trained from lab data as described in Section 6.1, totaling 1 h23 min. Training data for the mine was collected entirely in the lab and the resulting model was successfully applied to the mine.
7.1 Mine Experiment Results
We compared classical factor graph-based state estimator VILENS [10]; which uses vision, IMU and kinematics (and hereafter we refer to as FG-VIK) to our method FG-VIKD which uses the same sensors as well as the learned displacement factor. In the mine visual sensing is severely affected, forcing FG-VIK to rely on its proprioceptive sensing and, in these situations, the learned factor can help keep error low. Our experiment lasted 20 minutes and over 472 m of walking.
Ground truth was estimated using the robot's 's lidar to localize in prior, high quality, point cloud map. Table shows that RPE was reduced by 22 % by adding the learned factor. We also show TLIO and EKF-IKD as baselines although they do not use cameras. The trajectory and error over time shown in Fig. 6 along with images from the front facing camera times where the drift rate spiked. These spikes coincide with periods of poor illumination and visual tracking failure in the dark environment. The robot lights reflected off nearby surfaces triggering the camera reduce its exposure making the image darker. The performance improvement would likely be reduced in rad- ically different terrain such sand or snow; however, this experiment demonstrates generalization from the artificial lab terrains to real-world environments.
8 Conclusion
In this paper, we introduced learned displacement measurement derived from inertial sensing which can greatly reduce state estimation drift for legged robots in challenging situations. We showed how this measurement can be used both filtering and factor graph frameworks, and that the greatest benefits are when the robot slips or the terrain is deformable. Extensive experiments with the ANYmal quadruped robot showed a reduction of 37 % relative pose error for Elter-based state estimation. We have also shown how the learned displaced measurement can reduce error in vision driven factor graph framework by 22 % in underground mine. video demonstrating these experiments is provided supplementary material.
For future work, inclusion of kinematic data into the network will be explored. Directly using the joint states as input to the network would likely require much more complex architecture that might not generalize to different robot models. To this end, we seek to incorporate derivative inputs from the robot model, such ground reaction forces or foot velocities.

---

Acknowledgments
This research has been conducted as part of the ANYbotics research community. It was part funded by the EU H2020 Project THING (Grant ID 780883) and a Royal Society University Research Fellowship (Fallon).
References
[1] H. Kolvenbach, D. Wisth, R. Buchanan, G. Valsecchi, R. Grandia, M. Fallon, and M. Hut- ter. Towards autonomous inspection of concrete deterioration in sewers with legged robots. Journal of Field Robotics, 37(8):1314-122) 2020. doi.httss:/doiiorr//0... h0tg//..000/r0b2199996
[2] M. Bloesch, M. Burri, H. Sommer, R. Siegwart, and M. Hutter. The two-state implicit filter recursive estimation for mobile robots. IEEE Robotics and Automation Letters, 3(1):573-580, 2018. doi: 10. (09/LLA.2017.77770
[3] M. Camurri, M. Fallon, S. Bazeille, A. Radulescu, Barasuol, D. G. Caldwell, and C. Sem- ini. Probabilistic contact estimation and impact detection for state estimation of quadruped robots. IEFE Robotics and Automation Letters, 2(2): 102--100, 2017. doi:10.1 109/L.R. 2017.2652491.
[4] R. Hartley, J. Mangelson, L. Gan, M. G. Jadidi, J. M. Walls, R. M. Eustice, and J. W. Grizzle. Legged robot statemestimation through combined forward kinematic and preintegrated contact factors. IEEE International Conference on Robotics and Automation (ICRA), pages 4422- 4429, 2018. doi: doi:10.110 109.///9...A.011 8460748.
[5] W. Liu, D. Caruso, Ilg, Dong, A. I. Mourikis, K. Daniilidis, V. Kumar, and J. Engel. TLIO: Tight learned inertial odometry, IEEE Robotics and Automation Letters, 5(4):5653- 5660, 2020. doi: 10i:10.11 (009//LA..22...007....
[6] C. Chen, X. Lu, A. Markham, and N. Trigoni. IoNet: Learning to cure the curse of drift in inertial odometry. 32nd AAAI Conference on Artificial Intelligence, AAAI 2018, pages 6468- 6476, 2018
[7] P. Lin, H. Komsuo, and D. E. Koditschek. Sensor Data Fusion for Body State Estimation in Hexapod Robot With Dynamical Gaits. IEEE Transactions on Robotics, 22( 2006.
[8] M. Camurri, M. Ramezani, Nobili, and M. Fallon. Pronto: A Multi-Sensor State Estimator for Legged Robots in Real-World Scenarios. Frontiers in Robotics and AI, 7, 2020. doi: 10.3389/frobt.2000 202..0000000
[9] R. Hartley, M. Jadidi, Gan, Huang, J. W. Grizzle, and R. M. Eustice. Hybrid Contact Preintegration for Houual-Inrtial-Conttae State Estimation Using Factor Graphs. IEEE International Conference on Intelligent Robots and Systems, pages 3790, 2018. doi: 10.1109/1R0S S0..DIROS.20188.888
[10] D. Wisth, M. Camurri, and M. Fallon. Robust Legged Robot State Estimation Using Factor Graph Optimization. IEEE Robotics and Automation Letters, 4(4):4507-4514, 2019. doi: 10.1 R0.11099RAA.20192233377
[11] D. Wisth, M. Camurr, S. Das, and M. Fallon. Unified multi-modal landmark tracking for tightly coupled lida-visual-inertia odometry. IEEE Robotics and Automation Letlers, 6(2): 1004-1011, Apr 2021. doi: 10. 109/rr.202113056338
[12] S. Herath, H. Yan, and Y. Furukawa. Ronin: Robust neural inertial navigation in the wild: Benchmark, evaluations, Automation (ICRA), pages 46-3155, 2020. doi: 10. 109///CAA40945 202.99996.20099686
methods. In IEEE International Conference Robotics and
[13] M. Brossard, A. Barrau, and S. Bonnabel. AI-IMU dead-reckoning. IEEE Transactions on Intelligent Vehicles, 5(4):585-595 2020. doi: 10. (09/TTVV2000.298077)
[14] M. Zhang, M. Zhang, Y. Chen, and M. Li. IMU data processing for inertial aided navigation: recurrent neural network based approach. IEEE International Conference on Robotics and Automation (ICRA), 2021.

---

[15] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In IEEE Conference Computer Vision and Pattern Recognition (CVPR), pages 770-778, 2016. doi: 10.1109/CVPR.2016.33
[16] R. L. Russell and C. Reale. Multivariate uncertainty in deep learning. IEEE Transactions Neural Networks and Learning Systems, 2021.
[17] D. Wisth, M. Camurri, and M. Fallon. Preintegrated velocity bias estimation to overcome contact nonlinearities in legged robot odometry. In IEEE International Conference Robotics and Automation (ICRA), pages 392-398, 2020.
[18] C. Forster, L. Carlone, F. Dellaert, and D. Scaramuzza. IMU preintegration on manifold for efficient visual-inertial maximum-a-posterionn estimation. Robotics: Science and Systems, 11, 2015. doi: 101.1.....00/....000000 2015.XI.000
[19] J. Sturm, N. Engelhard, F. Endres, W. Burgard, and D. Cremers. A benchmark for the evalua- tion of RGB-D SLAM systems. JEEE/RSJ International Conference on Intelligent Robots and Systems, pages 573-580, 2012. doi:10.1 109/IOOO 2012.6385773.
[20] J. Lee, J. Hwangbo, L. Wellhausen, V. Koltun, and M. Hutter. Learning quadrupedal locomo- tion over challenging terrain. Science Robotics, 5(47), 2020. doi: 10. 126/sciroboticcsaab55555.
10


Reviews:
[{"review_id": "zPK7c0bTs1Z", "reviewer": "Reviewer_je1b", "summary": "This paper proposes a data-driven estimator for the odometry estimation of a legged robot from IMU data. In particular, the authors propose to learn a 1-D CNN to predict the displacement vector from a history of IMU states. They demonstrate the benefit of their learning-based odometry estimation by combining it with odometry from leg kinematics in an EKF estimator and in a factor graph estimator with a visual blackout. The proposed method improves relative pose error by 37 % in challenging scenarios when compared to traditional kinematic-inertial estimators.\n", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Comments:\n1. The proposed method is quite simple and elegant and seems to substantially improve the results compared to the state of the art. Using a learned prediction model to predict the displacement vector from buffered IMU data is a good place to leverage data-driven priors\n2. Section 6.2, 6.3 and 7.1 do a great job of extensively evaluating the proposed method with comparisons to all the relevant non-vision baselines. in particular, the real-world evaluation is quite convincing.    \n3. Although I understand that the scope of the paper is limited to improving odometry with missing vision/lidar input, it is unclear to me how practical is a setup with completely missing vision. For example, it would be interesting to see the effective improvements when a learning-based Visual Odometry method [A] is also adding estimates to the factor graph from very noisy lidar/perception inputs. I suspect the gains would not be as much in that case. \n4. It would be interesting to see ablations that plot the pose error as the history length of IMU changes, and also see how the performance changes with the loss function used in [5] instead of the huber loss function described in Section 4.1 (eqn 7). \n\n[A] Wang, Sen, et al. \"Deepvo: Towards end-to-end visual odometry with deep recurrent convolutional neural networks.\" 2017 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2017.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "Although, I have some general concerns related to the specific setup of completely missing vision sensors, and the effective benefits of the proposed method with learning-based visual odometry which can potentially handle very noise perceptual inputs, the proposed method will still contribute to improved odometry since the contributions of the paper are complementary to my concern. That setup might reduce the performance gains of the proposed method, but my concern is more to understand what problems practically occur in the field of odometry estimation. I think the paper is quite strong in empirical demonstration of its results. I am inclined to accept the paper.     ", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "Fair: Some knowledge of the area"}, "extra_text": {"issues": "One modification that would help in understanding the different components of the proposed method would be to include ablations that plot the pose error as the history length of IMU changes and also see how the performance changes with the loss function used in [5] instead of the Huber loss function described in Section 4.1 (eqn 7). Although this is not critical for the acceptance of the paper, this should definitely be included in the final version of the paper. \n\n"}}, {"review_id": "h4L5_8buW6W", "reviewer": "Reviewer_twnh", "summary": "This paper proposes a learned state estimator that can be fused with kinematics and IMU measurements as well as factor graphs to estimate robot state over long experiments, without significant drift. Authors present experiments in a mine where the camera image clearly shows a lot of noise that would make estimation hard using visual features. In such cases, the learned estimator can augment visual estimation reduce the robot state estimation drift.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. The paper is well-written and the use of learned models especially in the presence of large visual noise is well motivated. \n2. The robot experiments are well-conducted and run over long episodes, increasing the confidence in the performance of the estimator.\n3. The paper presents ways of using the proposed approach with both extended kalman filters that take IMU and IK data, as well as factor graphs that can take visual data as input. This makes the method versatile and easy to adapt to new problems.\n\nWeaknesses:\n1. The proposed approach requires re-learning of the estimator on each terrain. This is a major shortcoming of the approach as the major advantage of a learned estimator would be generalization to unseen terrains, where the parameters of baseline filters cannot be hand-designed. There is no discussion of this in the paper.\n2. The proposed approach only estimates the x,y,z position of the robot, not the orientation. This is also a major shortcoming which is not discussed. I would suspect that in situations such as slipping, the orientation estimate goes inaccurate, but no data is provided.\n3. While the paper compares their approach against Pronto which uses IMU and IK information, they do not compare against any learning-based baselines. Since the proposed approach learns an estimator on the ground profile it is tested on, but the baselines do not, the comparison is not fair. Some learning based baselines that might make experiments more compelling are [a], [b], [c]. A simple baseline would even be to learn the dynamics model used in the EKF.\n[a] Liu, Wenxin, et al. \"TLIO: Tight learned inertial odometry.\" IEEE Robotics and Automation Letters 5.4 (2020): 5653-5660.\n[b] Kim, Youngji, et al. \"Unsupervised Balanced Covariance Learning for Visual-Inertial Sensor Fusion.\" IEEE Robotics and Automation Letters 6.2 (2021): 819-826.\n[c] Teng, Sangli, Mark Wilfried Mueller, and Koushil Sreenath. \"Legged Robot State Estimation in Slippery Environments Using Invariant Extended Kalman Filter with Velocity Update.\" arXiv preprint arXiv:2104.04238 (2021).\n\nDetailed comments:\n1. The introduction should clarify that the proposed estimator only applies to 3D positions and not orientations.\n2. Eq 1 and Eq 2 are mixing discrete and continuous time conventions, and variables k, t\n3. Why do authors use MSE to start and switch to GML loss? What was the motivation behind this choice?\n4. Is each dimension considered independent is Eq 7? Is this a reasonable assumption, especially in situations like slipping? Can authors motivate this choice?\n5. Why do authors ignore covariance estimates when predicting state in Eq 8? Could covariances be useful in fusing the estimates from filter and NN better than simply adding the displacements?\n6. Section 5.2 is not very well described. What happens between visual events? Is the past estimate of state held constant or is the EKF run on proprioceptive data?\n7. In Section 6.1, is the data of 2h16m per terrain or across all terrains? Is the same model fit to all terrains or one per terrain?\n8. What is the frame of reference for experiments in Figure 5, 6, etc. Is y-movement forward?\n9. The main advantage of the proposed approach is in keeping the z-direction drift. However, in the experiments the z remains mostly fixed (except on ramp, where TSIF outperforms the proposed approach). The NN could just be learning to keep the z fixed. Moreover, the performance does not look better in x and y directions.  More discussion on poor performance in x on soft ground would be useful.\n10. How different is the mine from lab experiments? Though the video is impressive, overall the generalization to the mine terrain from lab is unconvincing. Why do authors use a vision-based approach here? How does EKF-IKD perform in this setting? This section needs to be motivated better and the choice of controller, approach, etc be described in more details.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The paper presents an incremental approach to estimation which involves a trained network predicts displacements in 3D positions based on IMU history. Similar ideas are present in literature on 6-dof pose estimates, like [1]. The paper could benefit from learning based baselines, recommended in the main review.\n[1]  Silva do Monte Lima, João Paulo, Hideaki Uchiyama, and Rin-ichiro Taniguchi. \"End-to-end learning framework for imu-based 6-dof odometry.\" Sensors 19.17 (2019): 3777.", "technical_quality": "Fair: Significant additional evidence required to support the claims.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "2: Unsure, difficult to assess.", "reviewer_expertise": "Very good: Comprehensive knowledge of the area"}, "extra_text": {"issues": "Main issues:\n1. No learning-based baselines. Since approach trains a model on each terrain separately, it gives the approach significant advantage over baselines, making comparisons unfair.\n2. only a 3D pose estimate, not 6-dof estimates of state presented in the paper."}}, {"review_id": "UT2P9K4pkA", "reviewer": "Reviewer_VSob", "summary": "The paper introduces a neural network for state estimation of a legged robot. The NN outputs positional displacements and their corresponding uncertainty based on raw IMU data. These displacements are then integrated into prior methods for state estimation with multiple sensors to output a more robust pose (with reduced sensor drift). Experiments were conducted for challenging terrains (like slippery or soft surfaces) and in a real-world mine environment.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Integrating a learned approach for state estimation is a powerful idea that might improve the handling of challenging situations. However, the experimental results show that the proposed IKD methods perform worse than the default TSIF method. Nonetheless, the learned method improves the corresponding approach without learning, however only with very similar training and test data distribution. We think that the paper does not put enough emphasis on the limitations of learning: a possible distribution mismatch and the (probably constrained) generalization capability compared to a model-based approach.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "The paper has an interesting idea, however the evaluation and discussion should be extended to support the claims (e.g. “our approach greatly reduces the drift of ...” in the abstract).", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "reviewer_expertise": "Fair: Some knowledge of the area"}, "extra_text": {"issues": "- We conclude from the experimental results that the proposed method performs worse than related work: For example, in comparison between the proposed EKF-IKD and the default TSIF state estimator, the proposed method performs 40% worse on average (RPE) in the lab experiments (Table 1). The abstract states a reduction by 37% in comparison to a traditional kinematic-inertial estimator, but doesn’t that include TSIF? Apart from the drift of the z-axis, the TSIF also shows qualitatively better results in Fig. 5 and 6. Why are the best results of the factor graph-based methods highlighted, although they are just different methods for solving the same task? We strongly suggest adding a baseline for the mine experiment.\n- While it is not inherently bad that the results are not better than related work, we think that the paper should acknowledge and might discuss reasons for that. \n- Again, we think that the discussion does not emphasize the obvious disadvantages of the learned approach. We suggest adding experiments for out-of-distribution terrains (like surfaces with different softness) to show the robustness of the proposed work. In particular, the robustness is a key motivation of your work (and arguably very important for real-world usage of your state estimator). In robot learning, we find it very important to get a feeling about the limitations of the approach. Why doesn’t the learned approach fix the drift in the z-direction completly?\n- Why doesn’t the NN get the joint states q as input? We think that the NN could detect (and then correct) challenging situations in particular based on a mismatch between IMU and joint data.\n- We suggest clearing up Section 3. From our perspective, there is no real “problem statement” or task in there. What should be minimized or calculated? The biases b are not used furthermore (as they are learned). Between which points of time are the position increments d_i defined?\n- Fig. 7 (left side) should be cleared up, e.g. by combining all three trajectories into one figure. As no start point is mentioned, it might make sense to shift and rotate the trajectories by a constant offset first.\n- The title of the tables should be above.\n- To make room for additional discussion: Fig. 5 and 6 could be made smaller and combined into one, and Section 2 and 5 could be shortened.\n"}}, {"review_id": "TZVFoQLiJi", "reviewer": "Reviewer_Kasg", "summary": "The paper proposes a method to estimate the position and orientation of a legged robot by combining the IMU data, the robot kinematics and visual observations. A focus of the paper is on a drift-free inertial odometry that uses a recurrent neural network. Another is on the application of the factor graph in order to exploit observations of landmarks that might be intermittently available. The method is effective particularly in environments where firm contacts between the robot feet and the ground are not available and the vision guidance is hardly expected due to bad optical conditions.\n", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "It seems reasonable to apply the neural network to the inertial odometry that might be able to lower the effect of sensor drift as well as the accumulation of numerical errors, so that a more robust estimation than a naive integral of the acceleration is expected with it. On the other hand, it is suspected that the method reduces the accuracy if the sufficient number of training data under sufficient variety of situations are not provided. The results in Table 1 pretty matches this prediction, and thus, is not surprising for the reviewer.\n\nThe technique around the factor graph is not clearly explained. Although the reviewer understands that the original technique by the authors is the use of learned transitions of the robot states (magenta lines in Figure 3), he/she does not understand why it connects every other nodes. It would be strongly suggested that the authors carefully explain how the graph is constructed and how several hypotheses of transitions between nodes including that based on the learned process are combined.\n\nRegarding the results in Figure 7 (with a view of the supplement movie), the reviewer can hardly justify the claim that FG-VIKD is better than FG-VIK since the closeness of the latest positions of each trajectory to the ground truth seems to be frequently switched.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.", "technical_quality": "Fair: Significant additional evidence required to support the claims.", "clarity_of_presentation": "Good: Minor revision needed to improve organization or clarity.", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "issues": "To clearly explain the technique around the factor graph.\n", "reviewer_expertise": "Very good: Comprehensive knowledge of the area"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors propose a learning-based proprioceptive state estimator for legged robots using IMU data.  Comparison of the proposed approach to EKF and a factor-graph based estimators is presented.  As pointed out by the reviewers, the authors should discuss why they choose to ignore using the kinematic information (joint encoders) as input to the neural network.  Comparison of the method with other learning-based estimators should be presented. Failures of the proposed method should be discussed, particularly inference on out-of-distribution terrains (relearning on each terrain may not be feasible). 

Having gone through the author response to reviewer comments and discussion, I am recommending an accept with poster.  I commend the authors on carefully addressing all the reviewer comments.
Decision: Accept (Poster)