<figure><img src="image_1.png" /></figure>

HAL
open science

# BiLSTM Network-Based Extended Kalman Filter for Magnetic Field Gradient Aided Indoor Navigation

Makia Zmitri, Hassen Fourati, Christophe Prieur

<figure><img src="image_2.png" /></figure>

## To cite this version:

Makia Zmitri, Hassen Fourati, Christophe Prieur. BiLSTM Network-Based Extended Kalman Filter for Magnetic Field Gradient Aided Indoor Navigation. IEEE Sensors Journal, 2022, 22 (6), pp.4781-4789. 10.1109/JSEN.2021.3091862. hal-03425006

HAL Id: hal-03425006

https://hal.science/hal-03425006v1

Submitted on 25 Feb 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# BiLSTM Network-Based Extended Kalman Filter for Magnetic Field Gradient Aided Indoor Navigation

Makia Zmitri, Hassen Fourati, and Christophe Prieur

**Abstract**— This paper proposes an innovative method to estimate the velocity of a moving body. This is achieved using solely raw data from a triad of low-cost inertial sensors, i.e. accelerometer and gyroscope, as well as a determined arrangement of magnetometer array. The proposed approach combines a magnetic field gradient-based Extended Kalman Filter (EKF), with a Bidirectional Long Short-Term Memory (BiLSTM) network. This is to better estimate the velocity, especially when the magnetic field disturbances are low, which causes other magnetic field-based methods to be inaccurate. The proposed method also makes it possible to well update the velocity regardless of sensor location, without any heavy computation or complex tuning, as the case for the Zero-Velocity Update Technique (ZUPT). The performance of the proposed approach is demonstrated through real experiments data using a Magneto-Inertial Tachymeter (MIT). The obtained results show the efficiency of the velocity estimation and possibly position, for different sensor placements and trajectory scenarios.

**Index Terms**— Extended Kalman Filter, indoor navigation, inertial and magnetic sensors, Neural Networks, velocity estimation

<figure><img src="image_4.png" /></figure>

## I. INTRODUCTION

During the last few years, indoor navigation has motivated a lot of research, as it represents a crucial problem to many applications, with the absence of a Global Positioning System (GPS) coverage. Recently, an innovative solution [10], [11] has been proposed to deal with this issue. It suggests complementing low-cost inertial sensors (accelerometers and gyroscopes) with an array of spatially distributed magnetometers. The technique takes advantage of the magnetic field disturbances that are present indoor and whose variation patterns can change over time. This is in order to reconstruct velocity, and consequently position, without relying on any prior mapping. This is based on the relation between the magnetic field gradient and the velocity, that demonstrates, under some conditions, the observability of the latter. For instance, in [3], it was discussed that the velocity is observable and accurately reconstructed when the magnetic field gradient is large and non-singular. However, phases with a low magnetic field gradient or static moments, defined with constant attitude and zero velocity, lead to drifts in the ve-

locity estimation. In [14], a magnetic field gradient-based Extended Kalman Filter (EKF) was developed, to deal with this problem. The choice of using an EKF comes from its efficiency in solving non-linear estimation problems thanks to its convergence proof, its uncertainty modeling and its low computational cost. However, the proposed scheme fails at some time steps to accurately estimate the velocity, in presence of a low magnetic field gradient. These velocity estimation errors, as small as they may be, lead to a noticeable position drift after integration. One of the most common solutions in the literature that is used to correct these velocity estimation errors is the Zero-Velocity Update Technique (ZUPT) [5]. Nevertheless, this solution is only efficient in foot-mounted pedestrian applications where zero-velocity moments can periodically occur. This is considered user-unfriendly as such sensor placement is uncomfortable and exposed. The method also requires continuous tuning of its parameters [7] in order to adapt to various walking speeds and different subjects. Another solution that has recently emerged in the field of indoor navigation is the use of Artificial Intelligence (AI) [1] that has offered a new perspective to the research in this area. In fact, using neural networks, drifts in velocity estimation are handled and position is accurately reconstructed [4], [6]. Nevertheless, in order to obtain high accuracy results, using only an AI-based solution, a large training dataset is required, which can be difficult to collect and process.

This work has been partially supported by MIAI@Grenoble Alpes, (ANR-19-P3IA-0003).

M. Zmitri, H. Fourati and C. Prieur are with Univ. Grenoble Alpes, Inria, CNRS, Grenoble INP, GIPSA-Lab, 38000 Grenoble, France. E-Mails: makia.zmitri@gipsa-lab.fr; hassen.fourati@gipsa-lab.fr; christophe.prieur@gipsa-lab.fr.
This also leads to heavy computation that needs a lot of time and memory usage. For these reasons, relying entirely on AI is not used in this paper. Alternatively, the key idea of this paper is to enhance the EKF-based approach previously proposed in [14], using a light AI-based solution.

This paper is focused on the velocity estimation, as well as other states (attitude, position, etc.), especially when the EKF is inaccurate due to low magnetic field disturbances [14], and when the ZUPT is inapplicable. An innovative way to tackle this problem is then to intelligently combine a light training-based neural network with the previously proposed magnetic field gradient-based EKF [14]. A similar solution that has shown to be fruitful in the field of intelligent vehicles was proposed in [2], where authors use a Convolutional Neural Network (CNN) to dynamically tune the noise parameters of an Invariant EKF (IEKF) by also exploiting velocity pseudo-measurements. In [13], a LSTM-based EKF was developed and its performance was evaluated using an open source dataset called Openshoe [8]. In this paper, an extension of [13] is proposed to take into account its drawbacks and improve the results of estimation. The main contributions are:

* a BiLSTM network is used instead of the standard LSTM applied in [13]. This architecture represents a special case of Recurrent NNs (RNNs), which are known to be very appropriate for time-series data thanks to their feedback connections, that differentiates them from regular feedforward NNs (ANNs), or CNNs that are mostly used for image-based applications. The bidirectional addition to the regular unidirectional LSTM enables the network during training to preserve information from both the past and the future, contrarily to the LSTM that only takes into account previous information;

* the used dataset lacked measurements of a magnetometers array because only one magnetometer was used, which led to the simulation of imprecise data through approximations from a theoretical array. In this paper, real magnetometers array measurements are employed using a Magneto-Inertial Tachymeter (MIT) provided by Sysnav company [9];

* the Openshoe dataset corresponds to foot-mounted IMU measurements and one single trajectory. This is considered very limited and doesn't demonstrate the generality of the proposed approach. In this paper, different and more user-friendly sensor placements (e.g. pocket, waist, etc.) are tested with a variety of trajectory scenarios;

In this paper, it is demonstrated how the EKF-BiLSTM overcomes both the EKF and the EKF-ZUPT methods, for all sensor placements and trajectories. Whether the MIT is placed on the waist, pocket or ankle, and no matter the trajectory's form, the proposed approach provides the best estimation accuracy of the states, especially the velocity and position. The difference in accuracy between

the proposed EKF-BiLSTM and the other methods is mostly seen for the low magnetic field gradient case, for instance when the MIT is placed in the pocket, where no large magnetic field disturbances are captured and no zero-velocity moments are usually detected.

## II. DYNAMIC MODELING AND STATEMENT OF THE PROBLEM

This paper targets the accurate reconstruction of the velocity of a rigid body when it navigates in a magnetically disturbed indoor environment. At the same time, the estimation of attitude, the magnetic field and its gradient, are necessary to reach this goal. This paper discusses also the accuracy of position estimation based on the reconstructed velocity. To do so, measurements from low-cost inertial sensors and a magnetometers network are employed. In the following, the index b (resp. n) denotes the coordinates of vectors in the body frame $R_b$ (resp. in the inertial frame $R_n$).

### A. Magnetic field gradient aided navigation

The presence of magnetic field disturbances in indoor environments represents the key point for the velocity estimation. In [10], a new technique was developed and it demonstrated the relationship between the magnetic field gradient and the velocity based on the equation below

$$ \frac{dB_b}{dt} = -\omega_b \times B_b + \nabla B_b v_b \quad (1) $$

The following variables can be defined in this equation:

* the velocity of the moving body $v_b = [v_{bx} \ v_{by} \ v_{bz}]^\top \in \mathbb{R}^{3 \times 1}$ in $ms^{-1}$;

* the magnetic field $B_b = [B_{bx} \ B_{by} \ B_{bz}]^\top \in \mathbb{R}^{3 \times 1}$, measured by a 3-axis magnetometer in Gauss (G);

* the Jacobian matrix $\nabla B_b \in \mathbb{R}^{3 \times 3}$, which represents the magnetic field spatial gradient in $Gm^{-1}$. This matrix can also be mapped by a unique bijection to a vector in $\mathbb{R}^{9 \times 1}$. Note here that the magnetic field gradient is symmetric and traceless. This means that it can be described using only 5 of its elements while the rest can be deduced [12]. This is beneficial when applying the EKF, where 5 values are estimated instead of 9;

* the angular velocity $\omega_b = [\omega_{bx} \ \omega_{by} \ \omega_{bz}]^\top \in \mathbb{R}^{3 \times 1}$, measured by a 3-axis gyroscope in $rads^{-1}$.

From this equation, one can note that the observability of $v_b$ is ensured when $\nabla B_b$ is non-singular (see [11]). Besides, to accurately reconstruct $v_b$, magnetic disturbances are necessary in the surroundings of the moving body. This means that $\nabla B_b$ should be large enough. Also, if the subject is static (constant attitude and zero velocity), observability issues occur [3].

### B. Inertial and magnetic measurements-based dynamic model

The continuous-time equations representing the variation of quaternion, velocity and position, as well as the
magnetic field and its gradient [12], are summarized in the state-space model (2). These equations were derived taken into account the relation between the inertial and magnetic measurements and standard kinematics laws.

$$ 
\left\{ 
\begin{aligned}
\frac{dq}{dt} &= \frac{1}{2}[\omega_{bq}^{\times}]q \\
\frac{dv_b}{dt} &= -\omega_b \times v_b + a_b - Rg \\
\frac{dB_b}{dt} &= -\omega_b \times B_b + \nabla B_b v_b \\
\frac{d\nabla B_b}{dt} &= T_b v_b + \nabla B_b[\omega_b^{\times}] - [\omega_b^{\times}] \nabla B_b \\
\frac{dM_n}{dt} &= R^{\top} v_b
\end{aligned}
\right. 
\quad (2) 
$$

This dynamic model contains the following variables:

* the attitude matrix $R \in \mathbb{R}^{3 \times 3}$ that belongs to $SO(3)$, the group of rotations in 3D Euclidean space, and representing the rotation from $\mathbb{R}_n$ to $\mathbb{R}_b$. This matrix can be expressed in terms of a unit quaternion, denoted by $q = [q_0 \ q_1 \ q_2 \ q_3]^\top \in \mathbb{R}^{4 \times 1}$, such as $R = R(q)$;

* the skew-symmetric matrix of $\omega_b$, denoted $[\omega_b^\times]$;

* the angular velocity expressed in quaternion form $\omega_{bq} = [0 \ \omega_b^\top]^\top$ and its corresponding skew-symmetric matrix $[\omega_{bq}^\times]$;

* the acceleration $a_b = [a_{bx} \ a_{by} \ a_{bz}]^\top \in \mathbb{R}^{3 \times 1}$, measured by a 3-axis accelerometer in $ms^{-2}$. In this paper the effects of Earth's rotation and Coriolis acceleration are neglected, and the gravity vector $g = [0 \ 0 \ g_z]^\top \in \mathbb{R}^{3 \times 1}$ is considered known, where $g_z = 9.8 \ ms^{-2}$;

* the position $M_n = [M_{nx} \ M_{ny} \ M_{nz}]^\top \in \mathbb{R}^{3 \times 1}$ in $m$;

* the tensor $T_b \in \mathbb{R}^{3 \times 3 \times 3}$ representing the first spatial derivative of $\nabla B_b$ in $Gm^{-2}$ [12]. Alternatively, this tensor is defined through a bijection as a matrix in $\mathbb{R}^{9 \times 3}$.

This model is used to design the magnetic field gradient-based EKF [14], where the state vector is $X = [q \ v_b \ B_b \ \nabla B_b \ M_n]^\top \in \mathbb{R}^{18 \times 1}$, and its corresponding estimate is $\hat{X}$. The input vector is $u = [\omega \ a_b \ T_b]^\top \in \mathbb{R}^{13 \times 1}$, and the output (measurement) vector is $y = [B_b \ \nabla B_b]^\top \in \mathbb{R}^{8 \times 1}$.

## C. Limitations of the magnetic field gradient-based EKF for velocity estimation

As discussed earlier, in order to fully reconstruct $v_b$, there should be large magnetic disturbances surrounding the moving body. In close to singular/low gradient conditions, the velocity suffers from unbounded estimation errors as discussed in [3]. A magnetically disturbed environment generates a magnetic field gradient which eigenvalues are around 0.05 $Gm^{-1}$. In case gradient is really lower, the performance of the EKF may decrease depending on the system dynamics. This phenomenon is illustrated by using an embedded MIT (see description

in Section IV-A) that is placed on the back pocket of a subject. The latter performs a circular trajectory in two different locations: first inside an office where there are sufficient magnetic disturbances, due to laptops, electric currents, etc. Second, along the building entrance/hallway, where there are less magnetic disturbances. In Fig. 1, eigenvalues of $\nabla B_b$, denoted $\lambda$, are plotted for the two tested locations, as they represent a direct evaluation on the presence of magnetic field disturbances [3]. As expected, the magnetic field gradient in the hallway is significantly lower than the one inside the office. Fig. 2

<figure><img src="image_5.png" /><figcaption>Fig. 1: Eigenvalues of the magnetic field gradient for the two locations</figcaption></figure>

displays the reconstructed trajectory using the magnetic field gradient-based EKF in both scenarios. Knowing that the performed trajectory is a circle, it is easy to conclude that the EKF performance is degraded under low magnetic interference ($\lambda \ll 0.05 \ Gm^{-1}$).

<figure><img src="image_6.png" /><figcaption>(a) EKF trajectory in the office (b) EKF trajectory in the hallway</figcaption></figure>

<caption>Fig. 2: Magnetic field gradient effect on the EKF estimation [14]</caption>

In this paper, an innovative approach is proposed to improve the velocity estimation, especially under low magnetic disturbances.
## D. Limitations of the ZUPT for velocity correction in EKF

The ZUPT is the most common method that is used in foot-mounted applications to continuously correct the velocity estimate by resetting its value to zero each time a stance phase (the entire period during which the foot is on the ground) is detected. This is done given the measurements $a_b$ and $\omega_b$ and through a binary hypothesis testing problem [7] that determines whether the MIT is stationary or not. The problem with the ZUPT is that in case the MIT is placed on any other body location beside the feet, it can either suffer from false detections, or not detect any zero-velocity instances. Its behavior is highly dependent on the chosen window size by the user, the detection threshold, the nature of the detector, etc. To illustrate these issues, an experiment is conducted where an EKF-ZUPT [14] is applied. Two databases are used, each one corresponding to a trajectory with a triangular shape. One with the MIT placed on the waist, and another with the MIT placed on the ankle. The Stance Hypothesis Optimal Detector (SHOE) [7] is chosen, the window size is set to be equal to 3 and the detection threshold is $0.03 * 10^5$. These values are inspired by the approach proposed in [8]. In Fig. 3, instances where the ZUPT detector indicates a zero-velocity are plotted for both the studied MIT placements. For instance, in Fig. 3a, when

<figure><img src="image_7.png" /><figcaption>Fig. 3: The velocity norm in a non foot-mounted framework (blue line) and the instances where the ZUPT detector indicates a zero-velocity (red dots)</figcaption></figure>

the MIT is placed on the waist, the ZUPT succeeds to accurately detect the zero-velocities at the beginning and the end of the trajectory (circled in green). However, it also detects other zero-velocity moments during the trajectory despite that the velocity norm is not necessarily equal to zero. On the other side, in Fig. 3b, it is expected that much more zero-velocity moments are detected as this sensor placement is nearly similar to the one on the foot. However, the ZUPT only detects the static phases at the beginning and the end of the walking scenario (circled

in green). These false and/or undetected instances affect the accuracy of the EKF-ZUPT, which is demonstrated in Section IV. These results can vary to some extent with a different tuning of the ZUPT window and detection threshold parameters, but the same conclusions are made. The ZUPT cannot correct the velocity errors generated by the EKF of [14] in the case of low magnetic disturbances and when the MIT is placed somewhere other than the foot. In this paper, a new estimation algorithm is suggested when the ZUPT is not efficient. It is applicable to any sensor placement and is tuning-free.

## III. DEEP NEURAL NETWORK-BASED EKF DESIGN

In this paper, an innovative solution is proposed, that combines the EKF of [14] with an AI-based solution, to improve the performance of estimation, especially in the low magnetic field gradient case. Also, the solution is not dependent on any particular sensor location, as the case for the ZUPT. It applies on any trajectory scenario and is tuning-free. Usually, AI-based solutions are computationally heavy, as they rely on large datasets and a long training procedure to provide accurate prediction results. For instance, in [4], the proposed method combining a CNN and a BiLSTM, requires 9 hours of training for each parameter set, for a 800 minutes of data recordings. The proposed approach on the contrary, is not dependent on any heavy training, as it relies majorly on the magnetic field gradient-based EKF. The BiLSTM used in this approach, is trained with a very limited dataset (only 14 minutes of data recordings) and good estimation results are achieved.

### A. General overview of the BiLSTM-based EKF

A preliminary study of the neural network-based EKF was achieved in [13] and is represented in Fig. 4. It appears from the literature that the BiLSTM is very suitable for time series data, and it works even with limited training. The three main blocks that form the proposed approach

<figure><img src="image_8.png" /><figcaption>Fig. 4: Proposed EKF-BiLSTM estimation approach of $\hat{X}$</figcaption></figure>

are detailed as follows:

1) after being trained using the reference velocity, the BiLSTM network (green block) generates a velocity estimate $v_{BILSTM} = [v_{bx,BILSTM} \ v_{by,BILSTM} \ v_{bz,BILSTM}]^\top \in \mathbb{R}^{3 \times 1}$. The generated $v_{BILSTM}$ represents a pseudo-measurement that is added to the output vector of
the magnetic field gradient-based EKF (blue block), such as $y = [B_b \nabla B_b v_{BiLSTM}]^\top \in \mathbb{R}^{11 \times 1}$.

2) a measurement covariance adapter (yellow block) controls the level of confidence given by the EKF to this pseudo-measurement. This is by dynamically tuning the noise elements corresponding to $v_{BiLSTM}$ in the measurement covariance matrix $N = \text{diag}(N_1, N_2, N_3) \in \mathbb{R}^{11 \times 11}$, where $N_{1<i<3}$ are the measurement covariance matrices corresponding to $B_b, \nabla B_b$ and $v_{BiLSTM}$ respectively. In this paper, the tuning concerns the elements of $N_3 \in \mathbb{R}^{3 \times 3}$. Regarding $N_1$ and $N_2$, their values are fixed, taking into account the MIT noise characteristics.

3) The EKF uses the continuous-time dynamic model in (2), inertial and magnetic measurements, $v_{BiLSTM}$ and the determined $N_3$, to estimate $\hat{X}$.

## B. Magnetic field gradient-based EKF

The state-space model (2) can be written in the form of this nonlinear system

$$ \left\{ \begin{aligned} \dot{X}(t) &= f(X(t), u(t), \eta(t)) \\ y(t) &= h(X(t), \nu(t)) \end{aligned} \right. \qquad (3) $$

where $X(t)$ is the state vector at time $t$, $y(t)$ is the known output vector (measurement vector), $u(t)$ is the input, $f(.)$ is a nonlinear function that represents the state-space model, $h(.)$ is a nonlinear function that represents the measurement model, and $\eta(t)$ and $\nu(t)$ are the process and measurement noises, respectively, assumed to be zero-mean, white, Gaussian and uncorrelated.

Implementing the EKF requires the linearization of (3) around the current estimate $\hat{X}(t)$, by computing Jacobians of $f(.)$ and $h(.)$, at each time step, as follows

$$ F = \frac{\partial f}{\partial X}|_{\hat{X}(t), u(t)}, \quad H = \frac{\partial h}{\partial X}|_{\hat{X}(t)} \qquad (4) $$

This leads to the following form of the state matrix $F \in \mathbb{R}^{18 \times 18}$

$$ F = \begin{pmatrix} \frac{1}{2}[\omega_{bq}^{\times}] & 0_{4 \times 3} & 0_{4 \times 3} & 0_{4 \times 5} & 0_{4 \times 3} \\ A_1 & -[\omega_b^{\times}] & 0_{3 \times 3} & 0_{3 \times 5} & 0_{3 \times 3} \\ 0_{3 \times 4} & \nabla B_b & -[\omega_b^{\times}] & A_2 & 0_{3 \times 3} \\ 0_{5 \times 4} & A_3 & 0_{5 \times 3} & A_4 & 0_{5 \times 3} \\ A_5 & A_6 & 0_{3 \times 3} & 0_{3 \times 5} & 0_{3 \times 3} \end{pmatrix} \qquad (5) $$

with $A_1 = \frac{\partial(-Rg)}{\partial q} \in \mathbb{R}^{3 \times 4}$, $A_2 = \frac{\partial(\nabla B_b v_b)}{\partial \nabla B_b} \in \mathbb{R}^{3 \times 5}$,

$A_3 = \frac{\partial(T_b v_b)}{\partial v_b} \in \mathbb{R}^{5 \times 3}$, $A_4 = \frac{\partial(\nabla B_b[\omega_b^{\times}] - [\omega_b^{\times}] \nabla B_b)}{\partial \nabla B_b} \in \mathbb{R}^{5 \times 5}$,

$A_5 = \frac{\partial(R^\top v_b)}{\partial q} \in \mathbb{R}^{3 \times 4}$ and $A_6 = \frac{\partial(R^\top v_b)}{\partial v_b} \in \mathbb{R}^{3 \times 3}$.

Zero matrices and their dimensions are represented with $0_{i \times j}$, where $i$ corresponds to the number of rows and $j$ to columns.

The output vector is $y = [B_b \nabla B_b v_{BiLSTM}]$. This means

that the measurement matrix $H \in \mathbb{R}^{11 \times 18}$ can then be expressed as

$$ H = (0_{11 \times 4} \quad I_{11} \quad 0_{11 \times 3}) \qquad (6) $$

where $I_{11}$ is the identity matrix in $\mathbb{R}^{11 \times 11}$.

The state-space model represented above is the continuous-time version of the proposed system. During implementation, a discretization step is undertaken using the Runge-Kutta 4th order method. This implicates that $f(.)$ in (4) is replaced with $f_d(.) = RK4(f)$. The output model, however, is not discretized, as the MIT already provides discrete-time measurements via a digital processor. Thus, $y(t)$ is actually equivalent to $y(k) = h(X(k), \nu(k))$ and the output matrix $H$ is kept the same. The discretization step and the standard EKF equations are not further detailed in this paper.

## C. Measurement covariance adapter

In [13], it was demonstrated that the predicted $v_{LSTM}$ from the trained LSTM network is accurate at moments when the velocity is close to zero. See for more details [13]. The network also doesn't require heavy computation. This means that the LSTM prediction can be used to correct the EKF's velocity estimate during these moments (the same conclusions are made for a BiLSTM-based model). As mentioned in Section II-C, the instances with low gradient and/or close to zero velocity, affect the states' observability in the EKF [14], and then a drift in the estimated velocity is seen. It is proposed to add $v_{BiLSTM}$ to the EKF as a pseudo-measurement and control the times when it is taken into account to improve significantly the estimation accuracy. This is ensured by the measurement covariance adapter in Fig. 4 that dynamically controls $N_3$, which is the covariance matrix of the measurement $v_{BiLSTM}$. Such adaptation is based on Algorithm 1 and it works in two steps. First it assigns a large number to

EKF initialization;

for $i = 1$: length (testdata) do

$$ \left| \begin{array}{l} N_{3,i} = \alpha \mathbb{I}_3; \\ \text{EKF prediction;} \\ \text{if } v_{BiLSTM,i} \le \epsilon \text{ then} \\ \quad | N_{3,i} = \mathbb{I}_3; \\ \text{end} \\ \text{EKF update;} \end{array} \right. $$

end

Algorithm 1: Measurement covariance adapter in the EKF

$N_3$, using a large multiplication factor $\alpha \in \mathbb{R}_{>0}$. This is in order to give more confidence to the EKF estimation than the one of the BiLSTM. Second, the algorithm checks if $v_{BiLSTM}$ is below an adequate threshold $\epsilon \in \mathbb{R}_{>0}$. In that case, the adapter attributes the identity matrix to $N_3$ in order to trust more the BiLSTM prediction.
# IV. EXPERIMENTAL SETUP AND RESULTS

To evaluate the performance of the proposed approach, an experimental testbench is implemented. Inertial and magnetic measurements from a sensor module are used. Ground truth data is acquired from a motion capture system, and different comparisons are conducted.

## A. Magneto-Inertial Tachymeter (MIT) module

The used sensor is a Magneto-Inertial Tachymeter (MIT) provided by SYSNAV company [9]. The tachymeter consists of two components: a sensor module containing the different sensors (3-axis accelerometer, 3-axis gyroscope, and a planar arrangement of five 3-axis magnetometers: one in the center of the MIT and four around it), and a fusion module containing the battery and a Secure Digital (SD) card to save the measurements recorded by the sensor module.

## B. Reference system and data acquisition

All experiments are conducted in GIPSA-Lab, that is equipped with a motion capture system from Qualisys https://www.qualisys.com/. The room has 9 cameras that provide a high accuracy of ground truth position (1 mm) as well as ground truth attitude (Euler angles and rotation matrix). All processing, from calibration of the cameras, to the extraction of data, is handled using the Qualysis Track Manager (QTM) software. The MIT sensor is attached to a markers-based frame as shown in Fig. 5, such that its motion can be captured by the cameras during the recordings. The evaluation is performed with three sensor

<figure><img src="image_9.png" /><figcaption>Fig. 5: The MIT module with the markers-based frame</figcaption></figure>

placements on a subject: right ankle, right front pocket, and waist. Five trajectories, achieved by the subject, are examined: square, circle, diagonal rectangle, triangle and heart. Trajectories are carried out under normal walking, and they last 3.5 minutes on average each. The trajectories are conducted in the middle of the room in about 9 m² of surface. After recording the different trajectories, ground truth and MIT data are extracted and a synchronization procedure is conducted.

## C. Implementation details of the EKF-BiLSTM

The implementation of the EKF-BiLSTM approach is done under Matlab. The training and prediction of the

BiLSTM network is undertaken using the deep learning toolbox. The network starts with a sequence input layer of 9 features, representing the MIT raw data of acceleration, angular velocity, and magnetic field. Then, two BiLSTM layers containing both, 100 hidden units. Each BiLSTM layer is followed by a dropout layer with a probability of 0.25. Next, a fully connected layer to the velocity $v_{BiLSTM}$. Finally, a regression layer that computes the Mean Squared Error (MSE) between the reference velocity from the motion capture system and the BiLSTM predicted one. In fact, the ground truth position from the Qualysis system is used to calculate the corresponding ground truth velocity. The used optimizer for the training process is ADAM, with an initial learning rate of 0.0015, and the model is trained for 500 epochs. For each sensor placement, four trajectories are used to construct the training set, and the remaining one is for the testing. Note that around 1% of the training set is left for validation. The training takes less than 1 hour on a CPU (Intel Core i7 @1.9 GHz). This procedure is repeated 5 times so each trajectory can form the testing set one time. It is underlined that the training set remains very small ($\approx 14$ minutes, 9 features and $\approx 200000$ samples) compared to the state-of-the-art works [4] that used very large training databases.

## D. Effect of the measurement covariance adapter

In order to demonstrate the effect of the measurement covariance adapter used in [13] on the estimation process, an experiment is conducted. A walking rectangular trajectory is recorded with the MIT placed on the right ankle of a subject. Then, the proposed EKF-BiLSTM approach is applied with and without the measurement covariance adapter. The value of the multiplication factor of the adapter needs to be really high, so when it's assigned to $N_3$, the BiLSTM prediction is completely rejected, thus it is chosen such that $\alpha = 10^6$. The threshold needs to be lower than the velocity of an average movement ($\approx 1$ ms⁻¹), to represent a quasi-static state, without being equal to zero, thus it is chosen $\epsilon = 0.2$ ms⁻¹. In Fig. 6, the results of the velocity estimation corresponding to this experiment are plotted. To ease the reading, only the first axis $v_{bx}$ is plotted for about 10 seconds from the total time of the trajectory. The green line signal corresponds to when the pseudo-measurement $v_{BiLSTM}$ is trusted the entire time by the EKF. This means that no dynamic adaptation of $N_3$ is taken into account and its values are rather fixed to the identity matrix. In this case, the inaccurate prediction of $v_{BiLSTM}$ during moments when the velocity is higher than the threshold $\epsilon$ affects the velocity and position estimations. On the contrary, the blue line signal is found when Algorithm 1 is applied. A better velocity estimation accuracy is then observed after controlling the level of confidence given to $v_{BiLSTM}$ by the EKF using the measurement covariance adapter. In Fig. 7, the position reconstruction results are displayed. The same conclusions are made. Note here that, to ease the reading,
<figure><img src="image_10.png" /><figcaption>Fig. 6: Velocity reconstruction with the EKF-BiLSTM. The ground truth (red line) and the EKF-BiLSTM estimation with the measurement covariance adapter (blue line) and without it (green line)</figcaption></figure>

only the *x* and *y* axes of $M_n$ are plotted. This applies on all figures related to position estimation. However, all numerical results (such as in Table I, for example) take into account the 3 axes of the position vector $M_n$.

<figure><img src="image_11.png" /><figcaption>Fig. 7: Trajectory reconstruction with the EKF-BiLSTM. The ground truth (red line) and the EKF-BiLSTM estimation with the measurement covariance adapter (blue line) and without it (green line)</figcaption></figure>

Table I represents the distance error $d_e$ between the proposed EKF-BiLSTM position estimate and the ground truth one for both cases, with and without the measurement covariance adapter. It is defined such as

$$d_e = \| \hat{M}_n - M_n \| \quad (7)$$

It is observed that a better estimation accuracy is obtained when the measurement covariance adapter is used.

<table><caption>TABLE I: Distance error using the EKF-BiLSTM with and without the measurement covariance adapter</caption><thead><tr><th>d<sub>e</sub> (m)</th><th>Mean</th><th>Median</th><th>Max</th><th>Standard deviation</th></tr></thead><tbody><tr><td>Without the adapter</td><td>1.93</td><td>1.87</td><td>3.88</td><td>1.20</td></tr><tr><td>With the adapter</td><td>0.43</td><td>0.45</td><td>0.71</td><td>0.15</td></tr></tbody></table>

Note here that the proposed approach can be used to reconstruct any trajectory, and is not dependent on the sensor placement. Unlike the ZUPT, this approach is not

only detecting zero-velocity instances, but also those that are close to zero-velocity. In fact, it is certain that the velocity goes below the threshold $\epsilon$ at some point, for any trajectory and sensor placement. This means that the proposed approach is not limited to only foot-mounted applications. It also doesn't require any particular parameterization that is dependent on the walking pace or the morphology of the moving body. It is stressed here that $\epsilon$ is maintained the same for all trajectories and sensor placements, which makes the proposed approach tuning-free.

## E. Main results and performance of the EKF-BiLSTM

In this section, the estimation results from the proposed approach are presented. Comparisons are done with ground truth data. Results from only using the magnetic field gradient based-EKF (with and without ZUPT updates) are also displayed.

1) Magnetic field gradient and sensor placements: During this first experiment, a triangular trajectory is conducted for each sensor placement. Then, the magnetic field gradient is calculated using the magnetometers array from the MIT. In Fig. 8, the three eigenvalues of $\nabla B_b$ for the different MIT positions are plotted. What is demonstrated

<figure><img src="image_12.png" /><figcaption>Fig. 8: Eigenvalues of the magnetic field gradient for different sensor placements: ankle, waist, and pocket</figcaption></figure>

through Fig. 8 is that the magnetic field variations are more significant when the MIT module is placed on the ankle (red line). This can be explained by the metal structure of the floor in the motion capture room, which generates noticeable magnetic disturbances. The room also contains office equipment (laptop, metallic table, etc.) that produces significant magnetic disturbances. Yet, in order for these variations to be captured, the subject wearing the MIT (in the pocket or waist placements) needs to move close to this equipment. Otherwise, the recorded magnetic field gradient will not be as large as the one of the ankle placement. However, during all experiments, the subject was moving in the middle of the room and not very close to this equipment. For this reason, the pocket and waist sensor placements do not capture such high magnetic variations. It follows that the estimation results of the different compared algorithms (EKF, EKF-ZUPT
and EKF-BiLSTM) are most accurate for the ankle sensor position, as it has the largest magnetic disturbances. This will be detailed in the next sections.

2) **EKF versus sensor placements**: The estimation results of the magnetic field gradient-based EKF that is proposed in [14] are presented. This is in order to confirm that for different sensor placements, the estimation accuracy varies. Two sensor placements are chosen for comparison: ankle and waist. The same triangular trajectory as in Subsection IV-E.1 is considered. Fig. 9 displays a

<figure><img src="image_13.png" /><figcaption>Fig. 9: Position estimation from the EKF [14] when the MIT is on the waist and ankle. In red is the ground truth trajectory from the motion capture system, and in blue is the EKF estimated one</figcaption></figure>

comparison between the estimated trajectory from the EKF and ground truth, for the two sensor positions. From this figure, it is clear that the EKF provides better accuracy when the MIT is subject to enough magnetic field disturbances. This was expected as the EKF is highly sensitive to the magnetic field gradient as demonstrated in Section II-C. Then, in the next subsection, the results show how the proposed EKF-BiLSTM improves the estimation process when magnetic disturbances are low, for example when the MIT is on the waist. It is underlined here that adding the BiLSTM to the EKF in the case of high magnetic disturbances (MIT on the ankle) does not drastically improve the results from the case where only the EKF is used. In other words, the biggest impact of the BiLSTM algorithm on the magnetic field gradient-based EKF takes place when the EKF cannot perform well, under a low magnetic field gradient.

3) **EKF-BiLSTM results**: In this subsection, the performance of the EKF-BiLSTM approach is studied. First, in Fig. 10, the velocity estimation results are plotted for both the MIT placements (waist and ankle, respectively). To ease the reading, only about 60 seconds of each signal is plotted. Also, the axis $v_{bx}$ is chosen to be plotted. The other two axes have the same behavior. A comparison between using the magnetic field gradient-based EKF and the EKF-BiLSTM is undertaken. It is clear that the proposed approach outperforms the use of only the EKF, in both cases of lower and higher magnetic disturbances (waist and ankle positions). The difference in estimation accuracy between the EKF and EKF-BiLSTM is clearer

in Fig. 10a, which proves the efficiency of the proposed approach, especially in low magnetic field gradient situations. Such improvement on the velocity estimation by the EKF-BiLSTM impacts the position reconstruction for the studied triangular trajectory. In Fig. 11b, it is seen that for the case where there are enough magnetic disturbances, the EKF-BiLSTM succeeds to accurately estimate the trajectory with a small error. It also improves greatly the results when the MIT is on the waist (in Fig. 11a), by reducing the huge drift previously seen in Fig. 9a.

<figure><img src="image_14.png" /><figcaption>Fig. 10: Velocity estimation from the EKF-BiLSTM when the MIT is on the waist and ankle. In red is the ground truth velocity deduced from the motion capture system, in green is its estimate when using the EKF and in blue is when using the proposed EKF-BiLSTM</figcaption></figure>

<figure><img src="image_15.png" /><figcaption>Fig. 11: Position estimation from the EKF-BiLSTM when the MIT is on the waist and ankle. In red is the ground truth trajectory from the motion capture system, and in blue is the EKF-BiLSTM estimated one</figcaption></figure>

4) **Comparisons and discussion**: In Tables II and III, the results from all discussed methods are presented. For each sensor placement, the five trajectories mentioned in Section IV-B are tested and the mean of their results is presented. Table II displays the Root Mean Square Error (RMSE) and the Mean Absolute Error (MAE) of the estimated velocity with respect to the ground truth one. Table III contains the mean, median, maximum and Standard Deviation (STD) of the distance error between the true and the estimated position. By looking at Tables II and III, the proposed EKF-BiLSTM approach gives the best velocity and position estimation results
compared to the other methods (EKF and EKF-ZUPT). The advantage of the EKF-BiLSTM method is mostly seen on the pocket position, where the velocity and position estimates are highly better than the rest of the methods.

<table><caption>TABLE II: Velocity error mean of the different sensor placements</caption><thead><tr><th></th><th>Velocity (ms<sup>-1</sup>)</th><th>RMSE</th><th>MAE</th></tr></thead><tbody><tr><td rowspan="3">Waist</td><td>EKF</td><td>0.282</td><td>0.214</td></tr><tr><td>EKF-ZUPT</td><td>0.29</td><td>0.212</td></tr><tr><td>EKF-BiLSTM</td><td>0.114</td><td>0.074</td></tr><tr><td rowspan="3">Pocket</td><td>EKF</td><td>0.206</td><td>0.166</td></tr><tr><td>EKF-ZUPT</td><td>0.21</td><td>0.164</td></tr><tr><td>EKF-BiLSTM</td><td>0.106</td><td>0.078</td></tr><tr><td rowspan="3">Ankle</td><td>EKF</td><td>0.164</td><td>0.116</td></tr><tr><td>EKF-ZUPT</td><td>0.152</td><td>0.102</td></tr><tr><td>EKF-BiLSTM</td><td>0.088</td><td>0.05</td></tr></tbody></table>

<table><caption>TABLE III: Distance error mean of the different sensor placements</caption><thead><tr><th>d<sub>e</sub> (m)</th><th></th><th>Mean</th><th>Median</th><th>Max</th><th>STD</th></tr></thead><tbody><tr><td rowspan="3">Waist</td><td>EKF</td><td>5.982</td><td>5.874</td><td>11.95</td><td>3.668</td></tr><tr><td>EKF-ZUPT</td><td>5.79</td><td>5.624</td><td>11.61</td><td>3.528</td></tr><tr><td>EKF-BiLSTM</td><td>3.65</td><td>3.516</td><td>7.254</td><td>2.176</td></tr><tr><td rowspan="3">Pocket</td><td>EKF</td><td>11.35</td><td>11.15</td><td>23.28</td><td>7.154</td></tr><tr><td>EKF-ZUPT</td><td>11.29</td><td>11.17</td><td>23.15</td><td>7.094</td></tr><tr><td>EKF-BiLSTM</td><td>3.726</td><td>3.662</td><td>7.312</td><td>2.204</td></tr><tr><td rowspan="3">Ankle</td><td>EKF</td><td>2.402</td><td>2.288</td><td>5.216</td><td>1.456</td></tr><tr><td>EKF-ZUPT</td><td>2.072</td><td>1.966</td><td>4.53</td><td>1.42</td></tr><tr><td>EKF-BiLSTM</td><td>1.356</td><td>1.356</td><td>2.286</td><td>0.542</td></tr></tbody></table>

This proves again that the considered approach is very beneficial in the case where the magnetic field gradient is low. It is also important to highlight that the difference in accuracy between the tested algorithms is less remarkable for the ankle case. This is because the EKF can already perform well alone, due to the presence of a strong magnetic field gradient like explained in Section II-C. Note here that results between the EKF and the EKF-ZUPT are very similar, which proves again as demonstrated in Section II-D that the ZUPT is inefficient outside of the foot-mounted framework. This makes the proposed EKF-BiLSTM stand out, as it can be applied on any sensor placement, and can perform well under any magnetic field gradient condition.

## V. CONCLUSION

In this paper, a magnetic field gradient-based EKF is combined with a BiLSTM network in order to better estimate the velocity. This consequently improves the position reconstruction results, no matter the form of the trajectory. This method takes the BiLSTM prediction and feeds it to the EKF as a pseudo-measurement. Then a measurement covariance adapter dynamically tunes its corresponding noise parameters. The proposed approach represents a key solution to the indoor navigation problem, especially when dealing with low gradient/static trajectory cases. It also succeeds to provide accurate velocity and

position estimations when there are enough magnetic disturbances. Real experiments with an MIT module and ground truth data prove the effectiveness of the EKF-BiLSTM algorithm. The suggested technique competes with state-of-the-art solutions, especially for its generality and applicability to any use case, whether its on pedestrians, vehicles, or robots.

## CONFLICT OF INTEREST

The authors declare no conflict of interest.

## ACKNOWLEDGMENT

The authors would like to thank SYSNAV, in particular, M. Hillion, H. Meier and T. Bonis, for their assistance with the handling of the sensor module. The authors are also thankful to the team of Biomeca room at GIPSA-Lab, J. Dumon, C. Darj and S. Dycke, for their help with the experiments, from the conception of the markers frame to the data acquisition.

## REFERENCES

[1] M. Brossard. Some modern prototyping techniques for navigation based on multi-sensor fusion. PhD thesis, Mines ParisTech, France, 2020.

[2] M. Brossard, A. Barrau, and S. Bonnabel. AI-IMU dead-reckoning. IEEE Transactions on Intelligent Vehicles, 5(4):585-595, 2020.

[3] C.-I. Chesneau. Magneto-Inertial Dead Reckoning in inhomogeneous field and indoor applications. PhD thesis, Université Grenoble Alpes, 2018.

[4] T. Feigl, S. Kram, P. Woller, R. H. Siddiqui, M. Philippsen, and C. Mutschler. A Bidirectional LSTM for estimating dynamic human velocities from a single imu. In International Conference on Indoor Positioning and Indoor Navigation (IPIN), pages 1-8, Pisa, Italy, 2019.

[5] H. Fourati. Heterogeneous data fusion algorithm for pedestrian navigation via foot-mounted inertial measurement unit and complementary filter. IEEE Transactions on Instrumentation and Measurement, 64(1):221-229, 2015.

[6] F. Jamil, N. Iqbal, S. Ahmad, and D.-H. Kim. Toward accurate position estimation using learning to prediction algorithm in indoor navigation. Sensors, 20(16):4410, 2020.

[7] I. Skog, J.-O. Nilsson, and P. Händel. Evaluation of zero-velocity detectors for foot-mounted inertial navigation systems. In International Conference on Indoor Positioning and Indoor Navigation (IPIN), pages 1-6, Zurich, Switzerland, 2010.

[8] I. Skog, J.-O. Nilsson, P. Händel, and K.V.S. Hari. Foot-mounted INS for everybody - an open-source embedded implementation. In Position Location and Navigation Symposium (PLANS), pages 140-145, Myrtle Beach, SC, USA, 2012.

[9] Sysnav. Website. https://www.sysnav.fr/?lang=en.

[10] D. Vissière, A.P. Martin, and N. Petit. Using magnetic disturbances to improve IMU-based position estimation. In European Control Conference, pages 2853-2858, Kos, Greece, 2007.

[11] D. Vissière, A.P. Martin, and N. Petit. Using spatially distributed magnetometers to increase IMU-based velocity estimation in perturbed areas. In Conference on Decision and Control, pages 4924-4931, New Orleans, Louisiana, USA, 2007.

[12] M. Zmitri, H. Fourati, and C. Prieur. Improving inertial velocity estimation through magnetic field gradient-based Extended Kalman Filter. In International Conference on Indoor Positioning and Indoor Navigation (IPIN), pages 1-7, Pise, Italy, 2019.

[13] M. Zmitri, H. Fourati, and C. Prieur. Inertial velocity estimation for indoor navigation through magnetic gradient-based EKF and LSTM learning model. In International Conference on Intelligent Robots and Systems (IROS), Las Vegas, United States, 2020.

[14] M. Zmitri, H. Fourati, and C. Prieur. Magnetic field gradient-based EKF for velocity estimation in indoor navigation. Sensors, 20(20):5726, 2020.