# Triaxial knee joint kinematics estimation from inertial measurement units using LSTM neural networks

Daphnée Lalonde-Larocque, Neila Mezghani, Nathalie J Bureau, Rachid

Aissaoui, Nicola Hagemeister

► To cite this version:

Daphnée Lalonde-Larocque, Neila Mezghani, Nathalie J Bureau, Rachid Aissaoui, Nicola Hagemeister. Triaxial knee joint kinematics estimation from inertial measurement units using LSTM neural networks. Multidisciplinary Biomechanics Journal, 2025, 49th congress of the Société de Biomécanique, 10.46298/mbj.14518 . hal-04738000v2

HAL Id: hal-04738000

https://hal.science/hal-04738000v2

Submitted on 31 Jan 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Triaxial knee joint kinematics estimation from inertial measurement units using LSTM neural networks

Daphnée Lalonde-Larocque<sup>a*</sup>, Neila Mezghani<sup>b</sup>, Nathalie J Bureau<sup>c</sup>, Rachid Aissaoui<sup>a</sup>, Nicola Hagemeister<sup>a</sup>

<sup>a</sup> École de technologie supérieure (ÉTS), Montréal, Canada

<sup>b</sup> Université TÉLUQ, Montréal, Canada

<sup>c</sup> Université de Montréal, Montréal, Canada

* Corresponding author: daphnee.lalonde-larocque.1@ens.etsmtl.ca

Received date: 05/04/2024

Accepted date: 28/06/2024

Publication date: 31/01/2025

**Keywords:** gait analysis, LSTM neural networks, knee joint kinematics, IMU

© 2025 The Authors
Licence CC-BY 4.0
Published by Société de Biomécanique

## 1. Introduction

Gait analysis yields useful data to healthcare providers, whether it be in diagnosis, treatment, or rehabilitation of their patients. Optical motion capture systems are the current standard used in the industry since they generate the best accuracy. However, the complexity of this type of system restricts their usage to a lab environment. To overcome this limitation, the use of inertial measurement units (IMU) is explored as it offers an unconstrained condition alternative. Nonetheless, noise and drift induce inaccuracies, even more so for lower range of motion (ROM) movements such as the knee internal/external rotations (Ir/Er) and abduction/adduction (Ab/Ad) (Poitras et al. 2019).

Combined with proper optimization algorithms, integration of machine learning approaches such as neural networks (NN) offer an improvement in the accuracy of the predicted joint kinematics (Sharifi-Renani et al. 2021). Long short-term memory (LSTM) networks are a class of recurrent NN very well suited for time series data. By making predictions one frame at a time, LSTM uses feedback to remember long-term time dependencies (Mundt et al. 2021) and upgrade its internal state, thus propagating previous information while addressing the vanishing gradient problem (Rapp et al. 2021).

Multiple previous LSTM networks have been trained on databases usually containing not more than 75 different patients to estimate knee joint kinematics from IMUs data. While they usually perform great on angle estimations for higher ROM movements such as flexion/

extension (Fl/Ex), normalized-RMSEs (nRMSE) for lower ROM movements such as Ab/Ad and Ir/Er are usually very high, respectively representing around 12% and 14% of the actual range of the expected values (Rapp et al. 2021; Sharifi-Renani et al. 2021).

The goal of the present study is to train an LSTM network on a larger database and to separate the estimations of each movement to possibly prevent the larger domain of Fl/Ex data to affect the estimations on the lower ROM movements and obtain lower nRMSEs.

## 2. Methods

### 2.1 Database and pre-processing

To train the model, we merged marker-based motion capture data from two databases previously collected by CIUSSS de l'Est-de-l'Île-de-Montréal and Maisonneuve-Rosemont hospital, in collaboration with CHUM Research Center, TÉLUQ and ÉTS. The first database consists of 115 healthy subjects (52 men and 63 women, age: 38±17). The second one consists of 560 patients suffering of knee osteoarthritis (OA) to various degrees (216 men and 344 women, age: 62±9, OA grades 2-4).

Participants were asked to perform multiple 45 seconds walking trials (speed of 0.8 m/s) on a treadmill wearing the KneeKG system (Emovi inc, Canada) on one knee. The trajectories of the shank and thigh clusters of markers were captured at a speed of 60Hz using the Polaris SpectraTM camera system. Inertial data was simulated by deriving the clusters' markers positions,
hence providing 3D angular velocity data from the shank and thigh segments which are used as inputs for the model. Output data was the 3D rotations directly obtained from gait analysis using the KneeKG system.

## 2.2 LSTM Network

The model used for this study is an LSTM network which combines six (6) normalized angle vectors into a matrix to estimate three (3) knee joint kinematics vectors for each trial. The network contains an LSTM of 200 units, followed by a dropout layer of 0.3, then two hidden layers of sizes 150 and 100, respectively, both using a tanh activation function. Output layers all used a linear activation function.

A window size of 17 frames (W17) was tested using a window function to walk through each trial, providing 722 545 samples. Thus, the model's input is a [17 × 6] matrix and the outputs are three [17 × 1] matrices. Model training was performed on a total of 50 epochs with a batch size of 100. The learning rate was set to 0.0001 using an Adam optimizer.

The LSTM network was trained using a 10-folds cross-validation method. Prior to partitioning, 50 subjects were removed from the database to provide a final testing database for further analysis. Each partition contained 62±3 subjects. The models' estimation accuracy was quantified using nRMSE.

## 3. Results and discussion

Mean nRMSEs over the 10-folds was evaluated for each movement and is presented in Table 1. The model had an overall mean nRMSE of 5.15±0.45 %. Ab/Ad nRMSE was 5.67±0.52 %, Fl/Ex nRMSE was 3.00±0.60 % and Ir/Er nRMSE was 6.57±0.30 %. The model's estimation over each fold for the Fl/Ex axis had consistently the highest accuracy, as opposed to the estimations of the Ir/Er and Ab/Ad axes.

<table><caption>**Table 1.** 10-folds cross-validation's mean nRMSEs of each output layers</caption><thead><tr><td>Win. size</td><td>Ab/Ad (%)</td><td>Fl/Ex (%)</td><td>Ir/Er (%)</td><td>Mean (%)</td></tr></thead><tbody><tr><td>W17</td><td>5.94 ±0.53</td><td>3.00 ±0.60</td><td>6.57 ±0.30</td><td>5.17 ±0.48</td></tr></tbody></table>

Credit: Daphnée Lalonde-Larocque (2024)

Multiple studies have trained LSTM networks to predict joint angles of the knee in a similar manner, but

only Sharifi-Renani et al. (2021) provides nRMSE values for all three axes. Their study obtained an overall mean nRMSE of 7.5±3.1 % (6.6±2.3 % for Ab/Ad, 1.9±0.7 % for Fl/Ex and 14.1±6.4 % for Ir/Er). The model we trained yielded better accuracies for the lower ROM movements, as well as a better overall accuracy. On the other hand, nRMSE for the Fl/Ex movement of our model was 1% more than the one of Sharifi-Renani et al. (2021).

This study has a few limitations, such as the number of metrics used to quantify the model's estimations accuracy. To offer better grounds of comparison with the literature, more metrics will be added in the future. Also, it should be noted that gait measurements were taken in a laboratory environment, which could influence the subjects' normal gait patterns.

## 4. Conclusions

The proposed LSTM network yielded nRMSEs lower than what can currently be seen in the literature for Ab/Ad and Ir/Er movements, which leads us to continue exploring the hypothesis that outputs need to be separated from one another to prevent Fl/Ex data to affect estimations on the other axes.

## Acknowledgements

We would like to acknowledge the researchers who contributed to the databases: PA Vendittoli, M Choinière, M Durand, N Gaudreault.

## Funding

Funded by Marcelle Gauvreau's research chair of ÉTS.

## Conflict of Interest Statement

All authors have no conflicts of interest.

## References

Mundt, M., Johnson, W. R., Potthast, W., Markert, B., Mian, A., & Alderson, J. (2021). A comparison of three neural network approaches for estimating joint angles and moments from inertial measurement units. *Sensors*, 21(13), 4535. doi: 10.3390/s21134535

Poitras, I., Dupuis, F., Bielmann, M., Campeau-Lecours, A., Mercier, C., Bouyer, L. J., & Roy, J.-S. (2019). Validity and Reliability of Wearable Sensors for Joint Angle Estimation: A Systematic Review. *Sensors*, 19(7), 1555. doi: 10.3390/s19071555
Rapp, E., Shin, S., Thomsen, W., Ferber, R., & Halilaj, E. (2021). Estimation of kinematics from inertial measurement units using a combined deep learning and optimization framework. *Journal of Biomechanics*, 116, 110229. doi: 10.1016/j.jbiomech.2021.110229

Sharifi-Renani, M., Eustace, A. M., Myers, C. A., & Clary, C. W. (2021). The Use of Synthetic IMU Signals in the Training of Deep Learning Models Significantly Improves the Accuracy of Joint Kinematic Predictions. *Sensors*, 21(17), 5876. doi: 10.3390/s21175876