# Generalization of Marker Augmenter Learning Models on a New Motion Database for Inverse Kinematics

Hasnaa Ouadoudi Belabzioui, Charles Pontonnier, Georges Dumont, Pierre Plantard, Franck Multon

<figure><img src="image_1.png" /></figure>

## To cite this version:

Hasnaa Ouadoudi Belabzioui, Charles Pontonnier, Georges Dumont, Pierre Plantard, Franck Multon. Generalization of Marker Augmenter Learning Models on a New Motion Database for Inverse Kinematics. Multidisciplinary Biomechanics Journal, 2025, 49th congress of the Société de Biomécanique, 10.46298/mbj.14520. [hal-04738002v2](https://hal.science/hal-04738002v2)

HAL Id: hal-04738002

https://hal.science/hal-04738002v2

Submitted on 31 Jan 2025

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>
<figure><img src="image_3.png" /></figure>

# Generalization of Marker Augmenter Learning Models on a New Motion Database for Inverse Kinematics

Hasnaa Ouadoudi Belabzioui<sup>a, b</sup>, Charles Pontonnier<sup>a</sup>, Georges Dumont<sup>a</sup>, Pierre Plantard<sup>b</sup>, Franck Multon<sup>a</sup>

<sup>a</sup> Univ Rennes, Inria, CNRS, IRISA-UMR 6074, 35000, Rennes, France

<sup>b</sup> Moovency, Inc, 35051, Cesson-Sévigné, France

* Corresponding author: hasnaa.ouadoudi-belabzioui@ens-rennes.fr

Received date: 06/04/2024

Accepted date: 28/06/2024

Publication date: 31/01/2025

**Keywords:** walking data, fine-tuning, inverse kinematics

© 2025 The Authors
Licence CC-BY 4.0
Published by Société de Biomécanique

## 1. Introduction

Computer vision techniques now offer an alternative to traditional movement analysis by detecting human joints locations in standard videos. One versatile approach involves triangulating 3D body keypoints and tracking them with musculoskeletal models. OpenCap (Uhlrich et al., 2023) is a web-based tool that utilizes this method to compute motion metrics and estimate musculoskeletal forces from smartphone videos. It operates by generating 3D keypoint trajectories using Human Pose Estimation algorithms and then employing human marker augmenter learning models to map sparse joint center positions to additional anatomical markers positions. OpenCap has been trained and tested on various movements like walking and running, but its marker augmenter learning models were trained on a specific motion database. This study aims to adapt OpenCap's models to a new motion database using fine-tuning, a process of adjusting a pre-trained neural network for a specific task (Fu et al., 2023). Given the differences in the number of markers sets between OpenCap and ours, we only considered similar markers for evaluating inference results and for fine-tuning.

## 2. Materials and methods

### 2.1 Experimental data

Our study the motion database from Schreiber et al. (2019), with 50 participants walking at five speeds (0–m/s, 0.4–0.8 m/s, and 0.8–1.2 m/s). Each participant completed at least 3 trials per speed, totalling 1145

trials. Motion was recorded using a Qualisys optoelectronic system (23 cameras, 200Hz). Three dimensional trajectories of 52 cutaneous reflective markers were recorded. To better control the experimental set-up, we simulated the input 3D joint centers, based on Schreiber et al. (2019) full-body motion capture data, as a video HPE would provide. To this end, we used the Reed et al. (1999) regression equations to emulate the 3D joints $J=\{J1, ..., J20\}$ at each time.

### 2.2 Pre-processing

The OpenCap's learning model maps the function $\psi\Theta(\mathbf{V}, \mathbf{M}, \mathbf{L}) = \hat{\mathbf{Y}}$, mapping input features $\mathbf{V}$ (human pose modelled as 3D joints), subject weight $\mathbf{M}$, and subject height $\mathbf{L}$, to output 3D marker positions $\hat{\mathbf{Y}}$. To use it on the new dataset, we processed the database and adapted the learning model accordingly. Prior to training, the 3D marker positions were normalized relative to a root marker (the midpoint of hip keypoints), scaled by the subject height, and resampled at 60 Hz. Inverse kinematics is applied to both ground truth data (originally used to estimate the joint centers, as described above), and the resulting estimated 3D marker positions $\mathbf{Y}$. The list of markers predicted by the OpenCap Arm Model and Lower Body model was larger than the ones present in the dataset. Therefore, we considered a subset of the markerset to assess the inference and the fine tuning, excluding the following markers from the outputs of OpenCap: ['r_thigh1_study', 'r_thigh2_study', 'r_thigh3_study', 'L_thigh1_study', 'L_thigh2_study', 'L_thigh3_study', 'r_sh
1_study', 'r_sh2_study', 'r_sh3_study', 'L_sh1_study', 'L_sh2_study', 'L_sh3_study']]. The OpenCap set of anatomical markers corresponds to what is commonly used for marker-based motion capture.

## 2.3 Fine-tuning marker augmenter learning models

OpenCap human marker augmenter propose two LSTM models: the Body Model for body markers and the Arm Model for arm markers. The Body Model uses lower-limb and torso keypoints, while the Arm Model uses arm and torso keypoints. In our fine-tuning strategy, named Output Layers Fine-tuning (OLF), we freezing all layers except the last one, after this level, we added an additional output layer initialized using a normal distribution with a mean of zero and a standard deviation of 0.022. During the fine-tuning stage, a weight decay of 0.01 was applied to all tunable parameters, excluding biases, in accordance with the methodology outlined by Barone et al. (2017).

Our learning algorithms were trained on a NVidia RTXA3000 GPU. The optimal models were achieved using an early stopping technique. Training concluded when the loss failed to decrease with a minimum delta of $1 \times 10^{-4}$, with a patience value of 10 epochs. Adam optimization was employed with a batch size of 64 and a learning rate ($\alpha$) of $6 \times 10^{-6}$.

## 2.4 Evaluation

The evaluation utilized Leave-One-Out procedure per subject. Anatomical marker accuracy was assessed by mean Euclidean distance $RMSE_{markers}$ per trial, averaged across trials of the same movement type in millimeters. To assess the impact of the augmenter on the inverse kinematics, a geometric calibration to scale the model and an inverse kinematics step were applied to the inferred, fine-tuned and ground truth data to obtain the

joint coordinates (Muller et al., 2019). Joint coordinate accuracy was evaluated using $RMSE_{joint coordinates}$ per trial, averaged across trials of the same motion type in degrees.

# 3. Results and discussion

<table><caption>**Table 1: (Top)** Average root mean square error $RMSE_{markers}$ [mm] for the marker's positions and corresponding standard deviation $\rho$ are presented. **(Bottom)** Average root mean square error. $RMSE_{joint coordinates}$ [Degree] for joint coordinates and corresponding standard $\rho$.</caption><thead><tr><th></th><th>Inference [mm]</th><th>OLF [mm]</th></tr></thead><tbody><tr><td>Body model</td><td>57.86 ± 5.64</td><td>24.92 ± 6.98</td></tr><tr><td>Arm model</td><td>24.09 ± 3.31</td><td>18.54 ± 3.49</td></tr><tr><td></td><td>Inference [°]</td><td>OLF [°]</td></tr><tr><td>Left knee flexion angle</td><td>8,63 ±3,37</td><td>9,23 ± 2,08</td></tr><tr><td>Right knee flexion angle</td><td>16,26 ± 4,55</td><td>9,94 ± 1,52</td></tr><tr><td>Left ankle flexion angle</td><td>10,15 ± 2,58</td><td>10,48±1,97</td></tr><tr><td>Right ankle flexion angle</td><td>10,40 ± 2,94</td><td>10,58±2,38</td></tr><tr><td>Left foot inversion angle</td><td>25,68 ± 4,35</td><td>9,27±2,84</td></tr><tr><td>Right foot inversion angle</td><td>33,29 ± 2,94</td><td>8,03±2,01</td></tr></tbody></table>

The inference was conducted overall. Then, we fine-tuned both the OpenCap human marker augmenter, and it was tested on an unseen set of 25 subjects. As

<figure><img src="image_4.png" /><figcaption>**Figure 1.** Comparison of RIAS (Right anterior-superior iliac spine coordinates) marker results.</figcaption></figure>

y-axis = Cartesian coordinates (mm), x-axis = frames. Colors: red = ground truth, blue = fine-tuning, black = inference.
illustrated in Table 1 (Top) and Figure 1, both fine-tuned
models consistently outperformed the inference models,
underscoring their efficacy in reducing marker displace-
ment errors across various body parts. Additionally, the
standard deviations also decreased after fine-tuning,
suggesting that the models' predictions became more
consistent. The results presented in Table 1 (Bottom)
illustrate that the fine-tuning generally led to a reduc-
tion in the mean values and, in some cases, a decrease
in the standard deviation. The changes in angles after
fine-tuning may suggest better accuracy in the models'
predictions.

4. Conclusions and perspectives

This study evaluated fine-tuning impact for marker
position prediction and angle estimation, which con-
sistently yielding the lowest average RMSE values for
marker positions compared to inference results. As the
estimation of joint coordinates is conditioned by ana-
tomical markers, as well as geometric calibration and
inverse kinematics, exploring the impact of geometric
calibration would be an interesting perspective. However,
it was noted that part of the residual error may stem
from data emulation, highlighting the significant impact
of input skeleton topology and marker set on the learn-
ing models performance. In a future study, we aim at
applying similar strategies to experimental HPE data
to enhance the accuracy and generalization of such
marker augmenter to be used on field.

Funding

This work was partially funded by the Cifre convention
N°0936/2021, Moovency company, and the National
Agency under the Inverstments for the Future pro-
gram (PIA) with the grand ANR-21-ESRE-0030
(CONTINUUM project).

References

Barone, A. V. M., Haddow, B., Germann, U., & Sennrich, R. (2017). Regularization techniques for fine-tuning in neural machine translation. *arXiv preprint* arXiv:1707.09920.

Fu, Z., Yang, H., So, A. M. C., Lam, W., Bing, L., & Collier, N. (2023, June). On the effectiveness of parameter-efficient fine-tuning. In *Proceedings of the AAAI Conference on Artificial Intelligence* (Vol. 37, No. 11, pp. 12799-12807).

Muller, A., Pontonnier, C., Puchaud, P., & Dumont,
G. (2019). CusToM: a Matlab toolbox for muscu-
loskeletal simulation. *Journal of Open Source Software,
*4*(33), 1-3.

Uhlrich, S. D., Falisse, A., Kidziński, Ł., Muccini, J., Ko, M., Chaudhari, A. S., ... & Delp, S. L. (2023). OpenCap: Human movement dynamics from smartphone videos. *PLoS computational biology*, 19(10), e1011462.

Reed, M. P., Manary, M. A., & Schneider, L. W. (1999). Methods for measuring and representing automobile occupant posture (No. 1999-01-0959). *SAE Technical Paper*.

Schreiber, C., & Moissenet, F. (2019). A multimodal dataset of human gait at different walking speeds established on injury-free adult participants. *Scientific data*, 6(1), 111.