# Comparative evaluation of inpainted and marker-visible images using a deep neural network

Maxime Perek, Yoann Blache, Louane Boullier, Felix Lefebvre, Isabelle

Rogowski, Brice Guignard

<figure><img src="image_1.png" /></figure>

## To cite this version:

Maxime Perek, Yoann Blache, Louane Boullier, Felix Lefebvre, Isabelle Rogowski, et al. Comparative evaluation of inpainted and marker-visible images using a deep neural network. Multidisciplinary Biomechanics Journal, 2025, 50th congress of the.., 10.46298/mbj.16158. hal-05191710v2

HAL Id: hal-05191710

https://hal.science/hal-05191710v2

Submitted on 23 Oct 2025

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>
# Comparative evaluation of inpainted and marker-visible images using a deep neural network

Maxime Perek<sup>a*</sup>, Yoann Blache<sup>a</sup>, Louane Boullier<sup>a</sup>, Felix Lefebvre<sup>a, b</sup>, Isabelle Rogowski<sup>a</sup>, Brice Guignard<sup>a</sup>

Received date: 04/04/2025

Accepted date: 27/06/2025

Publication date: 27/10/2025

**Keywords:** markerless, inpainting, deep neural network

© 2025 The Authors
Licence CC-BY 4.0
Published by Société de Biomécanique

## 1. Introduction

Recent advances in markerless pose estimation tools, such as DeepLabCut (DLC; Mathis et al., 2018), have transformed human motion analysis by enabling accurate landmark identification without physical markers. These methods are designed to overcome the need for traditional skin markers and thereby simplifying protocols and improving participant comfort.

In practice, ground-truth measurements for accuracy assessment of markerless approaches can rely on a marker-based or any method that allows the reference to be labeled during the validation phase. When such markers remain visible in the validation images, they provide artificial cues on which the neural network (NN) can overfit, ultimately degrading real-world performance (Hein et al., 2021). Indeed, Rosskamp et al. (2024) showed that visible markers significantly reduce markerless landmark detection accuracy. Highlighting the need for artifact removal, conventional procedures typically require two separate image captures—one with markers and one without—to avoid visible markers in the final images. However, this approach is both time-consuming and prone to errors due to manual handling and repeated positioning of markers. Another approach would be to digitally remove markers using image inpainting techniques, which offer a promising alternative by automatically reconstructing masked image regions (Rosskamp et al., 2024). However, to our knowledge,

the effects of inpainting on NN performance remain unclear and warrant further investigation.

This study aimed to evaluate whether visible markers decrease the accuracy of anatomical landmark identification by a NN, and to investigate whether inpainting effectively mitigates this source of imprecision.

## 2. Methods

### 2.1. Participants

Ninety healthy participants (35 women; age: 23.5 ± 3.7 yrs; height: 173.3 ± 7.2 cm; body mass: 68.0 ± 8.5 kg; body fat percentage determined by impedance meter: 19.1 ± 5.9%) signed an informed consent, and were included in this study, which was approved by the local ethical committee (CER-UdL n°2022-10-13-002).

### 2.2. Procedure

Participants were asked to hold 12 static postures, mimicking arm elevations and the lateral scapular slide test. For each posture, two images were captured (iPad, Apple, Cupertino USA, 4032×3024px), either with a naked back, or with markers placed after palpation for 10 anatomical landmarks: 7th cervical vertebra (C7), 8th thoracic vertebra (T8), and bilaterally, posterior acromion (AA), trigonum spinae (TS), inferior scapular angle (IA), and lateral epicondyle (EPL). A guide was used to minimize movements between the two images. For pixel to millimeter conversion a 300 mm ruler was placed as close as possible to the participant in the thoracic plane.

<footnote><sup>a</sup> Universite Claude Bernard Lyon 1, LIBM, Laboratoire Interuniversitaire de Biologie de la Motricité, UR 7424, F-69622 Villeurbanne, France</footnote>

<footnote><sup>b</sup> TRINOMA, Villefort, France</footnote>

<footnote>* Corresponding author: maxime.perek@univ-lyon1.fr</footnote>
<figure><img src="image_3.png" /><figcaption>**Figure 1.** Marker-removal workflow; A: original image with visible markers (cropped at 1008x756 px from initial iPad image); B: mask used to identify the marker regions; C: Inpainted image (once again cropped at 1008x756 px).</figcaption></figure>

Credit: Perek.

## 2.3. Deep neural network training and evaluation

For each image, coordinates of the 10 anatomical landmarks were digitized using DeepLabCut (Mathis et al., 2018; Nath et al., 2019) and transferred to its counterpart image without markers. The DLC ResNet-50 NN was trained over 150,000 iterations using images without markers from 60 participants (701 images) and evaluated on the remaining 30 participants (351 images), selected randomly. For the evaluation phase, three sets of test images were compared: the original images without markers (the reference), original images with markers and the corresponding images in which markers were digitally removed using inpainting. The latter was performed using LaMa (Suvorov et al., 2021) (Figure 1). Briefly, binary masks (15 px diameter circles centered on each marker based on their labeled 2-D coordinates) were automatically generated and processed by LaMa's Fast Fourier Convolutional network to reconstruct the missing pixels using both global and local image context. A 0.6 confidence threshold was applied for prediction. The Euclidean distance between the manually labeled coordinates and those predicted by the NN were computed for each anatomical landmark and for the three sets of test images.

## 3. Results and discussion

<table><caption>**Table 1.** Euclidean distances (in mm) between landmark coordinates obtained via labelling and by the neural network for inpainted images, images with and without markers and their respective prediction point losses (in %)</caption><thead><tr><th>Input</th><th>Images without markers</th><th>Inpainted images</th><th>Images with markers</th></tr></thead><tbody><tr><td>Median</td><td>9</td><td>9</td><td>9</td></tr><tr><td>90<sup>th</sup> percentile</td><td>29</td><td>30</td><td>32</td></tr><tr><td>95<sup>th</sup> percentile</td><td>36</td><td>38</td><td>49</td></tr><tr><td>99<sup>th</sup> percentile</td><td>60</td><td>66</td><td>169</td></tr><tr><td>Max<br>Predicted points lost</td><td>86<br>10%</td><td>102<br>10%</td><td>934<br>22%</td></tr></tbody></table>

Credit: Perek.

Although the median errors were identical for the three sets of test images (Table 1), the NN exhibited better performance for inpainted images in comparison to marker-visible images. When a 0.6 confidence threshold was applied, fewer points were predicted for
marker-visible images. These outcomes highlighted that markers interfere with the NN's predictions (Rosskamp et al., 2024). Notably for each anatomical landmark, at least one prediction error exceeded 400 mm, except for EPL whose maximum error was 150 mm. In contrast, for inpainted images, the NN provided twice fewer losses and achieved the same performance as the reference. This is likely due to the NN being trained on markerless images, causing extreme errors when markers were visible. The overall error distribution indicated also that inpainting effectively mitigated large outliers. Moreover, the artifacts potentially introduced by LaMa appear to have a negligible impact on the NN's predictions. These findings support the potential of inpainting to reduce marker-induced artifacts, although further research is required to confirm improvements in anatomical landmark detection reliability.

## 4. Conclusions

This study is a first step toward validating an inpainting technique for digitally removing markers from images used in markerless analyses. These findings could serve as a benchmark for future developments, such as extending to 3D applications or removing unwanted objects in motion tracking. While this technology could reduce acquisition and processing times, workflow improvements are still needed to reach precision close to images without markers.

## Conflict of Interest Statement

None.

## Contributor Roles

MP: Conceptualization, Methodology, writing original draft; YB: Conceptualization, Methodology, Validation, Supervision, Writing-review & editing draft; LB: Data curation, Formal analysis, review & editing draft; FL: software, resources, review & editing draft; IR: Methodology, Validation, Supervision, review & editing draft; BG: Conceptualization, Methodology, Validation, Supervision, Writing-review & editing draft.

## References

Hein, J., Seibold, M., Bogo, F., Farshad, M., Pollefeys, M., Fürnstahl, P., & Navab, N. (2021). Towards markerless surgical tool and hand pose estimation. *International journal of computer assisted radiology and

surgery, 16, 799-808. https://doi.org/10.1007/s11548-021-02369-2

Mathis, A., Mamidanna, P., Cury, K. M., Abe, T., Murthy, V. N., Mathis, M. W., & Bethge, M. (2018). DeepLabCut: markerless pose estimation of user-defined body parts with deep learning. *Nature neurosci-ence, 21*(9), 1281-1289. https://doi.org/10.1038/s41593-018-0209-y

Nath, T., Mathis, A., Chen, A. C., Patel, A., Bethge, M., & Mathis, M. W. (2019). Using DeepLabCut for 3D markerless pose estimation across species and behaviors. *Nature protocols, 14*(7), 2152-2176. https://doi.org/10.1038/s41596-019-0176-0

Rosskamp, J., Weller, R., & Zachmann, G. (2024). Effects of markers in training datasets on the accuracy of 6D pose estimation. In *Proceedings of the IEEE/ CVF Winter Conference on Applications of Computer Vision* (pp. 4457-4466). https://doi.org/10.1109/WACV57701.2024.00440

Suvorov, R., Logacheva, E., Mashikhin, A., Remizova, A., Ashukha, A., Silvestrov, A., ... & Lempitsky, V. (2022). Resolution-robust large mask inpainting with fourier convolutions. In *Proceedings of the IEEE/ CVF Winter Conference on Applications of Computer Vision* (pp. 2149-2159). https://doi.org/10.1109/WACV51458.2022.00323