# Breathing Motion Compensation Based on Long Short-Term Memory Network for Minimally Invasive Pedicle Screw Placement

Ayoob Davoodi, Ruixuan Li, Kaat Van Assche, Matthias Tummers, Gianni

Borghesan, Emmanuel Vander Poorten

► To cite this version:

Ayoob Davoodi, Ruixuan Li, Kaat Van Assche, Matthias Tummers, Gianni Borghesan, et al. Breathing Motion Compensation Based on Long Short-Term Memory Network for Minimally Invasive Pedicle Screw Placement. CRAS 2024 - Conference on New Technologies for Computer and Robot Assisted Surgery, Sep 2024, Odense, Denmark. hal-05482513

HAL Id: hal-05482513

https://hal.science/hal-05482513v1

Submitted on 28 Jan 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# Breathing Motion Compensation Based on Long Short-Term Memory Network for Minimally Invasive Pedicle Screw Placement

Ayoob Davoodi, Ruixuan Li, Kaat Van Assche, Matthias Tummers, Gianni Borghesan, and Emmanuel Vander Poorten

KU Leuven, Department of Mechanical Engineering, Robot-Assisted Surgery Group, Belgium

## INTRODUCTION

Spinal diseases such as spinal degeneration and scoliosis might require pedicle screw placement (PSP) as a crucial step during surgical interventions, depending on the severity. This procedure requires the drilling of a hole for placing the screw. Thanks to imaging modalities such as computed tomography and intraoperative fluoroscopy, moving from an open approach to minimally invasive surgery (MIS) has been possible and has reduced patient complications after surgery. Still, it suffers from a lack of visual feedback for the surgeon, whereas robotic-assisted spine surgery combined with such imaging modalities can improve surgical outcomes for PSP. Yet, in such MIS procedures, physical motion, such as breathing motion, can induce shifts and deformations in the spine, leading to operation errors of approximately 2-3 mm [1]. In order to correctly identify the entry point, breathing motion needs to be compensated for. External sensors, such as an optical tracking system or range imaging, can be used to measure the motion of the skin or neighboring vertebrae, close to the entry point [2], [3]. However, Saghbiny et al. showed that the amplitude of breathing motion changes over vertebrae and has a variation of 68% from the lumbar to thoracic vertebrae [4]. Therefore, this work develops an approach for estimating the motion of each entry point, enhancing the accuracy. The proposed method uses a long short-term memory network (LSTM) on top of the inner control loop to estimate breathing motion parameters for each pedicle drilling individually, and its output is utilized to update the motion model for motion compensation during robot-assisted drilling for MIS-PSP.

## MATERIALS AND METHODS

The experimental setup consists of a robotic drill system with a robot arm (KUKA Robot Med7, Augsburg, Germany) and a custom-designed drilling system with a force/torque sensor (Nano25, ATI). An optical tracking system (FusionTrack 500, Atracsys, Switzerland) is used to track poses. An ex vivo pig spine is placed on a breathing platform, shown in Fig. 1 (a), which generates a one-degree-of-freedom breathing motion. During MIS-PSP, an optical marker is pinned to the spinous process of a vertebra, which is close to the interested drilling

This project has received funding from the European Union's Horizon 2020 research and innovation program under grant agreement No 101016985 (FAROS).

<figure><img src="image_2.png" /><figcaption>Fig. 1: (a) Illustration of the robotic drilling system for MIS-PSP. (b) Control block diagram of the proposed LSTM-based breathing motion compensation.</figcaption></figure>

vertebrae. Two challenges are identified. First, in an in vivo procedure, moving from vertebrae L5 to L1, the breathing amplitude changes from 2.2 mm to 3.7 mm, corresponding to a rise of 68% in amplitude [4]. Thus, the optical marker can not measure the local information for each vertebra correctly. Second, since the vertebrae levels change during drilling and the soft/hard tissues have an unknown stiffness, the robotic response time delay also changes for each pedicle drilling. These two challenges are addressed in this study: identifying the variable breathing amplitude through a scaling variable $\alpha$ and the variable time delay $\tau_0$ for each pedicle drilling. Because of the complex environment, a data-driven approach, such as LSTM, helps tackle this complicated problem. The LSTM ability to learn historical information and use prior knowledge to predict system behavior at future times has formed a key motivation for using this network for this application. The control block diagram of the proposed system is illustrated in Fig. 1 (b). An inner control loop is used to maintain the constant desired force $f_d$ during the pedicle drilling; on top of it, an LSTM-based motion compensation generates the corresponding motion profile for the controller. Each vertebrae motion $\hat{B}(t)$ can be modeled as $\hat{B}(t) = \alpha B(t - \tau_0)$. $B(t)$ is the periodic
<figure><img src="image_3.png" /><figcaption>Fig. 2: The LSTM-based deep learning network and the simulation environment.</figcaption></figure>

breathing motion that is estimated with

$$B(t) = a_0 + \sum_{n=1}^{3} a_n \cos(n\omega_0 t) + b_n \sin(n\omega_0 t), \quad (1)$$

where $a_0$, $a_n$, and $b_n$ are the estimated coefficients, $\omega_0$ is estimated breathing frequency, and $t$ is time. To estimate parameters from the measured breathing motion $\tilde{B}(t)$, the following cost function is optimized:

$$J = \int_{t-T_0}^{t} (\tilde{B}(t) - B(t))^2 dt, \quad (2)$$

where $T_0$ is the optimization horizon. Within this work, a 2-layer stacked LSTM with 50 neurons was used Fig. 2. Two fully connected layers, input and output of dimension 50, were added before and after the LSTM. The LSTM is trained with the following input and output vectors:

$$X(t) = [B(t), \frac{dB(t)}{dt}, f_e(t), \int f_e(t) dt]^T, Y(t) = [\alpha, \tau_0]^T, \quad (3)$$

where $\frac{dB(t)}{dt}$ and $f_e(t)$ are the derivative of modeled breathing motion and drill tip force error. To train the deep learning network, a simulation-based dataset is generated, the robotic drilling system is modeled within Matlab Simscape Multibody, where the environment is modeled with constant stiffness $K_e \in [2, 3.5]$ N/mm for one simulation, and a pig breathing profile is utilized to simulate breathing motion [4]. 600 combinations of $\alpha \in [0.3, 1.7]$, $\tau_0 \in [-1.5, 1.5]$ s and $K_e$ are generated with uniform distribution. For each combination, the corresponding breathing motion $\tilde{B}(t) = \alpha B(t - \tau_0)$ is simulated to generate the input $X(t)$ and output $Y(t)$ vectors of the LSTM. 70% of this matched info is used to train the network, and the remaining 30% are considered ground truth test data to infer the original $\alpha$ and $\tau_0$ that were used to simulate the system.

## RESULTS

The performance of the trained network is evaluated on 180 configurations of ground truth test data; an example

<figure><img src="image_4.png" /><figcaption>Fig. 3: An example of the time delay $\tau_0$ and breathing amplitude scale $\alpha$ estimation overtime.</figcaption></figure>

<table><caption>TABLE I: Deep learning-based estimation error</caption><thead><tr><th></th><th>Mean</th><th>Std</th><th>Max-abs</th><th>Mean-abs</th></tr></thead><tbody><tr><td>Amplitude scale</td><td>-0.0140</td><td>0.0382</td><td>0.149</td><td>0.0732</td></tr><tr><td>α - α<sub>LSTM</sub></td><td>(99.0%)</td><td>(2.72%)</td><td>(89.3%)</td><td>(94.8%)</td></tr><tr><td>Time delay [ms]</td><td>6.20</td><td>25.6</td><td>75.8</td><td>35.7</td></tr><tr><td>τ<sub>0</sub> - τ<sub>LSTM</sub></td><td>(99.6%)</td><td>(0.850%)</td><td>(97.5%)</td><td>(98.8%)</td></tr></tbody></table>

is depicted in Fig. 3. The error between ground truth and estimated values from LSTM $\alpha_{LSTM}$, $\tau_{LSTM}$ is used for evaluation; the mean error, standard deviation, maximum absolute error, and mean absolute error, are summarized in table I. The value below for each criterion shows the improvement percentage. The proposed system demonstrates a 35.7 ms time delay error and reduces the amplitude scale error to 0.0732.

## CONCLUSIONS AND DISCUSSION

This study presents a deep learning-based method for mitigating environmental and system uncertainty during breathing motion compensation for MIS-PSP. The proposed framework uses an LSTM-based method to individually identify the amplitude scale and time delay error for each pedicle drilling. Validation of the proposed system in simulation shows that it can reduce the error in time shift to 35.7 ms, and the amplitude scale error can be compensated 94.8%. This approach embeds intelligence in the robotic PSP system, making it robust to environmental changes. However, further validation through ex vivo or in vivo trials is essential to assess the performance of the method.

## REFERENCES

[1] Y. Liu et al., "Assessment of respiration-induced vertebral motion in prone-positioned patients during general anaesthesia," *The International Journal of Medical Robotics and Computer Assisted Surgery*, vol. 12, no. 2, pp. 214-218, 2016.

[2] B. Li, et al., "Respiratory motion estimation of tumor using point clouds of skin surface," *IEEE Transactions on Instrumentation and Measurement*, 2023.

[3] R. Dürichen et al., "Evaluation of the potential of multi-modal sensors for respiratory motion prediction and correlation," in *2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)*. IEEE, 2013, pp. 5678-5681.

[4] E. Saghbiny et al., "Design of an ex-vivo experimental setup for spine surgery based on in-vivo identification of respiration-induced spine movement," in *HSMR2023: The 15th Hamlyn Symposium on Medical Robotics*, 2023.