# All-Ultrasound-Guided Path Planning for Robotic Pedicle Screw Placement

Ayoob Davoodi, Ruixuan Li, Kaat Van Assche, Maikel Timmermans, Matthias Tummers, Gianni Borghesan, Kathleen Denis, Emmanuel Vander Poorten

► To cite this version:

Ayoob Davoodi, Ruixuan Li, Kaat Van Assche, Maikel Timmermans, Matthias Tummers, et al. All-Ultrasound-Guided Path Planning for Robotic Pedicle Screw Placement. 2024 10th IEEE RAS/EMBS International Conference for Biomedical Robotics and Biomechatronics (BioRob), Sep 2024, Heidelberg, Germany. pp.641-647, 10.1109/BIOROB60516.2024.10719904. hal-04961379

HAL Id: hal-04961379

https://hal.science/hal-04961379v1

Submitted on 21 Feb 2025

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# All-Ultrasound-Guided Path Planning for Robotic Pedicle Screw Placement

Ayoob Davoodi*, Ruixuan Li*, Kaat Van Assche, Maikel Timmermans, Matthias Tummers, Gianni Borghesan, Kathleen Denis, and Emmanuel Vander Poorten.

**Abstract**—Pedicle screw placement (PSP) is a challenging procedure in spine surgery, due to poor visualization of the vertebrae and blood vessels. This has led to the development of navigation systems that improve safety and clinical outcomes. Navigation further enabled minimal invasive approaches for PSP. Current navigation systems require a preoperative computed tomography (CT) scan, followed by manual planning by the surgeon to decide screw trajectories. Intraoperative fluoroscopy imaging may be employed to register the preoperative plan with the patient's anatomy frame or even design screw trajectory planning in real-time for each pedicle individually. This is a rather lengthy procedure that involves harmful radiation to which the surgeon is exposed repeatedly. This article explores a novel approach for PSP guidance that replaces manual planning while solely relying on 3D ultrasound (US) reconstruction, hence reducing the need for radiation. Based on an intraoperative US reconstruction, a set of anatomical landmarks of the spine are identified. Subsequently, a coordinate frame is computed per vertebra. The screw path is then generated automatically using the learned relations between the landmarks and optimal screw paths. A five-fold cross-validation is conducted on the posterior surface of CT spine data involving 90 PSP paths. The algorithm is then tested on another 50 PSP paths of the 3D-printed spine phantoms corresponding to human spine CT data. Results show that for 4 mm and 6 mm diameter screws, 98% and 84% of the computed trajectories are within the required surgical precision, respectively. Using the US-based path planning led to $-0.34 \pm 3.66^\circ$ and $-0.45 \pm 4.32^\circ$ orientation errors in the sagittal and axial planes, respectively.

**Keywords:** robotic spine surgery, ultrasound reconstruction, onsite path-planning, pedicle screw placement.

## I. INTRODUCTION

The incidence of spine diseases has gradually increased in recent years, posing a high burden on people's health [1]. Pedicle screw placement (PSP) is a procedure to stabilize the spine and provide supplemental posterior fixation [2], [3]. However, PSP remains a challenging task because of limited visibility and potential screw slippage. As such, freehand PSP could still result in up to 20.8% of misplaced screws that are not fully contained within the pedicle [4], [5]. Due to the proximity of the pedicles to sensitive structures such

<figure><img src="image_2.png" /><figcaption>Fig. 1. System workflow of the proposed US-based screw path planning. Processing steps belonging to the blue area form the basic components that produce drilling paths from US.</figcaption></figure>

as the spinal cord, incorrectly placed screws can perforate the pedicle wall and cause serious clinical complications, including neurological, vascular, or dural injury [6].

In the past decades, surgical navigation and robotic systems have been designed to replace or assist manual screw placement [7], [8]. These techniques reduce the operational burden on surgeons and improve the surgical success rate of lumbar fixation. The current workflow for navigated PSP is based on a preoperative computed tomography (CT) scan of the spine [9], [8]. The position and orientation of the screws is determined manually by the surgeon before the surgery. Then, during the surgery, fluoroscopy is used to confirm the preoperative planning. The whole procedure requires multiple preoperative consultations and exposes both the patient and the clinical staff to a considerable amount of ionizing radiation.

To reduce the radiation, recent work proposed novel methods for registering the preoperative planning to the patient frame [10]. The intraoperative 3D ultrasound (US) reconstruction of the spine proved to be an alternative [11], [12]. Other groups have looked into automating the planning procedure [13], saving time for the surgeon. However, these methods still rely on ionizing preoperative CT imaging [14], [15]. The state-of-the-art in this field is therefore divided into two parts: US reconstruction of the spine (SubSec.I-A) and automated path-planning for PSP (SubSec.I-B).

### A. US Spine Reconstruction

US imaging enabled the reconstruction of superior surfaces of vertebrae and was shown to be effective for the placement of pedicle screws and registering of patient anatomy intraoperatively. For example, Chan *et al.* developed a custom 3D US system to register a preoperative CT model into intraoperative space for screw placement [16], achieving an

<footnote>* These authors contributed equally to this work.</footnote>

<footnote>A. Davoodi, R. Li, K. Van Assche, M. Timmermans, M. Tummers, G. Borghesan, K. Denis, and E. Vander Poorten are with the Department of Mechanical Engineering, KU Leuven, Belgium. G. Borghesan is with Robot-Assisted Surgery (RAS) group, Department of Mechanical Engineering, KU Leuven, Belgium, and also with Core Lab ROB, Flanders Make, Belgium. (email: ayoob.davoodi@kuleuven.be)</footnote>

<footnote>This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 101016985 (FAROS) and Flemish Research Foundation (FWO) under grant agreement NO.G0A1420N (Radar-spine) and NO.1S36324N (Harmony) and NO.1S35422N (SpineVib).</footnote>
<figure><img src="image_3.png" /><figcaption>Fig. 2. The proposed robot-assisted US system consists of a robotic arm, a US machine, the US probe, and a 3D-printed spine phantom in an agar volume. The four displayed frames are: the world frame {$w$}(i.e., robot base frame), the robot end effector frame {$EE$}, the US probe frame {$US$}, and the vertebra frame {$v$}. The close-up shows a vertebra with its associated vertebra frame {$v$} and the PSP path going through the pedicles.</figcaption></figure>

average $0.4 \pm 0.4$ mm and $2.1 \pm 0.9$ entry point and trajectory error, respectively. Gonzalez *et al.* developed a radiation-free system, combining ultrasound and photoacoustic images to avoid pedicle screw misplacement and accidental bone breaches [17]. However, a remaining inconvenience in these methods is that surgeons need to manually select vertebral information from CT images.

## B. Automated Screw Path Planning for PSP

To reduce the time surgeons spend on surgical planning, researchers started looking into automated path planning for PSP [18], [19]. In 2016, Knez *et al.* proposed a novel method for automatic preoperative planning of thoracic pedicle screws. The proposed method extracted geometrical properties of vertebral structures by parametric modeling of the vertebral bodies and pedicles in 3D and combined them with structural properties [14]. Yu *et al.* also implemented a CT-based screw path planning framework by analyzing the cross-sections of the reconstructed 3D images [20]. Most of these path planning methods rely on geometrical similarity between vertebrae and can hence not be fully automated as patient specificity is not considered. Qi *et al.* proposed a deep learning-based screw path planning framework using preoperative CT scans of each vertebra [15]. The program established a local coordinate system and recognized the feature points to identify screw paths. As much as 93% of the computed paths were considered well-placed based on surgeon feedback.

In all these methods, planning is based on CT, subjecting patients to undesirable radiation and involving time-consuming preoperative consultations. By intraoperatively deriving the trajectories from US, these concerns are alleviated in this work. Faster methods would also facilitate intraoperative re-planning, e.g. to cope with effects of unintended patient motion.

## C. Contributions and Organization

The main contributions of this paper include 1) implementing a radiation-free robot-assisted US-based framework to provide intraoperative anatomical features of the spine; 2) developing a data-driven screw path planner to automatically compute the pedicle canal path based on anatomical features and vertebrae shapes. At this stage, the work focuses on minimal invasive screw placement and is validated on regular, synthetic and static vertebrae.

The remainder of this paper is organized as follows. Section II describes the methodology. This section discusses the US reconstruction, the proposed landmark detection framework and screw path computation. Section IV then presents the results validating the feasibility of the proposed method. Finally, Sections V and VI discuss the results and conclude this work.

# II. METHODS

## A. System Overview

Fig. 1 demonstrates the proposed framework of the robot-assisted PSP. It consists of three main components: robotic ultrasound scanning, automatic pedicle screw path computation, and robotic pedicle screw drilling. Firstly, robotic US scanning is performed on the phantom, which enables localizing the vertebrae in the robot or world frame {$w$}. The vertebrae phantom is placed in the robot workspace such that the spine axis is parallel to the robot x-axis and the spinous process is pointing up (the z-axis of the robot). The 3D US surface is reconstructed from the raw 2D US images and the robot end effector poses. By knowing the 3D surface, the local frame for each vertebra is established, and the screw path is planned in the vertebral frame {$v$}. Then, the planned screw path is converted to the robot base or world frame {$w$} to be executed by the robotic drilling system on the phantom vertebrae. Subsequently, the methods for landmark detection and automated screw path computation based on the 3D US reconstruction (blue area in Section I) are elaborated. In a complete setup, the computed screw paths are subsequently used as a reference for the robot-assisted drilling of pre-holes in the pedicles which takes place under the surgeon's supervision.

The practical setup consists of an US imaging system (Sonosite, FUJIFILM, USA) and a lightweight robotic arm (KUKA Med7, Augsburg, Germany), as shown in Fig. 2. The US system uses a 7.5 MHz linear probe. A frame grabber (Epiphan Systems Inc. Palo Alto, Canada) is employed to capture the US images via a USB port at 50 Hz. A custom-designed US probe holder is mounted onto the end effector of the robot. The phantoms are custom-designed by immersing the 3D-printed vertebrae, segmented from human spine CT scans, into agar volume.

## B. Robotic Ultrasound Reconstruction

3D ultrasound reconstruction consists of registering each acquired 2D US image to a 3D frame. Thanks to the robotic scanning, the pose of the US probe {$US$} in the world frame {$w$} is known at all times. Then, to determine the 3D spatial transformation from the image frame to the robot frame, a
<figure><img src="image_4.png" /><figcaption>Fig. 3. An example of a 3D US reconstruction of a vertebra and corresponding CT model. As the US probe is imaging perpendicular to the back, the US reconstruction includes only the posterior surface of the vertebra (the sagittal walls of the spinous process are not visible). The indicated features are the spinous process (sp), transverse processes (tp), and superior articular process (sap).</figcaption></figure>

calibration step is required. Here, the spatial and temporal calibration is implemented and validated with a custom-designed Z phantom, as described previously [21], [22].

A robotic scanning approach with admittance control is performed to acquire US images, as introduced in [23]. Before scanning, the operator co-manipulates the robot with the US probe to specific points on the skin covering the area of the vertebra. The taught points are stored, and the robot scans automatically over the phantom surface along these set points. To retain a good US image, the normal force along the probe axis is kept constant at 5 N. The US images and corresponding probe poses are recorded continuously and then processed.

Firstly, the raw US images are converted into grayscale and cropped to 480 × 480 pixels. Secondly, binary images are generated by applying a Gaussian filter and morphological operator (closing) to remove sparkles. A deep learning-based network, U-Net [24], segments the US images and extracts the edges of different anatomical features at the pixel level. The network is trained using a set of 400 manually collected 2D US images from the agar phantom. The recorded images are manually labeled, and the set is augmented to 800 images by left-right flip and random rotation. The models are trained for 20 epochs with a $1e^{-4}$ learning rate. The edges are extracted from the segmented images with a Canny edge detection algorithm and used to 3D reconstruct the upper surface of the vertebra, as shown in Fig. 3. Finally, the reconstructed point cloud is generated in the robot base, which is here the same as the world frame {$w$}.

### C. Landmark Detection and Establishment of Coordinate Systems

From the 3D reconstruction of the upper surface of the vertebra expressed in frame {$w$}, special vertebra feature

<figure><img src="image_5.png" /><figcaption>Fig. 4. Illustration of landmark definition on the vertebra surface of CT phantom in the axial plane and the sagittal plane. The indicated features are spinous process (sp), transverse processes (tp), and superior articular process (sap).</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>Fig. 5. The procedure of establishing the local coordinate system {$v$} on each vertebra. The axes are defined by the landmarks detected on the vertebral surface.</figcaption></figure>

points are recognized, upon which a local vertebra frame {$v$} can be established. The features that are recognized are spinous process (sp), transverse processes (tp), and superior articular process (sap) as shown in Fig. 4.

For each vertebra, a bounding box is generated into the world frame {$w$} (see Fig. 2) in order to isolate it from the other vertebrae. First, remembering that the x-axis of the world frame {$w$} is parallel to the spine axis, the points at the end of the transverse processes are marked as $p_{tpL}$ and $p_{tpR}$ by considering the y-coordinates and extracting the leftmost and rightmost point of the surface. The points $p_{tpL}$ and $p_{tpR}$ generate a vector $v_1 = p_{tpR} - p_{tpL}$.

Second, the far dorsal point of the vertebra is marked as the extremity of the spinous process $p_{sp}$ by extracting the point with maximum value in the z-coordinate from the surface of the vertebra. Subsequently, to have continuous top mesh of spinous process, a region $S_{sp}$ is defined as comprising all pixels within 4 mm in the z-coordinate of the point $p_{sp}$. By extracting from the region $S_{sp}$ the most superior and inferior points along the x-axis, two other points, respectively $p_{spSup}$ and $p_{spInf}$, are marked on the spinous process. Points $p_{spSup}$ and $p_{spInf}$ generate then a vector $v_2 = p_{spInf} - p_{spSup}$.

Third, the local vertebra frame {$v$} is established as follows:

$$o = \frac{p_{tpR} + p_{tpL}}{2}, \qquad (1)$$

$$y = \frac{v_1}{|v_1|}, \quad z = \frac{v_2 \times y}{|v_2 \times y|}, \quad x = y \times z, \qquad (2)$$

$$R = [xyz], \qquad (3)$$

where $o = [o_x, o_y, o_z]^T$ is the origin of the frame {$v$} with respect to frame {$w$}, while $x$, $y$, $z$, and $R$ are the unit normal vectors and the corresponding rotation matrix, respectively.
Then, the transformation matrix from the vertebra frame {$v$} to the world frame {$w$} is defined as

$$ w_v T = \begin{bmatrix} [x y z] & [o_x o_y o_z]^T \\ \mathbf{0} & 1 \end{bmatrix} = \begin{bmatrix} \mathbf{R} & \mathbf{o} \\ \mathbf{0} & 1 \end{bmatrix}. \quad (4) $$

After the vertebral frame {$v$} is defined, two extra points located on the superior articular processes are extracted. Considering only the region $x < 0$ in the vertebra frame, the points with maximum value in the z-coordinate are retrieved, once for $y > 0$ and once for $y < 0$. These two extra points are marked $p_{sap_L}$ and $p_{sap_R}$, respectively. Fig.4 displays the landmarks detected from the upper surface of the vertebra.

Screw paths are defined as a set of entry point EP and a screw direction unit vector $v = [v_x v_y v_z]^T$ for each pedicle. The vector $v$ is represented with an angle of rotation $\theta_a$ about the x-axis in the axial plane and an angle $\theta_s$ about the y-axis in the sagittal plane as follows:

$$ \left\{ \begin{array}{l} \theta_s = atan2(-v_y, v_z) \\ \theta_a = atan2(v_x, \sqrt{v_y^2 + v_z^2}). \end{array} \right. \quad (5) $$

Then the screw direction unit vector becomes

$$ \mathbf{v}_{L,R} = [\sin(\theta_a) - \sin(\theta_s)\cos(\theta_a)\cos(\theta_s)\cos(\theta_a)]^T. \quad (6) $$

Since the developed method is based on the posterior surface of the vertebrae and the anterior part of the vertebrae cannot be reconstructed from the US image, the required drilling depth can not be seen on the 3D US surface. Therefore, this study also investigates the feasibility of inferring the drilling depth $d$ from the upper surface of the vertebra.

### D. Screw Path Computation

Due to the similarity in lumbar vertebral structures among humans, there's potential for predicting vertebral morphometry through statistical correlations of anatomical dimensions. Past studies suggest that this insight could be used to create simplified geometric models of the spine [25], [26]. Hence a linear model is applied to estimate the correlation between the landmarks and the screw path. The linear model is fit by a linear regression with coefficients that minimize the residual sum of squares between the observed targets in the data set and the predicted path by the linear approximation. For each vertebra, the measured vector of landmarks is defined as $v_i \mathbf{a}$, which is expressed in the vertebra frame. For each output screw path, the output vector is $v_i \mathbf{b}$, which depicts the screw path for both pedicles of the vertebra, is given as follows:

$$ v_i \mathbf{a} = [\mathbf{p}_{tP_L}, \mathbf{p}_{tP_R}, \mathbf{p}_{sap_L}, \mathbf{p}_{sap_R}, \mathbf{p}_{sp}, \mathbf{p}_{sp_{SUP}}, \mathbf{p}_{sp_{Inf}}]_{1 \times 21}, \quad (7) $$

$$ v_i \mathbf{b} = [EP, \theta_s, \theta_a, d]_{1 \times 6}, \quad (8) $$

where EP, $\theta_s$, and $\theta_a$ are the entry point position and the screw path angle in the sagittal and axial plane, and $d$ is the required drilling depth extracted ideally from desired path in CT. A linear model $M_{21 \times 6}$ converts $v_i \mathbf{a}$ to $v_i \mathbf{b}$ with $v_i \mathbf{a} M = v_i \mathbf{b}$. The parameters of this model are optimized such that errors

between the predicted screw path and the measured path are minimized.

$$ J = \sum_{i=1}^{N} (v_i a M - v_i b)^2 \quad (9) $$

with index $i \in [1, 2, \dots, N]$ the index of each screw path, and $N$ the total number of screw paths.

## III. EXPERIMENT AND EVALUATION

### A. Cross Validation on CT Dataset

The proposed system is first validated on a public CT dataset of human spines [27], [28]. There are 45 vertebrae from 9 volunteers (L1 to L5) resulting in 90 possible screw paths (2 paths × 5 vertebrae × 9 volunteers). For means of validation, the posterior surface was extracted from the vertebral body manually in the CT images. Screw paths were defined manually such as to pass through the middle of the pedicles and not cause any breach; such kind of path has a maximum clearance from the pedicle wall and starts from on the surface of the vertebrae and ends before the anterior cortical layer; it does not cause perforation to the cortical layer [29]. These manually planned paths serve as ground truth for evaluation. A five-fold cross-validation is performed, in which 80% of the data is used for training, and 20% is used for validation. Then, the trained model on a CT-planned path shows the ideal case when the US scanning is able to provide the posterior surface completely.

### B. Testing on US Reconstruction

The testing is subsequently conducted on the 3D reconstructed models from the US scans. In total, 5 lumbar spine phantoms (L1 to L5) are scanned and reconstructed, resulting in 50 screw paths (2 paths × 5 vertebrae × 5 phantoms). For evaluation, the planned screw path is registered to the preoperative CT frame by using iterative closest point registration with its associated 3D US reconstructed surface [30]. Then the geometry errors, including position and orientation errors, between the computed screw path and ground truth screw path are calculated. The orientation errors are measured in the axial plane $e_{\theta_a}$ and sagittal plane $e_{\theta_s}$ (see Fig.6).

<figure><img src="image_7.png" /><figcaption>Fig. 6. Definition of the position and orientation error in axial and sagittal views. The planned screw path is in red, while the preoperative plan is in dark. The upper surface of the vertebra is white, while the vertebral body and pedicle are labeled in orange.</figcaption></figure>
### C. Surgical Gertzbein Grading Outcome

In the context of screw planning, success is influenced by the screw diameter. If the screw diameter is inappropriately large, a breach will happen, causing damage to the surrounding nerves and blood vessels. Whereas if the diameter of the screw is too small, the stability of the screw will be affected. To evaluate the surgical outcome of pedicle screw placements, Gertzbein grading is introduced as an index to demonstrate how successful PSP is in the actual surgical situation [31]. Gertzbein grading of a planned screw path is achieved by extrapolating the drill bit diameter to common pedicle screw diameters, such as 4, 6, and 7 mm, along the planned trajectory [32]. In Gertzbein grading, four different scales are defined as Grade A (within the pedicle without cortical breach), Grade B (up to 2mm breach, minor cortical breach), Grade C (more than 2 and up to 4 mm breach, moderate cortical breach), and Grade D (more than 4 mm breach, severe cortical breach) [31]. According to this ranking, the Grade A and B are considered safe (no pedicle breach or below 2mm), and the screw path is labeled successful; otherwise, it is unsuccessful. The clinical accuracy of a method or procedure is calculated as the ratio of all successful trajectories to all planned trajectories.

## IV. RESULTS

### A. Validation on CT Phantoms

Tab. I demonstrates the results of the cross-validation on the CT data. The mean position and orientation errors are calculated over the 90 screw paths. The mean position error is $-0.17 \pm 1.69$, $0.20 \pm 1.43$ and $0.01 \pm 1.43$ mm along x-, y- and z-axes, respectively. The orientation error is $0.35 \pm 3.93$ and $0.06 \pm 3.76^{\circ}$ in the sagittal and axial plane. The average depth error, marked as *d* in Tab. I, is $0.02 \pm 5.06$ mm along the screw direction. The minimal error is $0.75 \pm 3.89$ mm on subset 4, while the maximum error is $5.03 \pm 4.22$ mm on subset 3.

Tab. III shows the Gertzbein Grade for three different screw diameters; the screw path for each vertebra is generated from the CT superior surface. While the screw diameter increases from 4 mm to 7mm, the success rate over the 90 screws decreases from 100% to 94%.

### B. Results on US Reconstruction

Tab. II demonstrates the screw path errors on the 50 US planned trajectories. The position error is $0.48 \pm 2.18$, $0.13 \pm 2.16$ and $0.06 \pm 4.08$ mm along the x-, y-, and z-axes. The orientation errors are below 0.5 degree. However, the error of the planned depth is $-2.87 \pm 4.93$ mm.

Tab. IV shows the Gertzbein Grade for three different screw diameters following the US-based planned screw paths registered on the 3D CT model of each vertebra. With a screw diameter of 4 mm, the success rate over the 50 screws was 98%, while when the screw diameter increased to 6 mm, the success rate decreased to 84%. Finally, 28 out of 50 screws achieve Grade A for a 7 mm screw.

Fig. 7 demonstrates US-based planned screw path on the US reconstructed surface from top, axial and sagittal views. For visualization, the 3D US reconstruction is registered with the

<table><caption>TABLE I
VALIDATION OF THE COMPUTED SCREW PATHS WITH THE CT DATA.</caption><thead><tr><th rowspan="2">Error</th><th colspan="5">Cross Validation Subset</th><th rowspan="2">Average<br>(1 to 5)</th></tr><tr><th>1</th><th>2</th><th>3</th><th>4</th><th>5</th></tr></thead><tbody><tr><td>x[mm]</td><td>1.02<br>± 1.94</td><td>-0.16<br>± 2.14</td><td>-0.81<br>± 1.73</td><td>-1.93<br>± 2.80</td><td>1.02<br>± 1.93</td><td>-0.17<br>± 1.69</td></tr><tr><td>y[mm]</td><td>-0.76<br>± 1.6</td><td>0.31<br>± 1.59</td><td>0.65<br>± 1.14</td><td>-0.04<br>± 2.28</td><td>0.83<br>± 2.18</td><td>0.20<br>± 1.43</td></tr><tr><td>z[mm]</td><td>-0.23<br>± 5.21</td><td>-1.00<br>± 4.54</td><td>0.19<br>± 3.16</td><td>2.35<br>± 3.26</td><td>-1.31<br>± 4.83</td><td>0.01<br>± 3.39</td></tr><tr><td>θ<sub>s</sub> [°]</td><td>0.35<br>± 6.22</td><td>-0.51<br>± 4.19</td><td>-0.03<br>± 4.11</td><td>-1.48<br>± 4.16</td><td>3.43<br>± 5.76</td><td>0.35<br>± 3.93</td></tr><tr><td>θ<sub>a</sub> [°]</td><td>-0.12<br>± 3.36</td><td>-0.26<br>± 3.89</td><td>3.08<br>± 2.63</td><td>-0.74<br>± 4.42</td><td>-1.65<br>± 7.75</td><td>0.06<br>± 3.76</td></tr><tr><td>d<br>[mm]</td><td>-0.98<br>± 7.45</td><td>1.27<br>± 6.09</td><td>-5.03<br>± 4.22</td><td>0.75<br>± 3.89</td><td>4.10<br>± 5.09</td><td>0.02<br>± 5.06</td></tr></tbody></table>

<table><caption>TABLE II
ERROR OF THE COMPUTED SCREW PATH FROM US RECONSTRUCTION.</caption><thead><tr><th>Error</th><th>x[mm]</th><th>y[mm]</th><th>z[mm]</th><th>θ<sub>s</sub> [°]</th><th>θ<sub>a</sub> [°]</th><th>d[mm]</th></tr></thead><tbody><tr><td>Value</td><td>0.48<br>± 2.18</td><td>0.13<br>± 2.16</td><td>0.06<br>± 4.08</td><td>-0.34<br>± 3.66</td><td>-0.45<br>± 4.32</td><td>-2.87<br>± 4.93</td></tr></tbody></table>

CT model. The computed screw paths (blue) are compared to the ground truth screw paths (black).

## V. DISCUSSION

This work investigates the feasibility to plan PSP screw trajectories solely based on US. Compared to CT, US only allows visualizing the upper surface of the vertebral bones. For example, the vertebral body shows a very low coverage as it is only visible through the gap in the intervertebral space [33]. Further, US is typically more prone to noise than CT or fluoroscopy. Li et al. found an overall 3D representation error of $1.10 \pm 0.66$ mm in an earlier paper [23]. Even if a landmark is perfectly detected from US, its position can thus be offset from the real position. Nevertheless, the experiments show encouraging results.

To understand to what extent the sensor performance affects the planning, first a planning based on CT is done, but only employing the upper surface of the bone from the CT data. In a sense, this situation would correspond to using a perfect ultrasound sensor. Despite the reduced amount of CT data, the planned screw path still achieves acceptable position and orientation accuracy. The deviation of the average position error is within 2 mm along the x- and y-axes. The positional error along the z-axis is $0.01 \pm 3.39$ mm. One can notice a deviation in the geometry of the spinous process from L1 to L5, which impacts the determination of the landmarks. The average orientation error is approximately 0.35 and 0.06 degrees for $\theta_s$ and $\theta_a$ respectively.

With an average position error of 0.48 and 0.13 mm along the x- and y-axes, the screw path estimation relying on US
<figure><img src="image_8.png" /><figcaption>Fig. 7. Example of a planned screw path (blue line) based on the US reconstructed surface (gray) from top view (a), axial view (b), and sagittal view (c). The 3D US reconstruction is registered with the preoperative surgical plan (black) and CT model (red).</figcaption></figure>

<table><caption>TABLE III
GERTZBEIN GRADE OF PLANNED SCREWS FROM CT RECONSTRUCTION.</caption><thead><tr><th rowspan="2">Screw<br>Diameter [mm]</th><th colspan="4">Gertzbein Grade</th><th rowspan="2">Clinical Accuracy [%]<br>((A+B)/total)</th></tr><tr><th>A</th><th>B</th><th>C</th><th>D</th></tr></thead><tbody><tr><td>4</td><td>89</td><td>1</td><td>0</td><td>0</td><td>100</td></tr><tr><td>6</td><td>81</td><td>7</td><td>2</td><td>0</td><td>97</td></tr><tr><td>7</td><td>69</td><td>16</td><td>5</td><td>0</td><td>94</td></tr></tbody></table>

<table><caption>TABLE IV
GERTZBEIN GRADE OF PLANNED SCREWS FROM US RECONSTRUCTION.</caption><thead><tr><th rowspan="2">Screw<br>Diameter [mm]</th><th colspan="4">Gertzbein Grade</th><th rowspan="2">Clinical Accuracy [%]<br>((A+B)/total)</th></tr><tr><th>A</th><th>B</th><th>C</th><th>D</th></tr></thead><tbody><tr><td>4</td><td>41</td><td>8</td><td>1</td><td>0</td><td>98</td></tr><tr><td>6</td><td>33</td><td>9</td><td>7</td><td>1</td><td>84</td></tr><tr><td>7</td><td>28</td><td>11</td><td>8</td><td>3</td><td>78</td></tr></tbody></table>

reconstruction also obtains a good accuracy. However, the standard deviation is larger than for paths estimated using CT data. Deviations can be attributed to the noise in the US signal and the difficulty in exactly pinpointing the respective geometrical features for each of the vertebral levels. Further, one can remark that the estimated screw depth is more difficult to infer. In a complete setup, this limitation could be overcome with independent breach-detection through other sensors such as done by Saghbiny *et al.* [34], [35]. In such a setup, online breach detection would enable ideal screw length selection and prevent breaching.

Compared to the state-of-the-art approaches (i.e., CT-based planning), the proposed US-based system would still need to be improved. Yu *et al.* achieved a 96.3% path planning accuracy for 164 cervical pedicle screws. Their entry point obtained 1.2 mm accuracy, while the exit point yielded a 2.2 mm accuracy [20]. Qi *et al.* reported a 93% success rate based on the Gertzbein Grade [15]. Bertram *et al.* reported a quantitative and qualitative assessment by using a commercial registration-based screw trajectory suggestion model [36]. The automatically planned trajectories achieved 92.9% for Grade

A and B. Compared to these works, the here reported All-US-based approach obtained a 84% success rate on the Gertzbein Grade for 6mm screw diameter. However, the results from this work could still be improved by designing dedicated scanning patterns that are developed to cover larger surfaces of the spine surface, as well as by an improved estimation of the pedicle area through a more accurate correlation between anatomical landmarks and pedicles. One could e.g. explore querying statistical shape models to estimate the pedicle area. The latter could also help choose a more suitable screw diameter from the US reconstruction as pedicles are hard to investigate only based on the upper surface. This would further enable selecting the appropriate screw diameter and compute a more representative Gertzbein Grade for our approach.

Note that the proposed algorithm is very fast because of low computation cost. The current implementation computes each screw trajectory in 5s from a point cloud of a vertebra, making the approach also appealing for intraoperative use or for quick verification of whether plans are still valid (e.g. after potential patient-movement). The current work aimed to reduce the burden on the surgeon for the PSP path planning. However, reaching the goal of being clinically practical requires investigating the developed approach to a realistic model or human cadaver. The current results showed that using the US as an intraoperative imaging modality is capable of providing surgeon-like path planning.

Regardless of these promising results, some points of attention could be considered in future work to improve the proposed framework. Firstly, future research could investigate a new approach to estimate the screw depth, inferred from the upper surface only, without the vertebral body. Secondly, the estimation could benefit from a more powerful segmentation approach. Methods that are able to assess the reliability of a given landmark would be highly advantageous. More or better landmarks could be detected from the missing area around the spinous and transverse processes. Thirdly, data-driven approaches to estimate incision point, orientation, and pedicle size based on deep-learning-based methods could also be explored such that hidden trends or relations could be encoded implicitly in the network. It is clear that a lot of data would need to be acquired at this point. Fourth, the drilling trajectory was
now computed analytically from identified feature points. The relations were here defined by an engineer. It would be relevant to involve clinicians in this process as they may identify more successful strategies (e.g. aligning closer to their daily clinical practice).

## VI. CONCLUSION

This work reports the first result of US-based path planning method for assisting PSP. The anatomical characteristics are recognized from the US point cloud. A regression model is then employed to derive drilling trajectories in the vertebral coordinate frames. The experimental results on the US reconstruction show $0.48 \pm 2.18$, $0.13 \pm 2.16$ and $0.06 \pm 4.08$ mm position accuracy along the x-, y-, and z-axes, respectively. The orientation error is $-0.4 \pm 3.66$ and $-0.45 \pm 4.32^\circ$ along the sagittal and axial plane. All in all, the proposed method demonstrates promising potential to reduce radiation and workload on surgeons.

## REFERENCES

[1] B. I. Martin et al., "Trends in lumbar fusion procedure rates and associated hospital costs for degenerative spinal diseases in the united states, 2004 to 2015," *Spine*, vol. 44, p. 369-376, Mar. 2019.

[2] N.-F. Tian et al., "Pedicle screw insertion accuracy with different assisted methods: a systematic review and meta-analysis of comparative studies," *Eur. Spine J.*, vol. 20, no. 6, pp. 846-859, Jun 2011.

[3] A. S. Vaishnav, Y. A. Othman, S. S. Virk, C. H. Gang, and S. A. Qureshi, "Current state of minimally invasive spine surgery," *Journal of spine surgery*, vol. 5, no. Suppl 1, p. S2, 2019.

[4] D. Lau, S. W. Terman, R. Patel, F. La Marca, and P. Park, "Incidence of and risk factors for superior facet violation in minimally invasive versus open pedicle screw placement during transforaminal lumbar interbody fusion: a comparative analysis," *Journal of Neurosurgery: Spine*, vol. 18, no. 4, pp. 356-361, 2013.

[5] X. J. Su et al., "Comparison of accuracy and clinical outcomes of robot-assisted versus fluoroscopy-guided pedicle screw placement in posterior cervical surgery," *Global Spine Journal*, vol. 12, no. 4, pp. 620-626, 2022.

[6] M. Farshad, M. Betz, N. A. Farshad-Amacker, and M. Moser, "Accuracy of patient-specific template-guided vs. free-hand fluoroscopically controlled pedicle screw placement in the thoracic and lumbar spine: a randomized cadaveric study," *European Spine Journal*, vol. 26, pp. 738-749, 2017.

[7] K. A. Shafi, Y. A. Pompeu, A. S. Vaishnav, E. Mai, A. Sivaganesan, P. Shahi, and S. A. Qureshi, "Does robot-assisted navigation influence pedicle screw selection and accuracy in minimally invasive spine surgery?" *Neurosurgical Focus*, vol. 52, no. 1, 2022.

[8] B. o. Jung, "Robotic-assisted versus fluoroscopic-guided surgery on the accuracy of spine pedicle screw placement: A systematic review and meta-analysis," *Cureus*, vol. 16, no. 2, 2024.

[9] Y. L. Yang, D. S. Zhou, and J. L. He, "Comparison of isocentric c-arm 3-dimensional navigation and conventional fluoroscopy for c1 lateral mass and c2 pedicle screw placement for antlatoaxial instability," *Clinical Spine Surgery*, vol. 26, no. 3, pp. 127-134, 2013.

[10] H.-E. Gueziri, C. Santaguida, and D. L. Collins, "The state-of-the-art in ultrasound-guided spine interventions," *Medical Image Analysis*, vol. 65, p. 101769, Oct. 2020.

[11] R. Li et al., "Ultrasound-based robot-assisted drilling for minimally invasive pedicle screw placement," *IEEE Transactions on Medical Robotics and Bionics*, 2024.

[12] P. Tu et al., "Ultrasound image guided and mixed reality-based surgical system with real-time soft tissue deformation computing for robotic cervical pedicle screw placement," *IEEE Transactions on Biomedical Engineering*, vol. 69, no. 8, pp. 2593-2603, 2022.

[13] J. Zhao et al., "Pedicle screw path planning for multi-level vertebral fixation," *Medical Physics*, vol. 51, no. 3, p. 1547-1560, Jan. 2024.

[14] D. Knez, B. Likar, F. Pernuš, and T. Vrtovec, "Computer-assisted screw size and insertion trajectory planning for pedicle screw placement surgery," *IEEE transactions on medical imaging*, vol. 35, no. 6, pp. 1420-1430, 2016.

[15] X. Qi et al., "An automatic path planning method of pedicle screw placement based on preoperative ct images," *IEEE Transactions on Medical Robotics and Bionics*, vol. 4, no. 2, pp. 403-413, 2022.

[16] A. Chan et al., "3d ultrasound navigation system for screw insertion in posterior spine surgery: a phantom study," *International Journal of Computer Assisted Radiology and Surgery*, pp. 1-11, 2022.

[17] E. A. Gonzalez, A. Jain, and M. A. L. Bell, "Combined ultrasound and photoacoustic image guidance of spinal pedicle cannulation demonstrated with intact ex vivo specimens," *IEEE Transactions on Biomedical Engineering*, vol. 68, no. 8, pp. 2479-2489, 2021.

[18] J. Lee, S. Kim, Y. S. Kim, and W. K. Chung, "Optimal surgical planning guidance for lumbar spinal fusion considering operational safety and vertebra-screw interface strength: Optimal surgical planning guidance for lumbar spinal fusion," *Int. J. Med. Robot. Comput. Assist. Surg.*, vol. 8, no. 3, pp. 261-272, Sep 2012.

[19] R. Wicker and B. Tedla, "Automatic determination of pedicle screw size, length, and trajectory from patient data," in *Proc. 26th Annu. Int. Conf. IEEE Eng. Med. Biol. Soc.*, Sep 2004, pp. 1487-1490.

[20] Z. Yu et al., "Application of a novel 3d drill template for cervical pedicle screw tunnel design: a cadaveric study," *European Spine Journal*, vol. 26, pp. 2348-2356, 2017.

[21] R. Li, K. Niu, and E. V. Poorten, "A framework for fast automatic robot ultrasound calibration," in *2021 International Symposium on Medical Robotics (ISMR)*, 2021, pp. 1-7.

[22] R. Li, Y. Cai, K. Niu, and E. V. Poorten, "Comparative quantitative analysis of robotic ultrasound image calibration methods," in *2021 20th International Conference on Advanced Robotics*, 2021, pp. 511-516.

[23] R. Li et al., "Robot-assisted ultrasound reconstruction for spine surgery: from bench-top to pre-clinical study," *International journal of computer assisted radiology and surgery*, pp. 1-11, 2023.

[24] O. Ronneberger, P. Fischer, and T. Brox, "U-net: Convolutional networks for biomedical image segmentation," in *Medical Image Computing and Computer-Assisted Intervention 2015*, 2015, pp. 234-241.

[25] B. Ding, T. Zhou, and J. Zhao, "A novel system for accurate lumbar spine pedicle screw placement based on three-dimensional computed tomography reconstruction," *Journal of Orthopaedic Translation*, vol. 23, pp. 101-106, 2020.

[26] M. E. Kunkel, H. Schmidt, and H.-J. Wilke, "Prediction equations for human thoracic and lumbar vertebral morphometry," *Journal of anatomy*, vol. 216, no. 3, pp. 320-328, 2010.

[27] online data set: [Online]. Available: https://www.yeggi.com/q/lumbar+spine/

[28] data set: [Online]. Available: https://www.embodi3d.com/blogs/entry/401-9-great-3d-printed-models-of-the-spine/

[29] D. Knez, J. Mohar, R. J. Cirman, B. Likar, F. Pernuš, and T. Vrtovec, "Manual and computer-assisted pedicle screw placement plans: a quantitative comparison," in *Computational Methods and Clinical Applications for Spine Imaging: 4th International Workshop and Challenge*, 2016, pp. 105-115.

[30] W. Guan, W. Li, and Y. Ren, "Point cloud registration based on improved icp algorithm," in *2018 Chinese Control And Decision Conference (CCDC)*, 2018, pp. 1461-1465.

[31] S. D. Gertzbein and S. E. Robbins, "Accuracy of pedicular screw placement in vivo," *Spine*, vol. 15, no. 1, pp. 11-14, 1990.

[32] G. Burström et al., "Feasibility and accuracy of a robotic guidance system for navigated spine surgery in a hybrid operating room: a cadaver study," *Scientific reports*, vol. 10, no. 1, p. 7522, 2020.

[33] H.-E. Gueziri, O. Rabau, C. Santaguida, and D. L. Collins, "Evaluation of an ultrasound-based navigation system for spine neurosurgery: a porcine cadaver study," *Frontiers in Oncology*, vol. 11, p. 619204, 2021.

[34] E. Saghbiny, J. Da Silva, C. Chaimi, T. Chandanson, G. Morel, and R. Vialle, "Toward automatic bone breach detection for spine surgery using tissue bio-electrical conductivity sensing," in *Conference on New Technologies for Computer and Robot Assisted Surgery*, 2022.

[35] L. Leblanc, E. Saghbini, J. Da Silva, A. Harlé, S. Vafadar, T. Chandanson, R. Vialle, G. Morel, and B. Tamadazte, "Automatic spinal canal breach detection during pedicle screw placement," *IEEE Robotics and Automation Letters*, 2024.

[36] U. Bertram et al., "Man versus machine: Automatic pedicle screw planning using registration-based techniques compared with manual screw planning for thoracolumbar fusion surgeries," *The International Journal of Medical Robotics and Computer Assisted Surgery*, p. e2570, 2023.