<figure><img src="image_1.png" /></figure>

HAL
open science

# Robotic Path Re-planning for US Reconstruction of the Spine

Kaat Van Assche, Ruixuan Li, Ayoob Davoodi, Matthias Tummers, Mouloud Ourak, Gianni Borghesan, Nicola Cavalcanti, Philipp Fürnstahl, Emmanuel Vander Poorten

► To cite this version:

Kaat Van Assche, Ruixuan Li, Ayoob Davoodi, Matthias Tummers, Mouloud Ourak, et al. Robotic Path Re-planning for US Reconstruction of the Spine. IEEE Transactions on Medical Robotics and Bionics, 2025, 7 (2), pp.755-767. 10.1109/TMRB.2025.3550662. hal-04985909

HAL Id: hal-04985909

https://hal.science/hal-04985909v1

Submitted on 11 Mar 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Open licence - etalab
# Robotic Path Re-planning for US Reconstruction of the Spine

Kaat Van Assche, Ruixuan Li, Ayoob Davoodi, Matthias Tummers, Mouloud Ourak, Gianni Borghesan, Nicola Cavalcanti, Philipp Fürnstahl, and Emmanuel Vander Poorten

**Abstract**—Robotic ultrasound (US) scanning of the spine is becoming an increasingly viable radiation-free alternative to CT scans and fluoroscopy. However, due to the complex shape of the vertebra, three-dimensional (3D) US reconstructions generated from two-dimensional (2D) US scans often lack important anatomic information, such as the spinous process. This paper investigates scanning strategies that reorient the probe during US scanning to improve surface coverage of 3D US reconstructions. A two-scan procedure with a path re-planning algorithm is presented. The proposed algorithm uses information from a first exploratory scan to generate an improved imaging trajectory whereby the US probe is near-perpendicular to the targeted bone surface. The results show a 30.4%, 42.3%, and 75.0% improvement in surface coverage on a synthetic phantom, cadaver, and human volunteers, respectively, achieving up to 56% surface coverage on human volunteers. These results emphasise the value of exploiting information about the underlying anatomy to optimise the scanning trajectory. The increased surface coverage of the 3D US reconstructions will provide higher quality radiation-free visualisation, extending the role of US as a complementary imaging modality for safe and effective diagnosis and spine interventions.

**Index Terms**—Ultrasound guidance, 3D ultrasound reconstruction, Spine surgery, Robotic path planning

## I. INTRODUCTION

ULTRASOUND (US) technology is increasingly applied in clinical practice as a radiation-free imaging modality for visualising the patient's anatomy. Unlike imaging modalities based on X-rays, such as Computed Tomography (CT) and fluoroscopy, US does not emit ionisation radiation, which is harmful to both the patient and the clinician. Aside from safety, US is also compact, real-time, cost-effective, and non-invasive. As such, it is receiving increasing research interest, even for spine surgical interventions [1]. For example, for diagnostic purposes, spine curvatures (Cobb angle) can be calculated from three-dimensional (3D) US visualisations to identify structural spinal disorders such as scoliosis [2], [3]. US can also be used to provide intraoperative guidance for procedures

This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement NO.101016985 (FAROS) and Flemish Research Foundation under grant agreement NO.GOA1420N, NO.1SHGA24N, and NO.1S363222N.

K. Van Assche, R. Li, A. Davoodi, M. Tummers, M. Ourak, G. Borghesan, and E. Vander Poorten are with the Robot-Assisted Surgery group, Department of Mechanical Engineering, KU Leuven, Leuven, Belgium (kaat.vanassche@kuleuven.be).

G. Borghesan is with Core Lab ROB, Flanders Make, Leuven, Belgium.

N. Cavalcanti and P. Fürnstahl are with the Research in Orthopedic Computer Science Group, Balgrist University Hospital, University of Zurich, Zurich, Switzerland.

<figure><img src="image_3.png" /><figcaption>Fig. 1. Surface meshes of 3D US reconstructions of robotic US scans on a lumbar vertebra of a synthetic phantom at a constant probe pitch angle of 0°, 5°, 15°, 25°. Top right shows the reconstructed reference dorsal CT surface.</figcaption></figure>

such as pedicle screw placement [4], [5], needle injections [1], or point of care examination of spine injuries [6].

In US images, bone appears as a hyperechoic line representing the interface between the bone cortex and the adjacent tissues [7]. The direction of the reflected US wave depends on the inclination of the bone surface. The intensity of the reflection is maximal when the incident US wave arrives perpendicular to the surface [8]. Therefore, in addition to tissue attenuation, imaging artefacts, and limited field of view, the specific imaging angle can cause the bony surfaces to appear blurred and hard to discern from other structures. This complicates the understanding of two-dimensional (2D) US images and, hence, requires substantial experience and training of sonographers [9]. As a result, US has not yet become a standard imaging modality in orthopaedics [10].

### A. 3D US vertebrae reconstruction

As a single 2D US image is hard to interpret, interest has grown towards creating 3D US reconstructions based on segmented 2D US images to come to a more intuitive visualisation of the targeted anatomy. Vertebrae can be segmented and reconstructed from freehand US scanning systems [3], [11],
[12]. Probe manipulation puts a lot of stress on the sonogra-
pher's wrists. It was shown that 85.5% of sonographers scan
in pain and have a high risk of developing musculoskeletal
disorders [13], [14]. Furthermore, the quality of the produced
US scan highly depends on the sonographer's proficiency [9],
[15]. Therefore, research into robotic US scanning systems
has been conducted [16]–[19]. Not only can the robotic US
scanning system achieve similar 3D reconstruction results (e.g.
Cobb angles) as an expert sonographer [17], but it can also
allow for a more consistent performance and can more easily
maintain constant scanning parameters [16]. For example,
Zhang et al. showed that using a US scanning system allowed
to optimise the scanning speed and contact force during
automatic scanning of the human spine [18]. Li et al. and
Suligoj et al. developed force-controlled robotic US scanning
systems that respectively maintain a consistent contact force of
$5.64 \pm 1.10$ N and $6.11 \pm 1.18$ N with the skin surface to ensure
proper skin-to-probe contact and achieve a good acoustic
coupling [19]–[21]. The resulting 3D US reconstructions from
Li et al. achieved 3D registration errors between 1.59 mm and
1.93 mm on human cadavers [20].

B. *Efforts to improve 3D US reconstructions*

However, US image quality is highly variable, and 3D US reconstructions of the spine are typically incomplete. State of the art works [20], [3] experience challenges to capture some regions of the spine, such as the sides of the spinous process. For scoliosis diagnosis, where only the curvature of the spine needs to be determined, the current information may be sufficient. However, when considering US-guided spine surgery and e.g. assistance during pedicle screw placement (PSP), it is important to increase the coverage of the spine surface to identify important landmarks. For example, in lumbar PSP the transverse process, the facet joint, and the lamina provide important information to accurately plan drilling paths [22].

Recent efforts to improve 3D US reconstruction quality focused on using deep learning to improve automatic segmentation of bone contours from 2D US images [3], [20]. However, even the best segmentation algorithm will fail if there is no visible contour due to suboptimal imaging planes. It is thus crucial to visualise along appropriate imaging planes. Along these lines, [23] and [24] developed approaches to mimic the decision-making process of an expert sonographer. In [25], Ma et al. propose a self-normal-positioning approach, relying on four laser distance sensors, to ensure that the US probe maintains orthogonal to the skin surface. Li et al. achieve similar results using a force control approach [20].

However, image-quality also depends on the underlying
anatomic structure, which is not addressed in prior works.
To the best of the authors’ knowledge, no existing systems
explicitly account for the underlying spine anatomy. Fig. 1
shows how the 3D US reconstruction of the vertebra differs
significantly as a function of the constant probe pitch angle.
This pitch angle, also referred to as the probe tilting angle,
is defined as the rotation of the US probe about its xᵢ-axis
(see Fig. 1). When visualising complex bony structures such
as the spine, optimising the orientation of the US probe

relative to the (initially unknown) underlying bony surface,
rather than with respect to the skin surface, could improve
reconstruction. Given the complex bone surface, an adaptive
pitch angle seems more appropriate than a certain constant
pitch angle. Therefore, this paper proposes a new path re-
planning algorithm to determine imaging angles accounting
for the underlying anatomy.

C. Evaluation of 3D US reconstructions

To systematically assess the value of a particular scanning
strategy, performance metrics are required. However, as far
as the authors are aware, there is no standard for evaluat-
ing the quality and completeness of 3D US reconstructions.
The state-of-the-art often assesses the quality of anatomical
reconstructions by comparing important geometries required
for the clinical application, such as the Cobb angle and
the femoral anteversion angles for scoliosis and osteometry
respectively [3], [17], [26]–[28]. The limitation of these single
geometry-based approaches is that the entire shape of the
reconstruction is not considered. By comparing a 3D US
reconstruction to a ground truth (e.g. a 3D CT reconstruction),
the correctness of the shape can be evaluated. Li et al. calculate
the “3D representation error” by taking the root-mean-square
error (RMSE) of the closest distances from each point of
the US point cloud to the CT point cloud [20], [29]. This
evaluation metric is close to what is referred to as precision.
However, as reported by Maboudi et al., both correctness
(precision) and completeness (recall or surface coverage) are
important when evaluating 3D reconstructions [30]. Evaluating
the surface coverage of the 3D US reconstruction is especially
important in this study, but should not lead to excessive cost
of precision or scanning time. Gueziri et al. used a surface
coverage metric to evaluate 3D US spine reconstructions, but
the metric is not normalised and hence difficult to compare
between different vertebrae and models [31]. In the computer
vision community, Scott et al. and Border et al. present
a combination of metrics to evaluate the trade-off between
surface coverage, path planning efficiency (ratio of actual and
optimal path length) and computation time to quantify the path
planning performance [32], [33]. However, to the best of the
authors’ knowledge, this type of path planning evaluation has
not been used for medical applications. This study aims to
apply the metrics presented by Maboudi et al., Scott et al.
and Border et al. to create metrics specific to path planning
for 3D US reconstructions of vertebrae.

D. Research objectives

Compared to the state-of-the-art, the present work:

* presents a path re-planning algorithm to optimise the US probe pitch angle with respect to the bone surface;

* introduces a set of objective performance metrics for assessing the quality of spine reconstruction;

* assesses the importance of US probe orientation through validation via robotic US-scans on a synthetic phantom, on cadavers, and on healthy human volunteers.

The paper is structured in seven sections. After this intro-
ductory section, Section II provides an overview of the robotic
<figure><img src="image_4.png" /><figcaption>Fig. 2. The different frames of the robotic system: robot base {$b$}, end-effector {$ee$}, probe tip {$t$}, vertebra {$v$}, and US image frame {$us$}. The transformation matrix from {$us$} to {$ee$}, required for image calibration, is defined as $^{ee}_{us}T$. The robot scans medio-laterally with an 'S'-shape scanning trajectory across the lumbar spine.</figcaption></figure>

3D US reconstruction system; including the robot control, an image processing pipeline to generate the reconstructions, and an automatic landmark extraction method. Section III describes the path re-planning algorithm and Section IV presents the evaluation metrics that are proposed to analyse the quality of the resulting 3D US reconstructions. Section V outlines the experimental setup, the planned trajectories per experiment and the corresponding results. These results are discussed in Section VI, followed by a conclusion in Section VII.

## II. ROBOTIC 3D US RECONSTRUCTIONS

### A. Robotic system and initial path generation

A robotic system, as described by [20], was used to execute the desired scanning paths. The developed system employs an admittance-based hybrid control scheme to ensure both positional accuracy and a constant contact force between the US probe and the skin. This constant contact force ensures proper skin-to-probe contact and patient safety by avoiding applying excessively large forces. Fig. 2 shows the robotic system and defines the robot base {$b$}, the end-effector {$ee$}, the tip of the robot-held US probe {$t$}, and the vertebra {$v$} coordinate frames. A constant force $F_{t,z}$ is applied along the $z_t$-axis between the probe and the human body. Using a 6 degree of freedom (DOF) force torque sensor, the wrench $w_m$ applied on the skin tissue is measured and the difference from the desired wrench $w_d = [0,0,F_{z,t},0,0,0]^T$ is computed. This wrench difference $w_e$ is then converted to a change in the reference pose using an integral controller. This approach minimises the steady-state force error along the $z_t$ direction so that the contact force converges towards the targeted reference value [20]. Along the other axes, the environmental interaction forces, which consist of contact force and friction force, will cause a certain deviation from the targeted reference pose due to the admittance control [34]. Similar to [20], all 6-DOF of the US tip are controlled in this way. However, an additional functionality was added that stops the torque control along

the $x_t$-axis of the US probe. This is done by setting the $x_t$-axis compliance coefficient to 0, which gives force control in only 5-DOF. The 6th-DOF, the rotation about the $x_t$-axis, is controlled in position control such that the pre-calculated US probe pitch angles about the $x_t$-axis can be accurately realised. The desired robot joint angles are then obtained using the expression graph-based Task Specification Language (eTaSL) [35] and the applied joint torque is computed internally by the robot controller.

The initial scanning path is then 'taught' to this robotic system, in first instance with user-determined orientations, such that the US probe is approximately perpendicular to the skin. The operator 'teaches' this path by co-manipulating the US probe on the robot arm to a set of waypoints on the spine. In the present study, three waypoints were chosen, namely the start ($p_1$), middle ($p_2$), and end ($p_3$) of each vertebra, as the US probe moves medio-laterally along the vertebra. The transverse process closest to the origin of {$b$} served as $p_1$, the spinous process as $p_2$ and roughly the end of the other transverse process as $p_3$. To scan multiple vertebrae, an 'S'-shape scanning trajectory was followed, as shown in Fig. 2. Considering the probe geometry, this is the shortest path that allows the spine to be scanned vertebra-per-vertebra. In this configuration, the probe can be conveniently tilted about the $x_t$-axis to optimise surface coverage with little additional torque and, hence, limited tissue deformation. The user-defined waypoints form a set of M consecutive poses $\sigma = \{\sigma_0, \dots, \sigma_M\}$ along the trajectory. The entire reference path is then determined by concatenating linear interpolations between pairs of consecutive waypoint poses.

### B. US calibration and segmentation for 3D reconstruction

An automatic US spatial calibration was implemented and validated with a custom-designed Z-phantom, as described in [36]. A spatial calibration was performed to determine the homogeneous transformation matrix $^{ee}_{us}T$ and the scaling factor matrices $T_s$ from the US image frame {$us$} to the robot end effector frame {$ee$}.

As a step for 3D US reconstruction, 2D US images were automatically segmented to extract the bone contour. This segmentation pipeline consisted of the following steps. Firstly, the raw US images were converted into greyscale and cropped. Secondly, images were processed by applying a Gaussian filter and morphological operator to remove sparkles. Then, a deep learning-based network, BCDU-Net [37], was employed to segment the US images and extract, at pixel level, the edges of different anatomic regions. Finally, the contours were extracted from the segmented images with a Canny Edge detection algorithm and used in the 3D reconstruction. The resulting 3D point cloud is then automatically filtered using the open source Open3D statistical outlier removal function [38].

The bone contrast in US images of synthetic phantoms and humans differs due to different US attenuation rates of the surrounding tissue. In this work, two models were trained with the same network architecture. One was trained on images from a synthetic spine phantom, and a second was trained on images obtained from in-vivo human volunteer
<figure><img src="image_5.png" /><figcaption>Fig. 3. Landmark points required for automatic landmark extraction of the vertebra in the posterior view (left) and lateral view (right) of a dorsal surface of a CT reconstruction.</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>Fig. 4. An example result of the transverse process (red) and spinous process (blue) extraction of the vertebra on the dorsal surface of a CT reconstruction shown in the posterior (left) and lateral view (right).</figcaption></figure>

lumbar spines. The 2D US images used for training were manually collected on different individuals than the test data and excluded from the experiment. The recorded images were cropped to 800 × 800 pixels and manually labelled. The dataset was augmented by left-right flip and random rotation. The segmentation model for the synthetic phantom was trained using a dataset consisting of 750 images and 2250 after augmentation. The human segmentation model was trained on 900 augmented images. Both models were trained for 20 epochs, with a batch size of 2 and a learning rate of $1e^{-4}$. The images from volunteers that were used for training were excluded from the validation experiments. The mean F1-score, intersection over union (IoU) and balanced accuracy of this segmentation model are, respectively, 0.76, 0.61, and 0.89 for the synthetic phantom model and 0.67, 0.52, 0.91 for the human segmentation model.

### C. Importance of probe orientation

The results in Fig. 1 show the resulting US reconstructions of a lumbar vertebra (L3) from a synthetic phantom where the probe is scanned at a fixed pitch angle of 0°, 5°, 15°, and 25° about its x-axis. At 0° pitch angle, the reconstruction shows limited surface coverage of the spinous process, but clear coverage of the transverse processes (now near-perpendicular to the probe). With increasing probe pitch angle, the surface coverage of the spinous process increases, whereas the coverage of the transverse process reduces. This is because the probe is not perpendicular to the transverse process anymore, but oriented more towards the sides of the spinous process. These qualitative results demonstrate the importance of adjusting US probe pitch angle with respect to the underlying bone surface.

### D. Automatic landmark extraction

To analyse the effect of varying US probe pitch angle, it is important to understand the impact on the quality of

US reconstruction of the key anatomical vertebral features. To evaluate this, a 3D CT reconstructed point cloud of the vertebra (relative to the CT frame {ct}) is registered to the corresponding 3D US reconstruction as explained in Section V. This section hereto describes an automatic landmark extraction method to extract the transverse processes and the spinous process sets of the dorsal surface of the CT point cloud of the vertebra. The automatic method identifies certain landmarks points on the vertebra. As shown in Fig. 3 and Fig. 4, the vertebra frame {v} is defined such that the y-axis is approximately parallel to the spine axis and the x-axis is along the length of the vertebra. The vertebra is then positioned relative to the robot such that {v} is parallel to the corresponding axes of {b}, as shown in Fig. 2. The CT vertebra point cloud is split into three sets: $p_{ct,x}$, $p_{ct,y}$, $p_{ct,z}$, representing the coordinates along x, y, and z direction, respectively. The automatic landmark extraction process is described in the following steps according to the notation in Fig. 3 and the procedure below.

1) Identify the points at the end of the transverse processes by extracting the right-most ($p_R$) and the left-most ($p_L$) point along x. From these points, the length l of the vertebra is calculated

$$l = \| p_R - p_L \| \quad (1)$$

2) Identify the top point of the spinous process, $p_T$, corresponding to max($p_{ct,z}$).

3) Determine the set $S_{T,tip}$, the points on the top surface of the spinous process that are within $d_{tip}$ mm along the z-axis from $p_T$.

4) Identify $p_{T,sup}$, the most superior, and $p_{T,inf}$, the most inferior point along y of $S_{T,tip}$.

5) Determine the points belonging to the spinous process set $S_{sp}$ by selecting the points that are located between $p_{T,sup}$ and $p_{T,inf}$ along the y-axis and are less than $l_{sp}$ away from the average of $p_{T,sup}$ and $p_{T,inf}$, along the x-axis.

6) Determine a subset of the transverse process set $S_{L,tip}$, the set of points at the tip of the left transverse process that are within $l_{tip}$ of $p_L$, along the x-axis.

7) Find the points $p_{L,sup}$ and $p_{L,inf}$ at the medial side of $S_{L,tip}$ along the y-axis to determine the width $w_L$ of the left transverse process

$$w_L = p_{L,sup,x} - p_{L,inf,x} \quad (2)$$

8) Determine the entire left transverse process cluster $S_{tp,L}$ such that the width of the cluster along the y-axis does not exceed $w_{max}$. This is determined through an iterative process: starting from $p_L$, all the points at distance $d_{sect}$ from $p_L$ along x are selected. This distance $d_{sect}$ is initially 1 mm iteratively increased by 1 mm until the width is above $w_{max}$. All the points within the final section now make up $S_{tp,L}$.

9) Repeat step 6-8 for the right transverse process to find $S_{R,tip}$, $p_{R,sup}$, $p_{R,inf}$, $w_R$, and $S_{tp,R}$.

The sets $S_{sp}$, $S_{tp,L}$, and $S_{tp,R}$ of the CT point cloud are then used to determine the surface coverage of the spinous process
<figure><img src="image_7.png" /><figcaption>Fig. 5. The workflow of the robotic US two-scan approach and a detailed flowchart of the path re-planning algorithm. $S_{a,i}$ is the line segment normal to the vertebra outline $S_v$ at a point $p_{v,i}$ on $S_v$. $S_{a,i}$ extends from $p_{v,i}$ to the skin surface $S_s$ and $\alpha_{max}$ is the probe pitch angle limit.</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>Fig. 6. Visualisation of a 3D US reconstruction along with the orange scanning path and constructed projection plane of the first scan of a vertebra (L3) of the synthetic phantom.</figcaption></figure>

and transverse processes in the US reconstruction. An example of the landmark sets derived for the upper surface of the phantom CT is shown in Fig. 4, where $d_{tip}$ is set to 10 mm, $l_{sp}$ to $\frac{1}{16}$ mm, $l_{tip}$ to 7 mm, and $w_{max}$ to $1.5w_L$ or $1.5w_R$. After analysis, these parameters were found to be practical and were chosen for all models. However, a further in-depth study of optimal parameters for different groups of people would be useful for future work.

### III. PATH RE-PLANNING ALGORITHM

The aim of the path re-planning algorithm is to establish better US probe pitch angles about the $x_t$-axis, such that the US probe is always as perpendicular as possible to the underlying spine surface. This should enhance the intensity of the US waves reflected back to the probe and as such improve the 3D US segmentation. Fig. 5 shows how the proposed path re-planning method follows a two-scan approach for each vertebra. In the first scan, the US probe moves in a straight line medio-laterally along the vertebra, maintaining near-perpendicular contact with the skin (Section II-A). This scan is necessary to obtain initial information about the underlying spine geometry. For the second scan, the trajectory is re-planned based on the information from the first scan. The

algorithm calculates optimal pitch angles about the $x_t$-axis along the return trajectory. The desired pitch angle about the $x_t$-axis is then controlled by a position controller as explained in Section II-A.

The algorithm requires the probe tip poses of the first US scan and the corresponding 3D US reconstruction, expressed with respect to $\{b\}$. The full 'S'-shape path, described in Section II-A, is then automatically split into segments corresponding to the medio-lateral probe movement, such that each part contains at least one vertebra as seen in Fig. 6. This is done by intersecting the plane with the underlying 3D US reconstructed spine along the scanning path, as shown in Fig. 6. For each vertebra, a plane is constructed with origin $p_2$ and normal $y_v$. The voxels of the reconstruction within the field of view (FOV) of the US probe are projected onto this plane. Fig. 7(a) shows the projection, which represents the vertebral outline visible by the US probe during the scan. The FOV is defined by the width (along $x_t$) of the US probe. For example, for a probe width of 55 mm, such as the linear probe used for the human trials, all the points within $\pm 27.5$ mm distance along the $y_v$-axis from the plane are projected onto it. These limits are illustrated with two black dotted lines in Fig. 6. The resulting 2D projection $x_v-z_v$ of the single vertebra is also illustrated as in Fig. 7(a).

The uppermost projected voxels along $z_v$ (black points in Fig. 7(a)) are smoothed by means of a moving average and represented as a list of $k$ points

$$S_u = \{p_{u,0}, p_{u,1}, \dots, p_{u,k}\}. \qquad (3)$$

These points are fit and interpolated by a univariate spline function $f$, with order $o$ and is discretised in a set of $j$ points, that is then used to represent the (incomplete) vertebra outline (see the blue line in Fig. 7(a))

$$S_v = f(S_u) = \{p_{v,0}, p_{v,1}, \dots, p_{v,j}\}. \qquad (4)$$

The skin surface, $S_s$, is represented as a series of interpolated US probe tip positions from the corresponding path segment of the first scan. These positions are obtained by exploiting the property that there is constant contact between the US probe
<figure><img src="image_9.png" /><figcaption>Fig. 7. Path re-planning approach. (a) The 2D projection of the reconstruction in Fig. 6 and a spline fit (blue) over the upper voxels, (b) example cases of the line segments normal to the spline and their intersection with the skin surface, and (c) selecting the max and min pitch angle per 10 mm zone. The required probe directions in (c) are moved to the centre of the zone for representation means.</figcaption></figure>

and the skin surface during the first scan (Fig. 7(a) orange line)

$$S_s = \{ \mathbf{p}_{s,0}, \mathbf{p}_{s,1}, \dots, \mathbf{p}_{s,k} \}. \quad (5)$$

From the spine outline $S_v$ and the skin surface $S_s$, the new poses on the skin surface are computed, such that the US probe scans maximally perpendicular to the vertebra surface. The steps applied to each point $\mathbf{p}_{v,i}$ are explained next.

1) Calculate the derivative of $S_v$, i.e. the gradient tangent to the spline and denoted as $S'_v$, at $\mathbf{p}_{v,i}$,

$$m_{t,i} = S'_{v}(x = \mathbf{p}_{v,i}). \quad (6)$$

2) Calculate the direction perpendicular to $m_{t,i}$, i.e. its negative reciprocal, also at $\mathbf{p}_{v,i}$,

$$m_{p,i} = \frac{-1}{m_{t,i}}. \quad (7)$$

3) Create an oriented segment along $m_{p,i}$ discretised in a list of $n$ points

$$S_{a,i} = \{ \mathbf{a}_{i,0}, \mathbf{a}_{i,1}, \dots, \mathbf{a}_{i,n} \}, \quad (8)$$

with start point $\mathbf{a}_{i,0} = \mathbf{p}_{v,i}$, end point $\mathbf{a}_{i,n} = \mathbf{p}_{v,i} + s_c m_{p,i}$, and where $s_c$ is a chosen scaling factor, such that the line segment intersects $S_s$.

4) Find the intersection point $\mathbf{p}_{q,i} = \mathbf{a}_{i,h}$, the h-th point in $S_{a,i}$ closest to $S_s$. Then, $\mathbf{p}_{q,i}$ and $m_{p,i}$ are used to define the pose on the skin such that the probe is aligned with the targeted bone surface.

After the location of the intersection is found, it is stored together with $m_{p,i}$ and the following cases are considered (see some example cases in Fig. 7(b)):

1) To ensure that that US probe scans on the patient's back and within the region of the first scan, the poses calculated for the second scan should be on the skin surface $S_s$ (between $\mathbf{p}_1$ and $\mathbf{p}_3$). This means that for line segments $S_{a,i}$ that do not intersect $S_s$, the gradient $m_{p,i}$ is iteratively either increased or decreased by 5° and the new intersection point is found until $S_{a,i}$ falls within the $S_s$.

2) US cannot see through bone. Therefore, the oriented segments $S_{a,i}$ that intersect the vertebra outline $S_v$ before reaching $S_s$ will not be visible in the US image. These poses are adjusted similarly as described in case 1 above.

3) The maximum allowed pitch angle of the US probe $\pm\alpha_{max}$ is also considered. Too large angles will prevent proper skin-to-probe contact and, hence, reduce the image quality. The pitch angle $\alpha$ is defined as a positive or negative angle around $\mathbf{x}_i$ with respect to $\mathbf{z}_b$-axis of the robot base frame. If the pitch angle derived from the direction $m_{p,i}$ exceeds $\pm\alpha_{max}$, it is automatically set to the closest limit $(+\alpha$ or $-\alpha)$ and the corresponding intersection point with the skin surface is found.

This procedure results in a list of new $\mathbf{p}_{q,i}$ coordinates and the set of improved pitch angles around the $\mathbf{x}_i$-axis, derived from the new gradients $m_{p,i}$. The corresponding coordinate along $\mathbf{y}_v$ are then found by matching the $\mathbf{x}_v$ component of the new $\mathbf{p}_{q,i}$ with the original interpolated robot tip coordinates.

Now that the list of tip positions and pitch angles has been computed, a trajectory is chosen that allows fast and continuous scanning motions and that accommodates the different successive pitch angles. The previous described procedure results in a large number of poses with points that may be located close to each other in space but that have significantly different pitch angles. A few example poses are shown in Fig. 7(c). To simplify the trajectory, $S_s$ is split into $N_w$ number of zones of equal width $w_z$. The minimum and maximum pitch angles are selected within each zone. The new trajectory is then constructed by assigning two poses per zone: a first pose at the start position of the zone with the minimum pitch angle, and a second pose at the end position of the zone with the maximum pitch angle. This means that while translating along the zone, the US probe will rotate between the minimum and maximum pitch angle. At the end of the zone, the US probe will then return to the minimum pitch angle of the next zone.

This results in the final trajectory for one vertebra. The procedure is repeated for each vertebra and all trajectories are combined to create the final improved 'S'-shaped trajectory.

## IV. EVALUATION OF 3D US RECONSTRUCTIONS

In this study, the metrics to evaluate 3D US vertebra reconstructions are split into three categories: metrics that gauge (1) reconstruction quality; (2) scanning path efficiency; and (3) scanning time. The first two metrics are normalised from 0 (worst) to 1 (best) to facilitate the evaluation of different-sized reconstructions. Note that the metrics are applicable for both robotic and handheld US-based reconstructions.
## A. Reconstruction quality (F-score)

The quality of the shape of the US reconstructed point cloud is quantified using two normalised metrics: (1) the precision and (2) the surface coverage of the US point cloud with respect to the CT reconstructed point cloud, as shown in Fig. 8. The top or dorsal surface of the CT point cloud of the vertebra is registered onto the US vertebra point cloud, such that all points are expressed with respect to the robot base frame {$b$}.

(1) The precision $P$, of a US point cloud is defined as the fraction of US points within a distance $d$ from their closest CT point among the total US points [30]

$$P(d) = \frac{\sum_{r \in R} [e_{r \to G} \le d]}{|R|}, \qquad (9)$$

where $e_{r \to G}$ is the distance between the reconstructed US point $r$ and the set of CT points $G$ (ground truth), $R$ is the set of US points (reconstructed) and $d$ is the threshold distance (allowed error). Here, the Iverson bracket $[.] \in \{0, 1\}$ and the cardinality of the set $|\cdot|$ are used [30]. The distance $e_{r \to G}$ is computed as

$$e_{r \to G} = \min_{g \in G} \|\mathbf{r} - \mathbf{g}\|. \qquad (10)$$

(2) The surface coverage $C$ of a US point cloud refers to the fraction of CT points that are within a distance $d$ from the closest US point among the total number of CT points [30]

$$C(d) = \frac{\sum_{g \in G} [e_{g \to R} \le d]}{|G|}. \qquad (11)$$

The distance between a CT point $g$ and all US points $R$ is

$$e_{g \to R} = \min_{r \in R} \|\mathbf{g} - \mathbf{r}\|. \qquad (12)$$

This evaluates the completeness of the 3D US reconstruction. The same threshold distance, namely $d = 2$ mm, is used for both precision and the surface coverage. This is a generally accepted value for expressing the clinical accuracy in spine procedures such as PSP [39].

The total reconstruction quality is then quantified into one single normalised metric, referred to as the F-score [30]. The F-score is defined as the harmonic mean of $C$ and $P$

$$F = 2 \left( \frac{1}{C} + \frac{1}{P} \right)^{-1} = \frac{2PC}{P+C}. \qquad (13)$$

The surface coverage is evaluated for both the entire 3D US reconstruction $C$ and separately for the transverse processes $C_{tp}$ and the spinous process $C_{sp}$, which are important clinical landmarks of the vertebra. The automatic landmark extraction of these regions is described in Section II-D.

## B. Scanning path efficiency ($\eta$-score)

For clinical applications, where time, safety, and the available workspace is limited, it is important to consider the scanning path that is required to achieve a certain reconstruction. A straight scanning path with a fixed probe orientation would be considered to require the smallest workspace and smallest robot work envelope. The scanning efficiency is again split into

<figure><img src="image_10.png" /><figcaption>Fig. 8. Evaluation criteria to determine the reconstruction quality based on (a) precision $P$ and (b) surface coverage $C$. The points marked with a green tick are within distance $d$ and the crossed points are not.</figcaption></figure>

two normalised metrics: (1) the scanning distance efficiency and (2) the probe orientation efficiency.

(1) The scanning distance efficiency $\eta_d$ is a normalised metric evaluating the aggregate distance covered by the scanning probe and is given by

$$\eta_d = 1 - \frac{D_a}{D_{\max}} = 1 - \frac{D_a}{2l}, \qquad (14)$$

where $D_{\max}$ is the maximum aggregate distance that is required to visualise the full vertebra. This value is set to be double the length $l$ of the vertebra along the $x_b$-axis, because that is the length of the longest path in this study. $D_a$ is the actual aggregate distance covered by the US probe. It considers only the components along $x_b$ and $y_b$, and is defined as

$$D_a = \sum_{i=0}^{N-1} (\| (\mathbf{p}_{f,i+1} - \mathbf{p}_{f,i}) \times \mathbf{z}_b \|). \qquad (15)$$

The US tip $\mathbf{p}_t$ coordinates are stored in a list of $N$ points $\mathbf{p} = \mathbf{p}_0, \dots, \mathbf{p}_N$. The exact movement along the $z_b$-axis is not pre-determined, but instead defined by the robot force control and primarily influenced by the subject breathing motion and the compliance of the robot. It, therefore, does not have a direct influence on the scanning path efficiency.

(2) The probe orientation efficiency $\eta_\theta$ is a normalised metric evaluating the aggregate angle rotated by the US probe

$$\eta_\theta = 1 - \frac{\theta_a}{\theta_{\max}}. \qquad (16)$$

In this work, the maximum total angle is set to be

$$\theta_{\max} = \frac{\pi}{3} \text{rad} \frac{l}{w_z}. \qquad (17)$$

This means that the US probe can rotate maximum $\frac{\pi}{3}$ rad along its $x_t$-axis for every $w_z$ along the length $l$ of the vertebra. The actual total angle $\theta_a$ of each US probe pitch $\alpha_i$ (about the $x_t$-axis) is

$$\theta_a = \sum_{i=0}^{N_w-1} (|\alpha_{\max,i} - \alpha_{\min,i}|). \qquad (18)$$

For applications in which the probe is rotated about its $y_t$- and $z_t$-axes, this definition can be extended.

The harmonic mean of both the distance and orientation metrics is represented by what is referred to as the $\eta$-score

$$\eta = \frac{2\eta_d\eta_\theta}{\eta_d + \eta_\theta}. \qquad (19)$$
<figure><img src="image_11.png" /><figcaption>Fig. 9. The labelled experimental setup used for the clinical trial. A similar setup was used for the phantom and cadaver trials. A zoom-in of the US probe and gel pad, only used for the clinical trial, is also shown.</figcaption></figure>

### C. US scanning time

The scanning time is especially important for clinical applications, where often a shorter scanning time is essential. Especially for intra-operative applications, minimising the scanning time, and hence, the operating time would maximise surgical capacity and productivity, and reduce healthcare costs. Costs related to medicine, e.g. anaesthesia, clinicians, the occupation of the operating room would all be reduced with a shorter scanning time. Shorter operating times could also lead to improved patient outcomes [40].

## V. EXPERIMENTS AND RESULTS

### A. Experimental setup

Phantom and cadavers experiments, and a clinical trial on healthy human volunteers were conducted for this study. Fig. 9 shows the general experimental setup. This setup consists of a B-mode US imaging system mounted on a lightweight robotic arm (KUKA Robot MED7, Augsburg, Germany). A custom-designed US probe housing securely connects the US probe to the robot wrist and ensures correct alignment by means of a metal 'L'-shaped connection part. Note that the optical markers on the US probe are not used in this experiment. A 6-DOF force torque (F/T) sensor (Nano25, ATI Industrial Automation Inc.) is assembled between this metal connection and the top of the US probe housing to measure the interaction forces and torques at the probe tip when in contact with the skin surface. The end effector is assembled onto a fast tool changer (G-SHW063-2UE, GRIP GmbH, Germany), connected to the wrist of the robot. A frame grabber (Epiphan Systems Inc. Palo Alto, California, USA) captures the US images via a USB port at 15 Hz. A PC workstation (Intel i7 2.6 GHz CPU, 64G RAM) is used for data acquisition and processing. This PC runs in Ubuntu 20.04 with Robot Operating System version 2 (ROS 2). For real-time robot control, the Open Robot Control Software (Orocos, version 2.9.0) is employed at 200 Hz. The 3D visualisation tools used for the segmentation and path re-planning algorithms, detailed in Section II-B and III, are developed and integrated with OpenCV (Intel Corporation, USA) and Visualisation Toolkit (vtk.org). The graphical user interface for human-computer interaction is developed with

<figure><img src="image_12.png" /><figcaption>Fig. 10. Five scanning paths and corresponding waypoints across the vertebra with different US probe pitch angles, $\alpha$. All paths were performed on the phantom. Only path 1 and 5 were performed in the cadaver and clinical trials.</figcaption></figure>

Qt (https://wwwovi.io). A NVIDIA RTX A4500 and NVIDIA Compute Unified Device Architecture (CUDA) are used to accelerate the processing and computation of the segmentation algorithm, requiring less than one minute to reconstruct a single vertebra. The robot and the force sensor communicate to the workstation by Ethernet.

During post-processing, US segmentation and reconstruction was performed as described in Section II-B. For the CT reconstruction, the vertebra was segmented as a rigid body and converted into 3D triangular surfaces (STL) using a medical-certified and commercially available segmentation software (Materialise Mimics Innovation Suite software, version 19.0, Materialise NV, Leuven, Belgium). The dorsal surface of the CT was manually extracted using blender v3.6 (blender.org), a free open source software. The dorsal CT surface was registered onto the US reconstruction using coarse point-to-point registration and fine-tuned with ICP registration [41]. For the path re-planning algorithm, a zone width $w_z$ of 10 mm was selected, which is equal to the width of the US probe. The angle limit $\alpha_{max}$ was set to 20° and the top 2D projected points were fit with a 3rd order spline to find $S_v$. The 3D US reconstructions were validated according to the evaluation criteria determined in Section IV.

The phantom, cadaver, and human trial experiments all used this setup. However, the main distinctions are as follows:

1) Synthetic phantom: due to logistics, the main distinction between the setups is the US machine, but no significant difference in the US images of the probes were observed. For the phantom experiments, a ML6-15 (15MHz) linear probe and GE Logiq US system (GE Healthcare, Illinois, USA) was used. This US system displays 1080 x 1920 pixel images with a depth of 80 mm. The synthetic US-compatible phantom is a commercial lumbar phantom (Model 034, CIRS, USA), which contains an anatomical model of the lumbar spine and additional soft tissue to mimic the human anatomy.

2) Cadaver and clinical trial: in these experiments, the Aixplorer Ultimate US machine with a SuperLinear SL10-2 (10MHz) probe (Hologic SuperSonic Imagine, USA) was used. This machine displays 1080 x 1920 pixel images with a depth of 77 mm. The healthy human volunteers, aged 18-35 years old, had a BMI between 19 and 25 kg/m$^2$. An important distinction for the human volunteer experiments is
<table><caption>TABLE I
EVALUATION OF 3D US RECONSTRUCTIONS OF DIFFERENT US SCANNING PATHS ACROSS L3 OF THE SYNTHETIC PHANTOM.</caption><thead><tr><th>Path</th><th>P</th><th>C</th><th>F</th><th>C<sub>tp</sub></th><th>C<sub>sp</sub></th><th>η<sub>d</sub><sup>1</sup></th><th>η<sub>θ</sub><sup>1</sup></th><th>η<sup>1</sup></th><th>t (s)<sup>1</sup></th></tr></thead><tbody><tr><td>1-Perp.</td><td><b>0.81 ± 0.03</b></td><td>0.56 ± 0.01</td><td>0.66 ± 0.01</td><td><b>1.00 ± 0.00</b></td><td>0.33 ± 0.01</td><td><b>0.50</b></td><td><b>1.00</b></td><td><b>0.67</b></td><td><b>58.50</b></td></tr><tr><td>2-Bell curve</td><td>0.75 ± 0.03</td><td>0.65 ± 0.01</td><td>0.69 ± 0.01</td><td>0.99 ± 0.00</td><td>0.50 ± 0.01</td><td>0.37</td><td>0.90</td><td>0.52</td><td>91.14</td></tr><tr><td>3-LR switch</td><td>0.67 ± 0.02</td><td>0.53 ± 0.02</td><td>0.59 ± 0.01</td><td>0.53 ± 0.11</td><td><b>0.58 ± 0.01</b></td><td>0.37</td><td>0.94</td><td>0.53</td><td>119.90</td></tr><tr><td>4-LR</td><td>0.74 ± 0.02</td><td>0.68 ± 0.02</td><td>0.71 ± 0.01</td><td>0.97 ± 0.01</td><td><b>0.58 ± 0.01</b></td><td>0.00</td><td>0.95</td><td>0.00</td><td>166.35</td></tr><tr><td>5-Re-scan</td><td>0.69 ± 0.02</td><td><b>0.73 ± 0.02</b></td><td><b>0.71 ± 0.00</b></td><td><b>1.00 ± 0.00</b></td><td>0.56 ± 0.02</td><td>0.38</td><td>0.19</td><td>0.26</td><td>505.62</td></tr></tbody></table>

<table><caption>TABLE II
EVALUATION OF 3D US RECONSTRUCTIONS FOR PERPENDICULAR AND RE-SCAN US SCAN ON 5 CADAVER VERTEBRAE</caption><thead><tr><th>Path</th><th>P</th><th>C</th><th>F</th><th>C<sub>tp</sub></th><th>C<sub>sp</sub></th><th>η<sub>d</sub></th><th>η<sub>θ</sub></th><th>η</th><th>t (s)</th></tr></thead><tbody><tr><td>1-Perp.</td><td><b>0.93 ± 0.05</b></td><td>0.26 ± 0.04</td><td>0.40 ± 0.06</td><td>0.34 ± 0.15</td><td>0.14 ± 0.05</td><td><b>0.50 ± 0.00</b></td><td><b>1.00 ± 0.00</b></td><td><b>0.67 ± 0.00</b></td><td><b>79.76 ± 4.30</b></td></tr><tr><td>5-Re-scan</td><td>0.87 ± 0.09</td><td><b>0.37 ± 0.04</b></td><td><b>0.52 ± 0.05</b></td><td><b>0.34 ± 0.14</b></td><td><b>0.26 ± 0.05</b></td><td>0.31 ± 0.18</td><td>0.43 ± 0.11</td><td>0.33 ± 0.19</td><td>410.99 ± 71.62</td></tr></tbody></table>

that a US gel pad (aiSon™ Technologies, Zurich, Switzerland, see Fig. 9) was attached to the US probe to ensure subject comfort and maintain skin-to-probe contact while changing probe orientations. Ethical approval was obtained for both the cadaver trial (Belgian National Council for Bioethics, No.NH019 2022-1-02) and human trial (Cantonal Ethical Committee of the Canton of Zurich, Switzerland, BASEC Nr.2023-00350, 2023-4-18). The latter was registered as a clinical trial (“Robot-Assisted US-Based Vertebral Segmentation for Pedicle Screw Trajectory Identification”, NCT05904418).

## B. Validation on synthetic phantom

To validate the effect of changing the probe pitch $\alpha$ on the synthetic phantom, a series of five different robotic US scanning paths were performed across one vertebra. The different paths are illustrated in Fig. 10. Note that the three waypoints were manually identified as described in Section II-A.

Path 1 (perp.) is simply a linear interpolation of the position and orientation of the chosen three points. The path is executed using force control, such that the probe is always near-perpendicular to the skin surface. A second group of paths are constructed from pre-planned pitch angles $\alpha$, based on the three waypoints. A maximum probe pitch $\alpha_{max}$ of 20° was chosen, ensuring both proper probe contact and safety.

* Path 2 (bell curve): 0° at the start point, $\alpha_{max}$ then $-\alpha_{max}$ at the middle point and 0° at the end point.

* Path 3 (LR switch): $\alpha_{max}$ at the start and maintained until the middle point, where it rotates to $-\alpha_{max}$ and translates to the end point.

* Path 4: (LR) $\alpha_{max}$ at the start and maintained until the end position, where it rotates to $-\alpha_{max}$ and performs a return scan with this fixed pitch until the start position.

Path 5 (re-scan) is generated following the workflow presented in Section III, based on the US reconstruction from Path 1. For all scanning paths on the phantom, the target force $F_{t,z}$ was set to 5 N [21]. Each scanning path was performed three times across the third lumbar vertebra (L3) of the synthetic phantom at 3 mm/s. Between each waypoint, the robot pauses for a buffer time of 1 s. The mean and standard deviations for each scanning path and criteria are shown in Table I. The mean total surface covered was found to be 0.56 ± 0.01 and 0.73 ± 0.02

for the perpendicular scan and re-scan, respectively. The mean $\eta$-score and time was 0.67 and 58.5 s; and 0.26 and 505.6 s for the perpendicular scan and re-scan, respectively.

## C. Validation on cadavers and healthy human volunteers

For both the cadaver and human volunteer experiments, only scanning Path 1 (perp.) and 5 (re-scan) were performed. Multiple lumbar vertebrae were scanned along an ‘S’-shape, as shown in Fig. 2. Three waypoints were identified per vertebra, as described previously. The target force $F_{t,z}$ was set to 8 N for the cadavers to ensure proper skin-to-probe contact. A lower force of 5 N was set for the human volunteers to ensure additional safety, similar to [21]. For the cadaver trial, the robot scanning speed was set to 3 mm/s. The scanning speed was increased to 4 mm/s for the human trial to reduce the scanning time.

Two cadavers were placed in the prone position and strapped to a surgical bed. The same position was kept for both the CT scan and the robotic US scan. In total, five vertebrae were scanned with the robotic US system. The evaluation results are presented in Table II. The mean total surface covered was found to be 0.26 ± 0.04 and 0.37 ± 0.04 for the perpendicular scan and re-scan, respectively. More specifically, the surface coverage of the spinous process was 0.14 ± 0.05 and 0.26 ± 0.05, respectively. The force $F_{t,z}$ along the z-axis of the probe tip frame was maintained around 8 N for both scans. The mean force was -7.72 ± 0.86 N (-3.5% deviation) and -7.69 ± 0.38 N (-3.8% deviation), respectively.

Similarly, the human volunteers were also asked to lie in a prone position for both the CT scan and the robotic US scan, as shown in Fig. 9. Five vertebrae were scanned on different volunteers with the robotic US system. To verify the path re-planning algorithm, only the vertebrae that included at least one point of the spinous process in the first perpendicular scan were selected. This ensures that there is sufficient information in the vertebra outline to generate useful poses for the re-scan. The volunteers were asked to stay still and breathe regularly during the US scans. The evaluation results are presented in Table III. The mean total surface covered was found to be 0.32 ± 0.09 and 0.56 ± 0.06 for the perpendicular scan and re-scan, respectively. The surface coverage of the spinous

<footnote n="1">¹The standard deviation was near zero for the scanning path metrics.</footnote>
<table><caption>TABLE III
EVALUATION OF 3D US RECONSTRUCTIONS FOR PERPENDICULAR AND RE-SCAN US SCAN ON 5 HEALTHY HUMAN VOLUNTEER VERTEBRAE</caption><thead><tr><th>Path</th><th>P</th><th>C</th><th>F</th><th>C<sub>tp</sub></th><th>C<sub>sp</sub></th><th>η<sub>d</sub></th><th>η<sub>θ</sub></th><th>η</th><th>t (s)</th></tr></thead><tbody><tr><td>1-Perp.</td><td><b>0.82 ± 0.08</b></td><td>0.32 ± 0.09</td><td>0.46 ± 0.11</td><td>0.35 ± 0.19</td><td>0.15 ± 0.07</td><td><b>0.54 ± 0.06</b></td><td><b>0.99 ± 0.01</b></td><td><b>0.70 ± 0.05</b></td><td><b>35.16 ± 1.54</b></td></tr><tr><td>5-Re-scan</td><td>0.74 ± 0.05</td><td><b>0.56 ± 0.06</b></td><td><b>0.63 ± 0.05</b></td><td><b>0.48 ± 0.11</b></td><td><b>0.38 ± 0.09</b></td><td>0.46 ± 0.08</td><td>0.57 ± 0.09</td><td>0.50 ± 0.06</td><td>177.10 ± 34.56</td></tr></tbody></table>

<figure><img src="image_13.png" /><figcaption>Fig. 11. Surface meshes of 3D US (orange) and CT (black) reconstructions of five human volunteer vertebrae generated from the first perpendicular scan and the second re-scan with the improved orientations. The re-scan reconstructions do not include the images from the perpendicular scan.</figcaption></figure>

process was 0.15 ± 0.07 and 0.38 ± 0.09, respectively. The triangulated surface meshes of the 3D reconstructions for each human vertebra and respective scans are shown in Fig. 11. Also for the human trial, $F_{t,z}$ was maintained to be around 5 N for both scans. For the perpendicular scan and re-scan, the mean force was -4.65 ± 0.43 N (-6.9% deviation) and -4.98 ± 0.51 N (-0.5% deviation), respectively. No distinct difference in force control between the two scans was observed despite the fact that the motion pattern in the re-scan scenario is more complex.

# VI. DISCUSSION

## A. The need for path-planning algorithms

For the initial phantom experiments, in which multiple pre-planned paths (bell curve, LR switch, and LR) and the re-scan were generated, the results demonstrate that different probe pitch angles indeed affect the reconstruction quality, the scanning efficiency, and the scanning time. The re-scan path achieves the highest overall coverage of 73%. The LR switch and LR path show the highest surface coverage of the spinous process as the probe is at its maximum pitch angle throughout the scan and hence most perpendicular to the spinous process. However, this also causes the LR switch to have the lowest surface coverage of the transverse process. The results from these initial paths further emphasise the importance of using the limited information about the underlying bone surface to optimise the probe orientation to improve surface coverage. There is no clear linear relationship between the F-score, η-score, and the time. For example, compared to the bell curve, the LR switch takes 28.76 s longer but the F-score is 0.10 lower and the scanning path efficiency barely differs. This shows the potential to optimise the scanning path efficiency

without reducing the reconstruction quality. Although the pre-planned paths look promising and less time consuming than the re-scan, on humans with vertebral deformities, unknown vertebral position and size, etc., more information about the underlying anatomy is required to plan an optimal path. These initial results motivate the further research into path-planning algorithms to improve 3D US reconstructions.

## B. Reconstruction quality

The cadaver experiments (Table. II) and human volunteer experiments (Table. III) focused on the first perpendicular path and the scanning path generated by the path re-planning algorithm (re-scan). In terms of surface coverage, the re-scan offered an average percentage increase of 30.4%, 42.3%, and 75.0% on the synthetic phantom, cadaver, and human volunteers, respectively. More specifically, the surface coverage of the spinous process increased by 69.7%, 85.7%, and 153.3%, respectively. The surface coverage of the transverse process remained the same for the phantom and the cadaver experiments, but increased by 37.1% on human volunteers. The larger increase for the human volunteers could be partially attributed to the better US image quality compared to thawed cadaver and to the use of the gel pad on the US probe. The gel pad could improve contact with the skin and, hence, improve the image quality for large probe pitch angles. However, the gel pad could also introduce image artefacts (e.g. reverberation artefacts). The effect of the gel pad on the image quality at different probe angles could be investigated in future research.

The human volunteer reconstructions now reach an overall mean surface coverage of 56%, with 48% coverage of the transverse process and 38% coverage of the spinous process. It is especially important that the chosen landmarks show
<figure><img src="image_14.png" /><figcaption>Fig. 12. An example of a 3D US reconstruction with little information from the perp. scan (left) and the corresponding re-scan (right).</figcaption></figure>

improved surface coverage. This will allow, for example, for more accurate intraoperative planning in procedures such as PSP [4], [5]. For these experiments, the maximum pitch angle was set to 20° to ensure constant probe-to-skin contact. However, with the gel pad in the human experiments, this angle could be increased, which could enhance the surface coverage of the spinous process even more. The results in Table III show a selection of reconstructions obtained during the human trial that had a visually high surface coverage and, hence, demonstrate the potential of using US an a complementary imaging technique. Fig. 12 provides an example of a first perpendicular scan with very low surface coverage. This shows that even with little information, the path re-planning algorithm works and is able to improve surface coverage, in this case from 7.4% to 20.1%. These values were calculated, using Eq. 11, with a cut-off distance *d* of 2 mm. However, we verified the impact of different *d*-values on the results, and a similar trend in the surface coverage *C* was observed.

For the phantom, cadaver trial, and human trial, the re-scan precision is, respectively, 14.8%, 6.5%, and 10.0% lower than the first scan. The precision could be affected by small errors in the US image-to-end effector calibration and the probe tip-to-end effector calibration, which would be amplified at larger probe pitch angles, such as in the re-scan. For the human trial, the precision of both scans is lower than the precision for the cadaver trial. This could be due to the breathing motion of the human volunteers, which causes small movement of the vertebrae during US scanning. Future work will look into creating a breathing compensation algorithm to reduce the effect on the 3D US reconstruction.

### C. Scanning efficiency

The re-scan scanning path efficiency scores lower across all criteria and models compared to the perpendicular path. This is expected as the perpendicular scan follows the simplest path with almost no change in orientation. The re-scan has more changes in probe pitch angle and an increased number of waypoints (with 1 s buffer time between each point). This especially affects the scanning time *t*, which for the re-scan is on average 3 minutes for one human volunteer vertebra and 5 times longer than the perpendicular scan. For multiple vertebrae, the scanning would take on average 9 minutes, in which the volunteer must remain still and breath steady. Even though this remained comfortable and often felt “like a massage” for the volunteers, this is still too long for a clinical

application where OR time is costly. In this initial version of the path re-planning algorithm, time optimisation was not one of the main focus points. As the probe now moves from its minimum to maximum pitch angle between each zone, some interpolated poses in between were redundant. Furthermore, for these experiments, the robot moved at a very slow speed, not because of US limitations, but rather to ensure safety of the human volunteer. For future work, the trajectory could be optimised further to reduce redundancy and the robot could be allowed to move faster while doing safe US scans.

### D. Evaluation metrics

The chosen normalised metrics allow for a detailed comparison of the different scanning paths and 3D US reconstructions across different models. Even with different robot scanning speeds between the cadaver and human trial, the $\eta$-score allowed for a fair comparison of the scanning paths. Works, such as [20], [23], [24], could benefit from the proposed evaluation metrics to demonstrate the effectiveness of their path planning algorithms. These evaluation metrics could also be used for US scanning and planning for reconstructions of other anatomic structures. The evaluation methodology that was designed to compare the reconstructions is almost fully automatic, except for manually obtaining the dorsal surface of the CT reconstruction and manual initial point-to-point registration of the CT onto the US reconstruction. The F-score, especially *P*, is dependent on the registration accuracy. Therefore, in future work, these steps could be automated to make a completely automatic and consistent evaluation workflow. The reconstruction metrics, *P* and *C*, also both require a chosen distance threshold *d*. This value must be chosen such that it is relevant for the application.

### E. Current challenges

The results of the re-scan demonstrate the need and feasibility of the path re-planning algorithm. The algorithm was robust and able to compute the improved poses, even when the vertebral levels were not aligned or if a scan included very limited information. However, a limitation of the path re-planning algorithm is that it relies on a first scan to provide information about the underlying bone. This lengthens the workflow (2.8 minutes extra per vertebra) and it requires processing time for the reconstruction as well (less than 1 minute per vertebra). Furthermore, if the reconstruction of the first perpendicular scan is insufficient such that no spline can be fit through the 2D projection, then the algorithm cannot generate a proper path for the re-scan. A solution could be to use a generic vertebra outline as input to the algorithm. This could potentially be scaled and registered for each vertebra based on the recognition of some distinct landmarks.

## VII. CONCLUSION

The path re-planning algorithm presented in this paper shows promising improvements of 42.3% and 75.0% in the total surface coverage of 3D US reconstructions of cadaver and healthy human vertebrae, respectively. The paper highlights
the importance of changing probe orientations with respect to the underlying bone surface and provides a new direction for path planning for robotic US scanning. Future work could look at optimising the scanning time and eliminating the need for a first scan by means of an online path re-planning algorithm or a generic vertebra outline, from a generated statistical shape model based on various CT scans. The chosen evaluation criteria including the F-score, an η-score for path efficiency, and the scanning time provide a ground for future comparison of scanning paths for US 3D reconstruction of both the spine and possibly other anatomy. With the current improved surface coverage, the 3D US reconstructions could provide an improved registration for PSP path planning [4], [5]. However, further focus on improving the raw US images, such as by optimising the scanning imaging plane, will facilitate further developments in this field and gradually improve the potential for the adoption of US in clinical orthopaedic applications.

## ACKNOWLEDGEMENTS

The authors would like to thank Orçun Taylan, Ann Van Ermen, and Kristof Reyniers at the UZ Leuven Anatomy and Surgical Technologies Skills Centre for their assistance in organising the cadaver study. The authors would also like to thank Walter Coudyzer at UZ Leuven and Laura Arango at UZH for taking and segmenting the human CT scans.

## REFERENCES

[1] H. E. Gueziri, C. Santaguida, and D. L. Collins, "The state-of-the-art in ultrasound-guided spine interventions," *Med. Image Anal.*, vol. 65, p. 101769, Oct. 2020.

[2] Y. P. Zheng and T. T. Y. Lee, "3d ultrasound imaging of the spine," *Advances Exp. Medicine Biol.*, vol. 1364, pp. 349–372, May 2022.

[3] T. Ungi et al., "Automatic spine ultrasound segmentation for scoliosis visualization and measurement," *IEEE Trans. Biomed. Eng.*, vol. 67, pp. 3234 – 3241, Nov. 2020.

[4] R. Li et al., "Ultrasound-based robot-assisted drilling for pedicle screw placement," *IEEE Trans. Med. Robot. Bionics*, pp. 818–828, Apr. 2024.

[5] A. Chan et al., "3d ultrasound navigation system for screw insertion in posterior spine surgery: a phantom study," *Int. J. Comput. Assist. Radiol. Surg.*, vol. 17, pp. 271–281, Feb. 2022.

[6] V. K. Vishnu et al., "Diagnostic utility of point of care ultrasound in identifying cervical spine injury in emergency settings," *Australas. J. Ultrasound Med.*, vol. 24, pp. 208–16, Nov. 2021.

[7] S. Bianchi, "Ultrasound and bone: a pictorial review," *J. Ultrasound*, vol. 23, pp. 227–257, Sept. 2020.

[8] B. Ihnatsenka and A. P. Boezaart, "Ultrasound: Basic understanding and learning the language," *Int. J. Shoulder Surg.*, vol. 4, p. 55, Jul. 2010.

[9] D. J. Blehar, B. Barton, and R. J. Gaspari, "Learning curves in emergency ultrasound education," *Academic Emergency Medicine*, vol. 22, pp. 574–582, May 2015.

[10] I. Hacihaliloglu, "Ultrasound imaging and segmentation of bone surfaces: A review," *Technol.*, vol. 5, pp. 74–80, Jun. 2017.

[11] D. Ottacher et al., "Positional and orientational accuracy of 3-d ultrasound navigation system on vertebral phantom study," *IEEE Trans. on Instrum. and Meas.*, vol. 69, pp. 6412–6419, Sept. 2020.

[12] H. Chen et al., "Compact and wireless freehand 3d ultrasound real-time spine imaging system: A pilot study," *Int. Conf. IEEE Eng. Medicine and Biol. Soc.*, pp. 2105–2108, Jul. 2020.

[13] G. Harrison and A. Harris, "Work-related musculoskeletal disorders in ultrasound: Can you reduce risk?" *Ultrasound: J. Brit. Med. Ultrasound Soc.*, vol. 23, pp. 224–230, Nov. 2015.

[14] C. Scholl and H. Salisbury, "Barriers to performing ergonomic scanning techniques for sonographers," *J. Diagn. Med. Sonogr.*, vol. 33, pp. 406–411, Sept. 2017.

[15] R. Dickson, K. Duncanson, and S. Shepherd, "The path to ultrasound proficiency: A systematic review of ultrasound education and training programmes for junior medical practitioners," *Australas. J. Ultrasound Med.*, vol. 20, pp. 5–17, Feb. 2017.

[16] K. Li, Y. Xu, and M. Q. Meng, "An overview of systems and techniques for autonomous robotic ultrasound acquisitions," *IEEE Trans. Med. Robot. Bionics*, vol. 3, pp. 510–524, May 2021.

[17] M. Victorova et al., "Comparison of ultrasound scanning for scoliosis assessment: Robotic versus manual," *Int. J. Med. Robot. Comput. Assist. Surg.*, vol. 19, p. 2468, Apr. 2023.

[18] J. Zhang et al., "Self-adaptive ultrasound scanning system for imaging human spine," *IEEE Trans. Ind. Electron.*, vol. 69, pp. 570–581, Jan. 2022.

[19] F. Suligoj et al., "Robust–an autonomous robotic ultrasound system for medical imaging," *IEEE Access*, vol. 9, pp. 67456–67465, May 2021.

[20] R. Li et al., "Robot-assisted ultrasound reconstruction for spine surgery: from bench-top to pre-clinical study," *Int J Comput Assist Radiol Surg*, vol. 18, pp. 1613–1623, Sept. 2023.

[21] M. Tirindelli et al., "Force-ultrasound fusion: Bringing spine robotic-us to the next 'level'," *IEEE Robot. Automat. Lett.*, vol. 5, pp. 5661–5668, Oct. 2020.

[22] C. H. Oh et al., "Technical report of free hand pedicle screw placement using the entry points with junction of proximal edge of transverse process and lamina in lumbar spine: Analysis of 2601 consecutive screws," *Korean J. Spine*, vol. 10, pp. 7–13, Mar. 2013.

[23] K. Li et al., "Image-guided navigation of a robotic ultrasound probe for autonomous spinal sonography using a shadow-aware dual-agent framework," *IEEE Trans. Med. Robot. Bionics.*, vol. 4, pp. 130–144, Feb. 2022.

[24] Q. Huang et al., "On mimicking human's manipulation for robot-assisted spine ultrasound imaging," *Robot. Intell. Automat.*, pp. 373–381, Aug. 2023.

[25] X. Ma et al., "A-see: Active-sensing end-effector enabled probe self-normal-positioning for robotic ultrasound imaging applications," *IEEE Robot. Automat. Lett.*, vol. 7, pp. 12475–12482, Oct. 2022.

[26] C. Yang et al., "Automatic 3-d imaging and measurement of human spines with a robotic ultrasound system," *IEEE Trans. Instrum. Meas.*, vol. 70, p. 7502713, May 2021.

[27] D. V. Nguyen et al., "Validation of 3d surface reconstruction of vertebrae and spinal column using 3d ultrasound data – a pilot study," *Med. Eng. Phys.*, vol. 37, pp. 239–244, Feb. 2015.

[28] C. Gebhardt et al., "Femur reconstruction in 3d ultrasound for orthopedic surgery planning," *Int. J. Comput. Assist. Radiol. Surg.*, p. 1001–1008, Apr. 2023.

[29] W. Kerr, P. Rowe, and S. Pierce, "Accurate 3d reconstruction of bony surfaces using ultrasonic synthetic aperture techniques for robotic knee arthroplasty," *Comput. Med. Imag. Graph.*, vol. 58, pp. 23–32, Jun. 2017.

[30] M. Maboudi et al., "A review on viewpoints and path planning for uav-based 3-d reconstruction," *IEEE J. Selected Topics Appl. Earth Observ. and Remote Sens.*, vol. 16, pp. 5026–5048, May 2023.

[31] H. E. Gueziri et al., "Evaluation of an ultrasound-based navigation system for spine neurosurgery: A porcine cadaver study," *Front. Oncol.*, vol. 11, p. 619204, Mar. 2021.

[32] W. R. Scott, G. Roth, and J.-F. O. Rivest, "View planning for automated three-dimensional object reconstruction and inspection," *ACM Comput. Surv.*, vol. 35, pp. 64–96, Mar. 2003.

[33] R. Border, J. D. Gammell, and P. Newman, "Surface edge explorer (see): Planning next best views directly from 3d observations," in *IEEE Int. Conf. Robot. Automat. (ICRA)*. IEEE Int. Conf. Robot. Automat., 2018, pp. 6116–6123.

[34] G. Ning, J. Chen, X. Zhang, and H. Liao, "Force-guided autonomous robotic ultrasound scanning control method for soft uncertain environment," *Int. J. Comp. Assist. Rad. Surg.*, vol. 16, pp. 2189–99, Dec. 2021.

[35] E. Aertbeliën and J. De Schutter, "etasl/etc: A constraint-based task specification language and robot controller using expression graphs," in *IEEE/RSJ Int. Conf. Intell. Robots Syst.*, 2014, pp. 1540–1546.

[36] R. Li, K. Niu, and E. Vander Poorten, "A framework for fast automatic robot ultrasound calibration," in *Int. Symp. Med. Robot. (ISMR)*, 2021.

[37] R. Azad et al., "Bi-directional convlstm u-net with densley connected convolutions," *IEEE Int. Conf. Comput. Vision Workshop (ICCVW)*, pp. 406–415, Oct. 2019.

[38] Q.-Y. Zhou, J. Park, and V. Koltun, "Open3d: A modern library for 3d data processing," *arXiv:1801.09847*, Jan. 2018.

[39] A. A. Aoude et al., "Methods to determine pedicle screw placement accuracy in spine surgery: a systematic review," *Eur. Spine J.*, vol. 24, pp. 990–1004, May 2015.

[40] K. Phan et al., "Anesthesia duration as an independent risk factor for early postoperative complications in adults undergoing elective acdf," *Global Spine J.*, vol. 7, pp. 727–734, Dec. 2017.

[41] P. Besl and N. McKay, "A method for registration of 3-d shapes," *IEEE Trans. Pattern Anal. Mach. Intell.*, vol. 14, no. 2, pp. 239–56, Feb. 1992.