<figure><img src="image_1.png" /></figure>

HAL
open science

# Haptic teleoperation of flexible needles combining 3D ultrasound guidance and needle tip force feedback

Marco Aggravi, Daniel a L Estima, Alexandre Krupa, Sarthak Misra, Claudio Pacchierotti

## ► To cite this version:

Marco Aggravi, Daniel a L Estima, Alexandre Krupa, Sarthak Misra, Claudio Pacchierotti. Haptic teleoperation of flexible needles combining 3D ultrasound guidance and needle tip force feedback. IEEE Robotics and Automation Letters, 2021, 6 (3), pp.4859-4866. 10.1109/LRA.2021.3068635. hal-03164691

HAL Id: hal-03164691

https://inria.hal.science/hal-03164691

Submitted on 10 Mar 2021

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.
# Haptic Teleoperation of Flexible Needles Combining 3D Ultrasound Guidance and Needle Tip Force Feedback

Marco Aggravi¹,*, Daniel A. L. Estima²,*, Alexandre Krupa³,arthak Misra²,⁴, and Claudio Pacchierotti¹

*Abstract*—We present a haptic teleoperation system capable of steering flexible needles under ultrasound imaging toward a target. With respect to similar works, this approach enables intuitive control of the needle motion while providing the user with 3D navigation and needle tip cutting force using a combination of kinesthetic and vibrotactile haptic feedback. The needle is tracked during the insertion using a 3D ultrasound probe. A friction estimation algorithm extracts salient information about the cutting force at the needle tip from a force sensor placed at the needle base. A grounded haptic interface enables natural 6-DoF control of the needle motion while providing kinesthetic feedback, and a wearable cutaneous interface on the forearm provides distributed vibrotactile sensations. We carried out a human subject study to validate the insertion system in a gelatine phantom and compare seven different feedback techniques. The best performance was registered when providing navigation cues through kinesthetic feedback and needle tip cutting force through cutaneous vibrotactile feedback. In this modality, results showed an 87% accuracy improvement with respect to providing no haptic feedback at all.

**Index Terms**—Haptics and Haptic Interfaces; Surgical Robotics: Steerable Catheters/Needles.

## I. INTRODUCTION

MOST robotic systems for flexible needle insertion still lack effective feedback capabilities [1]–[3]. In this respect, real-time visualization of the needle pose has been proven effective in increasing the clinician’s performance in many of these scenarios [4], using, e.g., magnetic resonance imaging (MRI) [5], computed tomography (CT) [6], or ultrasound (US) [7]. Most teleoperated robotic systems for needle insertion use one of these imaging modalities to provide visual feedback on the needle during the insertion. For example, Chevrie et al. implemented semi-automatic teleoperation of a flexible needle based on US tracking [8], [9]. It uses an iterative algorithm to fit a polynomial curve to the needle shape, while a 3-dimensional (3D) version of a Start Algorithm [10] tracks a spherical target. In [9], the operator controls the needle tip velocity using a

grounded haptic interface, while being provided with visual feedback via three orthogonal US slices that intersect with the needle tip and kinesthetic navigation feedback.

Haptic feedback is another important piece of feedback in robotic teleoperation, which has been proven effective for providing both navigation [1], [3], [9], [11], [12] and environmental information [2], [13]–[15]. For example, Pacchierotti et al. presented a haptic teleoperation system for flexible needles. It enables clinicians to maneuver the surgical tool while providing them with navigation guidance through kinesthetic and vibratory force feedback [3]. Majewicz and Okamura used a haptic interface to command the desired position of the needle tip in Cartesian space and provide force feedback representing kinematic constraints and the position error of the robotic system [16]. Seifabadi et al. developed a 5-degrees-of-freedom (5-DoF) parallel pneumatically actuated modular robot for teleoperated prostate biopsy under MRI [17].

However, it is rare to find teleoperation systems conveying feedback about the needle tip cutting/penetration force, which is often considered to be important [2]. This limitation is mostly due to the difficulty in isolating this force from other undesired components (e.g., friction). In this respect, De Lorenzo et al. proposed a coaxial needle insertion assistant that isolates the cutting force and provides it to the operator with kinesthetic feedback. A smaller inner needle is inserted inside a larger outer one. The outer needle covers the inner one, so as to prevent any friction force from acting on the latter. Then, the tip of the inner needle is left uncovered, so that it can penetrate the tissue while sensing the cutting force only [18]. Similarly, Elayaperumal et al. [19] and Khan et al. [20] used Fiber Bragg Grating (FBG) sensors to enhance force and shape sensitivity of medical instruments. Other approaches use external sensors, combined with needle and soft tissue models [21], [22].

This paper proposes a haptic-enabled robot-assisted teleoperation system for the insertion of flexible needles in soft tissue. It enables intuitive steering of the surgical tool while providing rich feedback information on needle-tissue interaction and navigation toward the target objective. Fig. 1 shows the system, while Fig. 2 describes how each relevant part is interconnected.

With respect to other works on the topic, the proposed system provides an unprecedented amount of environmental and guidance information, conveyed combining grounded (kinesthetic), ungrounded (vibrotactile), and visual feedback. For example, Chevrie et al. also presented an ultrasound-guided teleoperation system for flexible needles, but it only provides navigation feedback through kinesthetic stimuli [9]. The contributions of the paper can be summarized as follows:

* design of a real-time 3D friction force estimation algorithm, so as to provide feedback about the force applied

Manuscript received: October 15, 2020; Revised: January 14, 2021; Accepted: February 28, 2021.

This paper was recommended for publication by Editor Jee-Hwan Ryu upon evaluation of the Associate Editor and Reviewers' comments.

*M. Aggravi and D. A. L. Estima contributed equally to this work.

This work was supported by funds from The Netherlands Organization for Scientific Research (Innovational Research Incentives Scheme-VIDI: SAMURAI project no. 14855).

²D. A. L. Estima and S. Misra are with the Department of Biomechanical Engineering, University of Twente – Enschede, The Netherlands. ⁴S. Misra is also affiliated with the Department of Biomedical Engineering, University of Groningen and University Medical Center Groningen, – Groningen, The Netherlands. e-mails: {danielemos.97@gmail.com, s.misra@utwente.nl}

¹M. Aggravi and C. Pacchierotti are with CNRS, Univ Rennes, Inria, IRISA – Rennes, France. e-mails: {marco.aggravi, claudio.pacchierotti}@irisa.fr

³A. Krupa is with Inria, Univ Rennes, CNRS, IRISA – Rennes, France. e-mail: alexandre.krupa@inria.fr
by the needle tip when cutting/penetrating the tissue;

* design of rich vibrotactile-kinesthetic haptic feedback, so as to simultaneously convey navigation *and* cutting force information in a complete yet intuitive manner;

* integrate the proposed methods with a state-of-the-art ultrasound-driven needle steering algorithm, to attain an effective robot-assisted needle insertion system;

* carry out an extensive human subjects evaluation, aimed at finding the best feedback rendering technique as well as evaluate the system's overall performance (precision, repeatability, completion time, user's preference).

## II. MATERIALS AND METHODS

### A. Experimental setup

The remote system is shown in Fig. 1a. It consists of a Chiba biopsy needle (Angiotech MCN2208, CA) mounted on a 6-DoF force/torque sensor (ATI Nano 43, USA), which is in turn attached to a 6-DoF robotic manipulator (Omron Viper S650 Adept, JP). The needle length is 126 mm, with internal and external diameters of 0.48 mm and 0.70 mm, respectively. The needle's tracking is guaranteed by a 3D ultrasound (US) scanner (Analogic SonixTOUCH, USA) with a 4DC7 3/40 motorized 3D US probe. The 3D volumes are reconstructed in Cartesian coordinates after a 3 × 3 × 3 median filter is applied to remove some noise in the pre-scan data [23]. The steering control and the US-guided tracking algorithms are described in a previous work of our group [9] (see Sec. II-B). A gelatine phantom is prepared following the procedure in [22].

The local system is shown in Fig. 1b. It consists of a grounded haptic interface (Force Dimension Omega.6, CH), a custom wearable vibrating armband, and a screen. The Omega.6 is used as an impedance haptic interface: we measure the position of its end-effector/handle, controlled by the human operator, to steer the needle; at the same time, through the same handle, we provide the operator with kinesthetic feedback from the remote environment. The vibrating armband comprises four actuators, positioned evenly around the arm at 90 degrees from each other [24]. Finally, the screen shows the current ultrasound images and a real-time 3D reconstruction of the needle structure with respect to the target. The overall architecture of our teleoperated needle insertion system is illustrated in Fig. 2.

### B. Needle steering algorithm

The needle steering algorithm translates the velocity commanded by the user for the needle tip, $v_h$, into a 6-DoF velocity command for the robotic manipulator which houses the needle, $v_n$. To do so, we use a Jacobian matrix $J_s$ obtained with the redundancy formalism of the task-function framework [25]. This matrix links the 6-DoF velocity of the needle tip Cartesian frame $\{\mathbb{T}\}$ to that of the needle base Cartesian frame $\{\mathbb{B}\}$ (see Figs. 2 and 3). This control acts similarly to a velocity-control teleoperation (displacements at the local side with respect to the initial position are translated into velocities of the robotic end-effector), modified to consider task priorities as in the stack-of-tasks framework [25]. In this respect, we consider three tasks: i) ensure that the needle base stays aligned with the insertion point, limiting the lateral tissue deformation at the entry point; ii) have the direction of the tip velocity

<figure><img src="image_2.png" /><figcaption>(a) Remote system</figcaption></figure>

<figure><img src="image_3.png" /><figcaption>(b) Local system</figcaption></figure>

Fig. 1. Haptic-enabled teleoperation system. The user controls the robotic manipulator and, in turn, the flexible needle through an Omega.6 haptic interface. Navigation guidance and needle tip cutting force are conveyed through a combination of kinesthetic and vibrotactile feedback, provided by the grounded interface and a wearable vibrotactile device, respectively. Finally, the ultrasound (US) images and a reconstruction of the needle pose are shown on a screen posed in front of the user.

oriented toward the target (this task generates our navigation feedback, see Sec. III-A); and iii) ensure that the circular tip trajectory induced by the bevel is oriented in the desired direction, minimizing the needle bending.

In this work, we use the same framework presented in [9], [26], [27] for tracking the needle shaft from 3D ultrasound and determining the Jacobian that links the needle tip velocity to the base velocity. Moreover, as discussed in Sec. III-A1 and shown in Fig. 2, this framework is also used to compute the guiding reference $f_g$, which helps the human operator driving the needle toward the target.

### C. Friction Force Estimation

As we are interested in rendering the insertion/cutting force at the needle tip but our sensor is only placed at its base, we need to estimate (and then filter out) the friction force acting along the needle shaft.

The total force applied on the needle $f_s(l) \in \mathbb{R}^3$ at its base is registered by our F/T sensor. After we compensate for the needle (and its support) gravity contributions, force $f_s(l)$ is due to the friction between the needle shaft and the tissue $f_f \in \mathbb{R}^3$ as well as to the cutting force of the needle tip $f_t \in \mathbb{R}^3$ [21] (see Fig. 3),

$$f_s(l) = f_f(l) + f_t(l), \quad (1)$$

where $l \in \mathbb{R}^+$ is the length of the needle inside the tissue. Due to the needle bending, the friction force along the shaft has components in three directions (3D). Roesthuis et al. computed these components in a 2D plane using the needle slope as the angle between the needle's centre axis and the coordinate
<figure><img src="image_4.png" /><figcaption>Fig. 2. Haptic teleoperation system. The operator uses a grounded kinesthetic device and a wearable vibrotactile armband (left). A needle steering algorithm (center) translates the human operator's velocity commands $v_h$ into a velocity $v_n$ for the 6-DoF Viper robot (Omron, JP), which houses a 6-axis force/torque (FT) sensor (ATI, USA) and the needle (right). During the insertion, a friction force estimation algorithm filters the FT sensed forces $f_s$ and estimates those due to the penetration of the needle tip into the tissue, $f_t$. A 3D ultrasound (US) probe enables the tracking of the needle $x_{tip}$ and target $x_{targ}$ at runtime, while a steering algorithm calculates the best needle tip direction to reach the target objective, $d_{tip}^n$. The user receives navigation guidance (G) and needle cutting force (C) using a combination of kinesthetic (subscript K, forces $f_g$ and $f_{cf}$) and vibrotactile (subscript V, vibrating motors frequencies $m_g$ and $m_{cf}$) feedback. On the screen, the operator sees the ultrasound images and a real-time reconstruction of the needle pose.</figcaption></figure>

frame [28]. Extending this model to 3D, we can estimate the friction force by using the tangent to the needle shaft

$$ \mathbf{f}_f(l) = \int_{L-l}^{L} f_f(\mathbf{x}_b(c))dc, \qquad (2) $$

where $\mathbf{x}_b(c) \in \mathbb{R}^3$ is the position of a point of the needle at the curvilinear coordinate $c$, $t(\mathbf{x}_b(c)) \in \mathbb{R}^3$ corresponds to the normalized tangent vector at the location $\mathbf{x}_b(c)$, $f_f \in \mathbb{R}$ is defined as friction per unit inserted needle length [28], $l$ is again the inserted length, and $L$ is the total needle length.

By using (2), the needle 3D reconstruction, and the inserted length estimated from the US tracking [9], we can obtain the total friction force $f_f(l)$ over the needle inside the tissue, $l$. However, $f_f$ depends on several factors, such as the tissue properties and insertion velocity, and, in viscoelastic tissues, it will increase as the relative velocity of the needle with respect to the tissue increases [28]. To ease our estimation, we take advantage of the fact that in viscoelastic tissues the friction force has the same amplitude if the needle is being inserted or removed, as long as the needle velocity is the same [29]. For this reason, when removing the needle from the soft tissue, we can assume that $f_t(l)$ in (1) is equal to zero, thus leaving us with $f_s(l) = f_f(l)$.

We approximate the non-straight configuration of the inserted needle to a series of smaller straight segments. Then, we can calculate each segment's tangent $t(\mathbf{x}_b(c))$ and corresponding $f_f$ values. Summing the friction contribution of all segments provides us with the final friction force applied on the needle shaft, $f_s(l)$ [28]. Since the curvature of the needle affects friction [28], a correct estimation of the friction force also relies on the length of the considered segments (more segments equals a better estimation but also a longer computation time). In our case, we used ten segments that start really small and grow as the needle penetrates the gelatine, reaching a maximum length of 13 mm when the needle is fully inserted. In this respect, it would be useful to adjust the number of considered segments to the uncertainty of the current estimation, so as to always guarantee a minimum estimation quality.

To estimate $f_f$, we collected frictional data at different insertion velocities, and then we used such recording to

<figure><img src="image_5.png" /><figcaption>Fig. 3. The robotic insertion system. The needle is inserted into the soft tissue phantom at velocity $v_n$. The force/torque (FT) sensor registers the total force applied to the needle $f_s$, which is composed of the cutting force at the needle tip $f_t$ and the friction force along its shaft $f_f$. The ultrasound (US) tracks the pose of the needle tip $x_{tip}$ and the target $x_{targ}$ at run-time. The steering algorithm calculates the projection of $x_{targ}$ onto the plane perpendicular to the needle $P^\perp$, i.e., $x_{targ}^P$, so as to provide navigation information minimizing the distance $d_{tip}^n$ between $x_{tip}$ and $x_{targ}^P$. Needle insertion guidance toward the target, cutting force, the US images, and a reconstruction on the needle are provided to the user through a combination of visual, kinesthetic, and vibrotactile feedback.</figcaption></figure>

perform a linear regression for estimating the friction force per unit length $f_f$ for any insertion velocity. Of course, this approach can be easily extended to consider more complex friction models [30]. As Chevrie et al. [9] considered 0.005 m/s as the maximum insertion velocity, we included these five velocities in our analysis: $v = [v_1, ..., v_5]^T = [0.001, 0.002, 0.003, 0.004, 0.005]^T$ m/s. For each velocity $v_i$, $i = 1, ..., 5$, (that corresponds to the velocity of the needle along the z-axis of $\mathbb{B}$, i.e., $v_n^z$), we recorded the F/T sensor readings for three 50-mm-long insertions inside the soft-tissue phantom. Then, for each trial, we performed a linear regression to obtain the slope coefficient, and we averaged the results for each velocity. We only considered the data registered during
the retraction of the needle. This approach provides us with a set of five $f_f$ estimates $\mathbf{f}_f^e = [\hat{f}_{f,v_1}, \dots, \hat{f}_{f,v_5}]^T$, one for each considered velocity. We can now estimate at runtime the current $f_f$, depending only on the current needle axial velocity $v_n^z$,

$$f_f(v_n^z) = \begin{cases} \frac{v_n^z \hat{f}_{f,v_1}}{v_1} & \text{if } v_n^z < v_1, \\ \frac{\hat{f}_{f,v_i} + \frac{\hat{f}_{f,v_{i+1}} - \hat{f}_{f,v_i}}{v_{i+1} - v_i}(v_n^z - v_i)}{v_i} & \text{if } v_i < v_n^z < v_{i+1} \end{cases} \quad (3)$$

The upper term represents a linear regression between $v_1$ and zero velocity, while the lower term applies a linear regression between the two closest velocity estimates.

Figure 4a shows an example of insertion with constant velocity along the needle z-axis (needle shaft) $v_n^z$ of ±0.005 m/s. It highlights the registered readings along the z-axis of the F/T sensor $f_s^z$ (left ordinate axis, light blue) and the needle velocity along the direction of its shaft $v_n^z$ (right ordinate axis, orange), over time. It is visible how $f_s^z$ linearly decreases with the retraction (positive sign) or insertion (negative sign) of the needle. Figure 4b shows a detail (red box in Fig. 4a) of the force during the retraction of the needle, for all the considered velocities. Fig. 4c reports all the obtained $f_f$ coefficients. As expected, faster velocities result in higher force variations.

These data are used to filter out the friction force acting on the needle shaft from the total force registered by the F/T sensor. Doing so, we can provide the user with an estimate of the insertion/cutting force at the needle tip. Of course, our feedback rendering algorithms described below can use of any similar friction estimation algorithm, e.g., [18], [19], [21].

### III. FEEDBACK INFORMATION

The user drives the needle inside the soft tissue using a grounded haptic interface, as described in Sec. II-B. The system provides him or her with two types of feedback information: needle direction guidance (Sec. III-A) and cutting force rendering (Sec. III-B). The former helps the user steering the needle toward the given objective (e.g., the area to treat), while the latter provides information about the estimated insertion/cutting force at the needle tip. Both pieces of information are important, and we evaluate how it is best to provide them: through kinesthetic feedback (Secs. III-A1 and III-B1), delivered by the grounded haptic interface, or cutaneous feedback (Secs. III-A2 and III-B2), delivered by a vibrotactile armband.

#### A. Needle direction guidance (G)

Let us define $\mathbf{x}_{targ} \in \mathbb{R}^3$ as the position of the target point (see Fig. 3), $\mathbf{x}_{tip} \in \mathbb{R}^3$ that of the needle tip, $\mathbf{n}_{dir} \in \mathbb{R}^3$ as the normalized direction vector of the needle, and $\mathcal{P}^{\perp}$ as the plane originating in $\mathbf{x}_{tip}$ and with orthogonal vector $\mathbf{n}_{dir}$. Let us also define $\mathbf{x}_{targ}^P$ as the orthogonal projection of $\mathbf{x}_{targ}$ onto $\mathcal{P}^{\perp}$, and $\mathbf{d}_{tip} = \mathbf{x}_{targ}^P - \mathbf{x}_{tip}$ as the vector between the target point and the needle position. Our guiding problem, i.e., driving the user along the direction toward the target, becomes that of minimizing $\mathbf{d}_{tip}$. In other words, the closer the needle tip position $\mathbf{x}_{tip}$ is to $\mathbf{x}_{targ}^P$, the more aligned it is toward the target. In order to provide this guiding information with respect to the operator, as to make it easy and intuitive to understand, we transform the projected target position,

$$\mathbf{x}_{targ}^{P,n} = \mathbf{R}_{us}^{n} \mathbf{x}_{targ}^{P},$$

<figure><img src="image_6.png" /></figure>

<figure><img src="image_7.png" /><figcaption>Fig. 4. Friction force estimation. (a) Time vs. force applied on the needle $f_s^z$ (blue) while applying alternated step velocities $v_i = \pm 0.005$ m/s (orange) along the needle axis direction. (b) Time vs. force applied on the needle for different velocity magnitudes $v_i \in \mathbf{v}$ during needle extraction (magnification of the red square sector above). (c) Friction per unit length $f_f$ for different insertion velocities $v$.</figcaption></figure>

where $\mathbf{R}_{us}^n$ is the rotation matrix bringing $\mathbf{x}_{targ}^P$ from being defined in $\mathbb{W}$ to $\mathbb{T}$. Matrix $\mathbf{R}_{us}^n \in SO(3)$, indicating the rotation between the US frame and the needle tip, is available from the steering algorithm of [9]. With this transformation, the three-dimensional distance $\mathbf{d}_{tip}$ becomes two-dimensional with respect to the needle tip $\mathbb{T}$, since $\mathbf{x}_{targ}^{P,n}$ lies in the same plane $\mathcal{P}^{\perp}$ originated by $\mathbf{x}_{tip}$, i.e., $\mathbf{d}_{tip}^n = [d_{tip}^{n,x}, d_{tip}^{n,z}]^T$. Aligning the needle tip to point toward the target reduces $\mathbf{d}_{tip}^n$ to zero, since the origins of $\mathcal{P}^{\perp}$ and $\mathbf{x}_{targ}^{P,n}$ coincide.

The system can provide this guiding information using either 1) kinesthetic feedback or 2) vibrotactile cutaneous feedback.

1) Needle direction guidance using kinesthetic feedback ($G_K$): The feedback provided to the operator consists of a kinesthetic force $\mathbf{f}_g \in \mathbb{R}^3$ applied by the Omega.6 haptic interface,

$$\mathbf{f}_g = -\mathbf{K}_g(\mathbf{x}_h - \mathbf{x}_{h,0}). \quad (4)$$

where $\mathbf{x}_h$ and $\mathbf{x}_{h,0}$ are the current and rest position of the Omega.6 handle, respectively, and $\mathbf{K}_g \in \mathbb{R}^{3 \times 3}$ is an anisotropic stiffness characterized by low stiffness values along the axis aligned with the needle's target direction, i.e., that minimizing $\mathbf{d}_{tip}^n$, and high stiffness values along other axis [9]. In this condition, users feel a low resistance when moving the haptic interface along the direction toward the target, while they feel a strong resistance when moving in any other direction. This is the only type of haptic feedback conveyed by Chevrie et al. [9] during their teleoperation.
<figure><img src="image_8.png" /><figcaption>Fig. 5. Example of needle direction guidance with vibrotactile feedback, $G_V$. Vibrotactile sensations guide the user toward minimizing $d_{tip}$, i.e., the distance between the needle tip $x_{tip}$ and the projection of the target onto the plane perpendicular to the needle $\mathcal{P}^{\perp}$, i.e., $x_{targ}^P$. In this example, as $t_d < d_{tip}^{n,x} < t_u$, $t_d < d_{tip}^{n,z} < t_u$, $-d_{tip}^{n,x} < t_d$, and $-d_{tip}^{n,z} < t_d$, we calculate $m(1)$, $m(2)$ using the middle term in (5) and $m(3)$, $m(4)$ using the first term.</figcaption></figure>

## 2) Needle direction guidance using vibrotactile feedback ($G_V$):
We map $d_{tip}^n$ onto the four vibrating motors of our wearable haptic armband. As the motors are positioned evenly around the arm at 90 degrees from each other, we can easily provide navigation cues along the transverse plane cutting the arm.

To do so, let us introduce the step-like activation function $m(x)$:

$$m(x) = \begin{cases} 0 & \text{if } x < t^d \\ x^m + (x^M - x^m) \frac{(x - t^d)}{(t^u - t^d)} & \text{if } t^d \le x \le t^u, \\ x^M & \text{if } x > t^u \end{cases} \quad (5)$$

where $t^d$ and $t^u$ indicate the length of the step. Function $m(x)$ is zero until $x$ reaches the lower limit $t^d$, then linearly grows from $x^m$ to $x^M$ between $t^d$ and $t^u$, and it finally plateaus at $x^M$ after that. Let us define $\mathbf{m} = [m_1, m_2, m_3, m_4]$ as the commanded vibrating motors frequencies (see Fig. 1). The minimum and maximum perceived activation frequencies of our motors is 60 and 280 Hz, respectively, [31]. By considering this, and by setting $t^d = 0.002$ m and $t^u = 0.010$ m as activation thresholds, our mapping is described as follows

$$\mathbf{m}_g = \begin{bmatrix} m_1 \\ m_2 \\ m_3 \\ m_4 \end{bmatrix} = \begin{bmatrix} m(d_{tip}^{n,x}) \\ m(d_{tip}^{n,z}) \\ m(-d_{tip}^{n,x}) \\ m(-d_{tip}^{n,z}) \end{bmatrix}. \quad (6)$$

Fig. 5 shows a representative example of this approach with $d_{tip}^n = [3.8, 9.6]^T$ mm. To reach the target, the operator needs to move the needle from $x_{tip}$ toward $x_{targ}^P$. The system conveys navigation information minimizing $d_{tip}^n$, i.e., since $t_d < d_{tip}^{n,x} < t_u$ and $t_d < d_{tip}^{n,z} < t_u$, we use both times the middle term in (5). This leads to the activation of two motors in the armband, guiding the user to move the needle tip toward $x_{targ}^P$.

## B. Cutting force rendering (C)

The second piece of information we want to provide is the cutting/insertion force at the needle tip, estimated with the help of the friction force estimation algorithm described in Sec. II-C. Again, the system can provide this information using either 1) kinesthetic feedback or 2) vibrotactile cutaneous feedback.

## 1) Cutting force rendering with kinesthetic feedback ($C_K$):
In Sec. III-A1, we use the anisotropic stiffness of matrix $K_{cf}$ to keep the user along the right direction of insertion. This matrix can be modified as to provide the operator with cutting force feedback instead of guiding directions. To do so, recalling (1), let us define $\hat{\mathbf{f}}_t = \mathbf{f}_t/f_M \in \mathbb{R}^3$ as the adjusted cutting force at the needle tip, $f_M \in \mathbb{R}^+$ as the maximum force that the haptic device can rendered (12 N for the Omega.6), and

$$\mathbf{K}_h = \begin{bmatrix} k_h & 0 & 0 \\ 0 & k_h & 0 \\ 0 & 0 & k_h \end{bmatrix}$$

as a positive diagonal isotropic stiffness matrix (we consider $k_h = 300$ N/m in our experiments). By setting

$$\mathbf{K}_{cf} = \mathbf{K}_h + \operatorname{diag}(\mathbf{K}_h \hat{\mathbf{f}}_t), \quad (7)$$

we are able to provide information about $\mathbf{f}_t$ with the same approach used in Sec. III-A1, i.e., $K_{cf}$ is an anisotropic stiffness matrix with higher stiffness along the directions spanned by $\mathbf{f}_t$.

Finally, as for Sec. III-A1, the feedback provided to the operator consists in a kinesthetic force $\mathbf{f}_{cf} \in \mathbb{R}^3$ applied by the Omega.6 haptic interface,

$$\mathbf{f}_{cf} = -\mathbf{K}_{cf}(\mathbf{x}_h - \mathbf{x}_{h,0}). \quad (8)$$

## 2) Cutting force rendering with vibrotactile feedback ($G_V$):
We use the same step-like activation function described in (5). However, this time we are interested in providing information about the force variation along the insertion axis $f_t^z$ (see Fig. 1 and Fig. 3). By setting again $x^m = 60$ Hz and $x^M = 280$ Hz as the minimum and maximum activation frequencies of our motors, with $t^d = 0.25$ N and $t^u = 0.75$ N activation thresholds, our mapping is described as follows

$$\mathbf{m}_{cf} = [m_1, \dots, m_4]^T = [m(f_t^z), \dots, m(f_t^z)]^T. \quad (9)$$

In this condition, users feel no vibrations whenever the estimated cutting force stays below $t^d = 0.25$ N. Then, all motors starts to vibrate with increasing intensity until the force reaches $t^u = 0.75$ N. To avoid spurious vibrations due to noise in the estimation of the force, we used a moving average filter to smooth $f_t^z$ before computing $m(f_t^z)$ (window size = 10).

# IV. EXPERIMENTAL EVALUATION

To evaluate the effectiveness of our teleoperated system, we carried out a human subjects study. A video is available as supplemental material and at https://youtu.be/IXZ-n4hzLFM.

## A. Experimental setup and task

We considered the robotic teleoperation system shown in Fig. 1 and described in Sec. II-A. The needle is inserted into a soft-tissue phantom made of gelatine mixture, to which silica powder is added to mimic the acoustic scattering of human tissue [1]. A small elliptical target, i.e., a 3-cm-long olive, is embedded in the gelatine and it is placed at around 9 cm from the surface where the needle is inserted (see also Fig. 3).

Participants were asked to steer the needle inside the gelatine until they punctured the outer surface of the target. They were provided with a combination of navigation and needle tip force
<figure><img src="image_9.png" /><figcaption>Fig. 6. Visual feedback showing US images and the needle reconstruction.</figcaption></figure>

feedback according to the condition being considered (see Sec. III). Before the beginning of each insertion, the needle was perpendicularly pre-inserted into the gelatine for 8 mm, so as to be able to initialize the US tracking. The point of insertion of the needle was randomly chosen among seven options (see video). This approach enabled us to use one gelatine per participant and ensure that the needle had to be always steered along a non-straight path to reach the target.

## B. Subjects

Thirteen participants took part in the experiment, including 2 women and 11 men (age 22–36 years old). One practice trial per feedback condition was allowed. 7 had previous experience with using haptic interfaces. Each participant tested one time each of the seven combinations of devices, for a total of 78 insertion trials. Users were asked to complete the task as precisely as possible taking into account the feedback received.

## C. Feedback modalities

We considered seven different feedback modalities:

* $G_N C_N$: neither needle direction guidance nor cutting force rendering is provided to the user;

* $G_N C_K$: cutting force rendering only, provided by the Omega.6 grounded kinesthetic interface as detailed in Sec. III-B1;

* $G_K C_N$: needle direction guidance only, provided by the Omega.6 kinesthetic interface as detailed in Sec. III-A1;

* $G_N C_V$: cutting force rendering only, provided by the wearable vibrotactile armband as detailed in Sec. III-B2;

* $G_V C_N$: needle direction guidance only, provided by the wearable vibrotactile armband as detailed in Sec. III-A2;

* $G_V C_K$: needle direction guidance provided by the vibrotactile armband (Sec. III-A2) and cutting force rendering provided by the Omega.6 kinesthetic interface (Sec. III-B1);

* $G_K C_V$: needle direction guidance provided by the Omega.6 kinesthetic interface (Sec. III-A1) and cutting force rendering provided by the vibrotactile armband (Sec. III-B2);

Visual feedback (US images and reconstruction of the needle) is always provided to the users through a screen placed in front of the grounded interface (see Fig. 6 and video). Fig. 7 shows a representative insertion in condition $G_K C_V$. The top figure shows the profile of the estimated cutting force $f_t^n$, while the bottom figure shows the evolution of the needle velocities during the insertion. When the needle punctures the olive, the force rises and activates the vibrotactile feedback (in blue).

<figure><img src="image_10.png" /><figcaption>Fig. 7. Cutting force $f_t^n$ and tip insertion velocities $v_n = [v_n^x, v_n^y, v_n^z]$ in a representative insertion in modality $G_K C_V$. The blue-shaded area represents the time when the armband was vibrating.</figcaption></figure>

## D. Results

Figure 7 shows the cutting force profile for one representative insertion in condition $G_N C_V$. The blue-shaded area shows when the armband was activated to inform the operator about the cutting force. The targeting accuracy for this trial was 0.8 mm, which is higher than that recently obtained by Chevrie et al. [9] for a similar setup and task. However, Chevrie et al. only provided navigation information through kinesthetic feedback, similarly to what we do in $G_K$. This promising result is confirmed by the analysis in the remainder of this Section.

As a measure of performance, we registered

(i) the targeting accuracy (how close the needle tip is to the target), i.e., $||x_{tip} - x_{target}||$ at the end of the task;

(ii) the percentage of trials the needle reached the target;

(iii) the insertion time, calculated as the time between the first movement of the needle and the end of the task;

(iv) the average insertion speed, calculated as the integral of the end-effector velocities along the insertion direction;

(v-vi) the perceived effectiveness as registered by the users, evaluated using subjective questionnaires.

To compare the different metrics, we ran one-way repeated-measures ANOVA tests ($a= 0.05$). All data passed the Shapiro-Wilk normality test. Since not all combinations between needle direction guidance ($G_N$, $G_V$, $G_K$) and cutting force rendering ($C_N$, $C_V$, $C_K$) are tested, and interactions between variables are expected, a two-way repeated measures design is not appropriate in this case. Figure 8a shows the (i) targeting accuracy. Mauchly's Test of Sphericity indicated that the assumption of sphericity had been violated ($\chi^2(20) = 63.118, p < 0.001$). The ANOVA test with a Greenhouse-Geisser correction revealed a statistically significant change in the task accuracy between conditions (F(1.819, 21.826) = 32.219, $p < 0.001$). Results of the post hoc analysis with Bonferroni adjustments are shown in Table I. As a complement, Figure 8b shows the (ii) percentage of successful insertions, i.e., when the needle punctured the target. Figure 8c shows the (iii) insertion time. The collected data passed the Mauchly's Test of Sphericity. The ANOVA test revealed no statistically significant change in the insertion time between conditions (F(6, 72) = 0.345, $p > 0.05$). Figure 8d shows the (iv) insertion speed. The collected data passed the Mauchly's Test of Sphericity. The one-way ANOVA test revealed no statistically significant change in the insertion speed between conditions (F(6, 72) = 0.764, $p > 0.05$).
<table><caption>TABLE I
EXPERIMENTAL EVALUATION</caption><caption>Statistical analysis (only significant <i>p</i> values are shown)</caption><thead><tr><th>vs.</th><th>G<sub>N</sub>C<sub>N</sub></th><th>G<sub>N</sub>C<sub>K</sub></th><th>G<sub>K</sub>C<sub>N</sub></th><th>G<sub>N</sub>C<sub>V</sub></th><th>G<sub>V</sub>C<sub>N</sub></th><th>G<sub>V</sub>C<sub>K</sub></th><th>G<sub>K</sub>C<sub>V</sub></th></tr></thead><tbody><tr><th>G<sub>N</sub>C<sub>N</sub></th><td></td><td>.005</td><td>.016</td><td>.001</td><td>.043</td><td>&lt;.001</td><td>&lt;.001</td></tr><tr><th>G<sub>N</sub>C<sub>K</sub></th><td>.002</td><td>-</td><td>.021</td><td>-</td><td>.001</td><td>&lt;.001</td><td></td></tr><tr><th>G<sub>K</sub>C<sub>N</sub></th><td>.001</td><td>-</td><td>-</td><td>.020</td><td>.004</td><td>&lt;.001</td><td></td></tr><tr><th>G<sub>N</sub>C<sub>V</sub></th><td>.001</td><td>.039</td><td>-</td><td>-</td><td>.011</td><td>.001</td><td></td></tr><tr><th>G<sub>V</sub>C<sub>N</sub></th><td>.001</td><td>-</td><td>.046</td><td>-</td><td>-</td><td>&lt;.001</td><td>&lt;.001</td></tr><tr><th>G<sub>V</sub>C<sub>K</sub></th><td>&lt;.001</td><td>&lt;.001</td><td>.044</td><td>.002</td><td>&lt;.001</td><td></td><td>.012</td></tr><tr><th>G<sub>K</sub>C<sub>V</sub></th><td>&lt;.001</td><td>&lt;.001</td><td>.003</td><td>.020</td><td>&lt;.001</td><td>-</td><td></td></tr></tbody></table>

In addition to the objective evaluation reported above, we also measured users' experience. Immediately after the experiment, subjects were asked to report the effectiveness of (v) each one of the seven feedback condition, of (vi) kinesthetic and vibrotactile feedback in providing either needle direction guidance or cutting force. Answers were registered using scales from 0 (not effective at all) to 10 (very effective). Figure 9a shows the perceived effectiveness for the seven experimental conditions. Mauchly's Test of Sphericity indicated that the assumption of sphericity had been violated ($\chi^2(20) = 50.997$, p < 0.001). The one-way ANOVA test with a Greenhouse-Geisser correction revealed a statistically significant change in the perceived effectiveness between conditions (F(2.612, 31.339) = 36.013, p < 0.001). Results of the post hoc analysis with Bonferroni adjustments are shown in Table I. Figure 9b shows the perceived effectiveness for kinesthetic and vibrotactile feedback in providing either needle direction guidance or cutting force ($C_V$ vs. $C_K$ and $G_V$ vs. $G_K$). Two paired samples t-test determined that this metric differed statistically significantly between conditions ($C_V$ vs. $C_K$: t(12) = 6.052, p < 0.001; $G_V$ vs. $G_K$: t(12) = -8.521, p < 0.001).

Finally, subjects were asked to choose the best overall feedback condition and which feedback they preferred for the two provided pieces of information. Eight subjects out of thirteen chose $G_K C_V$, four chose $G_V C_K$, and one chose $G_K C_N$. Nine subjects out of thirteen chose kinesthetic as the best feedback modality for providing guidance, while four chose vibrotactile. Seven subjects out of thirteen chose vibrotactile as the best feedback modality for providing cutting force rendering, while six chose kinesthetic.

V. CONCLUSIONS AND DISCUSSION

We presented an innovative haptic-enabled teleoperation system for the insertion of flexible needles in soft tissue. It combines a state-of-the-art needle tracking and steering framework with a new friction estimation algorithm, with the objective of providing the human operator with informative needle direction guidance and cutting force rendering feedback. To convey both pieces of information in a reliable and easy-to-understand way, we design feedback techniques combining grounded kinesthetic feedback and ungrounded/wearable vibrotactile feedback.

We evaluated the proposed approach in a human subjects experiment enrolling 13 subjects. Participants were asked

<figure><img src="image_11.png" /><figcaption>Fig. 8. Human-subjects study: objective results. Mean and standard deviation of the mean of (a) targeting accuracy, (b) insertion success rate, (c) insertion time, and (d) insertion speed for the seven conditions – from left to right in the figures – $G_N C_N$ (white), $G_N C_K$, $G_K C_N$, $G_N C_V$, $G_V C_N$, $G_V C_K$, $G_K C_V$ (black). The red bars in (a) indicate statistically significant differences.</figcaption></figure>

<figure><img src="image_12.png" /><figcaption>Fig. 9. Human-subjects study: subjective results. Mean and standard deviation of the mean of (a) perceived effectiveness on all conditions and (b) perceived effectiveness per information provided. In (a), from left to right, we have the seven conditions $G_N C_N$ (white), $G_N C_K$, $G_K C_N$, $G_N C_V$, $G_V C_N$, $G_V C_K$, $G_K C_V$ (black). In (b), from left to right, we have the four information provided conditions $C_V$, $C_K$ (red) and $G_V$, $G_K$ (blue). The red bars indicate statistically significant differences.</figcaption></figure>

to insert a flexible needle in a gelatine phantom until they reached a stiff target object. We tested the performance of the teleoperation system in seven experimental conditions, i.e., different combinations of kinesthetic/vibrotactile feedback to convey needle guidance and/or cutting force, measuring both objective and subjective metrics. Results show that conditions employing both kinesthetic and vibrotactile feedback always performed better than those employing only one of them. While this result might seem obvious (more information equals better performance), it is often believed that providing multiple pieces of haptic sensations at the same time results confusing – this was not our case. The most relevant performance differences in the objective results appeared in the target accuracy, while no statistically significant difference was found for the other objective metrics, i.e., participants were more precise but not faster. Our results are comparable with those of [32] in terms of insertion success rate and comparable with [1] and [3] in terms
of insertion accuracy, using visual plus vibratory and kinesthetic plus vibratory feedback, respectively. Another important result is the good match between objective and subjective results. In most cases, conditions that showed good performance in terms of targeting accuracy where also well appreciated by the users. Results also showed that needle direction guidance was significantly better followed/used when provided through kinesthetic feedback with respect to vibrotactile feedback. Conversely, vibrotactile feedback seemed to better render the cutting force, although its benefits with respect to kinesthetic feedback are less evident. This result is again shown both in the objective and subjective metrics. This preference of application for the two types of feedback, i.e., kinesthetic is better/preferred for needle guidance while vibrotactile for cutting force rendering, is not so surprising. Indeed, although vibrotactile feedback has been used for navigation, it is well known that it becomes quickly unpleasant if repeated too often [33]. As also suggested by one subject, this could be the main reason for a degraded performance. On the other hand, the cutting force rendering algorithm only generated vibrations toward the end of the task, when the target object was punctured. This limited activation span and the alarm/buzz nature of the vibrotactile feedback is probably to thank for the increased accuracy performance in the conditions employing it. Finally, we want to highlight a difference in terms of safety between vibrotactile and kinesthetic guidance. Kinesthetic feedback acts directly on the operator's console, making it harder to move away from the preplanned trajectory; on the other hand, vibrotactile feedback provides rich navigation cues while enabling the operators to easily disregard them if necessary, e.g., if the insertion trajectory must be changed suddenly due to an unexpected event.

REFERENCES

[1] M. Abayazid, C. Pacchierotti, P. Moreira, R. Alterovitz, D. Prattichizzo, and S. Misra, "Experimental evaluation of co-manipulated ultrasound-guided flexible needle steering," *Int. J. Med. Robot.*, vol. 12, no. 2, pp. 219-230, 2016.

[2] L. Meli, C. Pacchierotti, and D. Prattichizzo, "Experimental evaluation of magnified haptic feedback for robot-assisted needle insertion and palpation," *Int. J. Med. Robot.*, vol. 13, no. 4, p. e1809, 2017.

[3] C. Pacchierotti, M. Abayazid, S. Misra, and D. Prattichizzo, "Teleoperation of steerable flexible needles by combining kinesthetic and vibratory feedback," *IEEE Trans. Haptics*, vol. 7, no. 4, pp. 551-556, 2014.

[4] O. Gerovichev, P. Marayong, and A. M. Okamura, "The effect of visual and haptic feedback on manual and teleoperated needle insertion," in *MICCAI*, 2002, pp. 147-154.

[5] P. Moreira, L. Kuil, P. Dias, R. Borra, and S. Misra, "Tele-operated mri-guided needle insertion for prostate interventions," *J. Med. Robot. Res.*, vol. 4, no. 01, p. 1842003, 2019.

[6] E. Ben-David, M. Shochat, I. Roth, I. Nissenbaum, J. Sosna, and S. N. Goldberg, "Evaluation of a ct-guided robotic system for precise percutaneous needle insertion," *J. Vasc. Interv. Radiol.*, vol. 29, no. 10, pp. 1440-1446, 2018.

[7] M. Abayazid, G. J. Vrooijink, S. Patil, R. Alterovitz, and S. Misra, "Experimental evaluation of ultrasound-guided 3d needle steering in biological tissue," *Int. J. Med. Robot.*, vol. 9, no. 6, pp. 931-939, 2014.

[8] J. Chevrie, "Flexible needle steering using ultrasound visual servoing," Ph.D. dissertation, Rennes 1, 2017.

[9] J. Chevrie, A. Krupa, and M. Babel, "Real-time Teleoperation of Flexible Beveled-tip Needle Insertion using Haptic Force Feedback and 3D Ultrasound Guidance," in *Proc. IEEE Int. Conf. Robotics and Automation*, 2019, pp. 2700-2706.

[10] N. Friedland and D. Adam, "Automatic ventricular cavity boundary detection from sequential ultrasound images using simulated annealing," *IEEE Trans. Medical Imaging*, vol. 8, no. 4, pp. 344-353, 1989.

[11] M. Selvaggio, F. Abi-Farraj, C. Pacchierotti, P. Robuffo Giordano, and B. Siciliano, "Haptic-based shared-control methods for a dual-arm system," *IEEE Robot. Autom. Lett.*, vol. 3, no. 4, pp. 4249-4256, 2018.

[12] F. Abi-Farraj, C. Pacchierotti, O. Arenz, G. Neumann, and P. R. Giordano, "A haptic shared-control architecture for guided multi-target robotic grasping," *IEEE Trans. Haptics*, vol. 13, no. 2, pp. 270-285, 2019.

[13] F. Chinello et al., "A three revolute-revolute-spherical wearable fingertip cutaneous device for stiffness rendering," *IEEE Trans. Haptics*, vol. 11, no. 1, pp. 39-50, 2017.

[14] J. Bimbo et al., "Teleoperation in cluttered environments using wearable haptic feedback," in *Proc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems*, 2017, pp. 3401-3408.

[15] C. Pacchierotti, F. Chinello, and D. Prattichizzo, "Cutaneous device for teleoperated needle insertion," in *Proc. IEEE RAS & EMBS Int. Conf. Biomedical Robotics and Biomechatronics (BioRob)*, 2012, pp. 32-37.

[16] A. Majewicz and A. M. Okamura, "Cartesian and joint space teleoperation for nonholonomic steerable needles," in *Proc. IEEE World Haptics Conf.*, 2013, pp. 395-400.

[17] R. Seifabadi, S.-E. Song, A. Krieger, N. B. Cho, J. Tokuda, G. Fichtinger, and I. Iordachita, "Robotic system for mri-guided prostate biopsy: feasibility of teleoperated needle insertion and ex vivo phantom study," *Int. J. Comput. Assist. Radiol. Surg.*, vol. 7, no. 2, pp. 181-190, 2012.

[18] D. De Lorenzo, Y. Koseki, E. De Momi, K. Chinzei, and A. M. Okamura, "Experimental evaluation of a coaxial needle insertion assistant with enhanced force feedback," in *Proc. Int. Conf. IEEE Engineering in Medicine and Biology Society*, 2011, pp. 3447-3450.

[19] S. Elayaperumal, J. H. Bae, D. Christensen, M. R. Cutkosky, B. L. Daniel, R. J. Black, J. M. Costa, F. Faridian, and B. Moslehi, "Mr-compatible biopsy needle with enhanced tip force sensing," in *Proc. IEEE World Haptics Conf.*, 2013, pp. 109-114.

[20] F. Khan, R. J. Roesthuis, and S. Misra, "Force sensing in continuum manipulators using fiber bragg grating sensors," in *Proc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems*, 2017, pp. 2531-2536.

[21] A. M. Okamura, C. Simone, and M. D. O'Leary, "Force modeling for needle insertion into soft tissue," *IEEE Trans. Biomedical Engineering*, vol. 51, no. 10, pp. 1707-1716, 2004.

[22] N. J. Van De Berg, T. L. De Jong, D. J. Van Gerwen, J. Dankelman, and J. J. Van Den Dobbelsteen, "The influence of tip shape on bending force during needle insertion," *Scientific Reports*, vol. 7, no. 1, pp. 1-8, 2017.

[23] D. Lee and A. Krupa, "Intensity-based visual servoing for non-rigid motion compensation of soft tissue structures due to physiological motion using 4d ultrasound," in *Proc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems*, 2011, pp. 2831-2836.

[24] M. Aggravi, F. Pausé, P. R. Giordano, and C. Pacchierotti, "Design and evaluation of a wearable haptic device for skin stretch, pressure, and vibrotactile stimuli," *IEEE Robot. Autom. Lett.*, vol. 3, no. 3, pp. 2166-2173, 2018.

[25] S. B. Slotine, "A general framework for managing multiple tasks in highly redundant robotic systems," in *Proc. Int. Conf. Advanced Robotics*, vol. 2, 1991, pp. 1211-1216.

[26] J. Chevrie, N. Shahriari, M. Babel, A. Krupa, and S. Misra, "Flexible needle steering in moving biological tissue with motion compensation using ultrasound and force feedback," *IEEE Robot. Autom. Lett.*, vol. 3, no. 3, pp. 2338-2345, 2018.

[27] J. Chevrie, A. Krupa, and M. Babel, "Online prediction of needle shape deformation in moving soft tissues from visual feedback," in *Proc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems*, 2016, pp. 2375-2380.

[28] R. J. Roesthuis, Y. R. Van Veen, A. Jahya, and S. Misra, "Mechanics of needle-tissue interaction," in *Proc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems*, 2011, pp. 2557-2563.

[29] A. Asadian, R. V. Patel, and M. R. Kermani, "Dynamics of translational friction in needle-tissue interaction during needle insertion," *Ann. Biomed. Eng.*, vol. 42, no. 1, pp. 73-85, 2014.

[30] B. Armstrong-Hélouvry et al., "A survey of models, analysis tools and compensation methods for the control of machines with friction," *Automatica*, vol. 30, no. 7, pp. 1083-1138, 1994.

[31] S. Scheggi, M. Aggravi, and D. Prattichizzo, "Cooperative navigation for mixed human-robot teams using haptic feedback," *IEEE Trans. Human-Machine Systems*, vol. 47, no. 4, pp. 462-473, 2016.

[32] C. Rossa et al., "Multiactuator haptic feedback on the wrist for needle steering guidance in brachytherapy," *IEEE Robot. Autom. Lett.*, vol. 1, no. 2, pp. 852-859, 2016.

[33] L. Devigne, M. Aggravi, M. Bivaud, N. Balix, C. S. Teodorescu, T. Carlson, T. Spreters, C. Pacchierotti, and M. Babel, "Power wheelchair navigation assistance using wearable vibrotactile haptics," *IEEE Trans. Haptics*, vol. 13, no. 1, pp. 52-58, 2020.