# Ultrasound Mid-Air Haptics for Hand Guidance in Virtual Reality

Lendy Mulot, Thomas Howard, Claudio Pacchierotti, Maud Marchal

<figure><img src="image_1.png" /></figure>

## To cite this version:

Lendy Mulot, Thomas Howard, Claudio Pacchierotti, Maud Marchal. Ultrasound Mid-Air Haptics for Hand Guidance in Virtual Reality. IEEE Transactions on Haptics (ToH), 2023, 16 (4), pp.497-503. 10.1109/TOH.2023.3269521. [hal-04075641](https://hal HAL Id: hal-04075641)

https://inria.hal.science/hal-04075641

Submitted on 20 Apr 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
# Ultrasound Mid-Air Haptics
## for Hand Guidance in Virtual Reality

Lendy Mulot, Thomas Howard, Claudio Pacchierotti, Maud Marchal

***Abstract***—This paper presents and evaluates a set of mid-air ultrasound haptic strategies to provide 2-degree-of-freedom position and orientation guidance in Virtual Reality (VR). We devised four strategies for providing position guidance and two for providing orientation guidance. A human subject study evaluated the effectiveness of the proposed techniques in guiding users towards objectives in static and dynamic environments in VR, both in position and orientation. Results show that, compared to visual feedback of the virtual environment alone, the considered strategies significantly improve positioning performance in the static scenario. On the other hand, orientation guidance led to significant improvements only in the dynamic scenario.

***Index Terms***—Haptic Rendering, Human Performance, Mid-air Haptics, Virtual Reality

## I. INTRODUCTION

Providing guidance feedback in Virtual Reality (VR) is beneficial in a wide range of applications, including industrial, sport, and medical training. Guidance cues are often conveyed visually [1]. However, a large body of work studies the use of haptic interfaces for guidance in real-world applications, such as in turn-by-turn navigation [2], driving [3], teleoperation [4]–[6], locomotion [7]–[9], and for the support of people with sensory disabilities [10]. Notably, Kappers et al. [8] presented a review of hand-held haptic devices for human walking navigation. Haptic guidance can also be used to steer the movements of single limbs, notably for application of motor rehabilitation [11] or prosthesis. For example, Feygin et al. [12] used kinesthetic cues to train participants at following a 3D trajectory while holding the haptic end-effector in their hand. Asque et al. [13] used a similar kinesthetic approach to help motion-impaired users precisely point and click on a visual keyboard. Pezent et al. [14] delivered composite haptic guidance in a dynamic trajectory following task, whereby task forces are relayed via a kinesthetic haptic interface and guidance forces are provided via skin stretch. Battaglia et al. [15] studied the use of a skin stretch haptic device to provide intuitive proprioceptive feedback to indicate the hand closure of an underactuated prosthesis.

Haptic guidance in VR has not been extensively studied, compared to applications outside of VR. Examples of haptic guidance in VR include directional feedback on the wrist [16] or cheeks [17], and orientation feedback on the fingertips [18]. Guidance information in VR is nonetheless beneficial under many circumstances. First, immersion in VR removes any direct visual feedback of one's own body which may be used

to support proprioception. Second, a key application of VR is training, a field in which motion guidance appears essential.

Furthermore, VR interaction can benefit from unobtrusive haptic feedback techniques which leave users unencumbered and free to engage in concurrent interactions. In this respect, ultrasound mid-air haptics (UMH) is of particular interest as it provides haptic sensations without the need for the user to wear or grasp any interface [19].

UMH devices feature arrays of ultrasonic transducers which produce phase-shifted acoustic waves that constructively interfere at points in space, called focal points, and destructively interfere elsewhere, conveying haptic sensations by varying acoustic radiation pressure on the skin [20]. Despite the many promising applications of UMH [21], to the best of our knowledge, the technology has not yet been applied to providing guidance feedback in VR. However, it has been shown that UMH can be effective at providing guidance cues in real-world applications. Freeman et al. [22], Yoshimoto et al. [23], and Suzuki et al. [24] started to explore the possibility of using UMH for hand guidance. Freeman combined visual and haptic stimulation to help users correctly position their hand with respect to the ultrasound array for optimal feedback; Yoshimoto moved a focal point that users had to follow; and Suzuki guided users to follow a virtual haptic rail generated by a large UMH array.

For these reasons, we expect contactless mid-air haptics to be capable of providing rich guidance information to the unencumbered user in VR. For instance, VR training of fine motor skills for surgery (e.g. suture knot tying) may benefit from UMH guidance despite the technology's workspace restrictions. This paper presents and evaluates mid-air ultrasound haptics techniques, in continuity with the current state of the art, for providing 2-Degrees-of-Freedom (2DoF) positioning and orienting guidance feedback in static and dynamic VR environments.

## II. UMH STRATEGIES FOR HAND GUIDANCE

We designed 6 UHM guiding strategies in order to study user performances: 4 providing 2DoF position guidance cues and 2 providing 2DoF orientation ones, summarized in Fig. 1. The strategies were rendered using spatiotemporal modulation (STM), with a focal point speed of 8 m.s⁻¹ so as to maximize its perceived intensity [25]. The pattern shapes were chosen to expand on the work of Freeman [22] and Suzuki [24].

a) *Position guidance:* Considering the user's hand position $\mathbf{H} \in \mathbb{R}^2$ and the target position to reach $\mathbf{T} \in \mathbb{R}^2$, the distance vector between the user's hand and the target is defined as $\mathbf{T} - \mathbf{H}$. We devised four position guiding techniques:

* HAND (see Fig. 1(a)) provides information about the distance to the target, $d = \|\mathbf{T} - \mathbf{H}\|$, with no information

L. Mulot, T. Howard and M.Marchal are with University of Rennes, INSA, IRISA, France.

C. Pacchierotti is with CNRS, Univ Rennes, Inria, IRISA, France.

M. Marchal is also with IUF, France.

This work was funded under the ANR project "MIMESIS".
<figure><img src="image_3.png" /><figcaption>Fig. 1. (Left) Representation of the four positioning guidance strategies for two different target positions (top and bottom). Green solid circles and lines represent the haptic pattern rendered by the ultrasound haptic interface. In HAND-OFFSET and HAND-LINE, the red dashed circle shows the circumference on which the haptic circle is rendered when the target is far away. In all but HAND-LINE, the green cross shows where the haptic circle is centered. **T** and **H** crosses correspond to the target and hand positions, respectively. (Right) Representation of the two orienting guidance strategies for three different target positions (left, center, right). In the top images, black and gray vectors represent the normal vector the participant's hand and target hand, respectively. Orientation errors with respect to **α** and **β** require movements along the proximal-distal and radial-ulnar axis of the hand, respectively.</figcaption></figure>

regarding direction. To do so, we render a haptic circle with a varying radius at the center of the palm, **H**. The radius of the circle varies proportionally to the distance *d* to the target, with a maximum of *r* = 4 cm, to fit inside most hands, when *d* = 10 cm. The center of the circle is always rendered at the same point on the hand, **H**.

* TARGET (Fig. 1(b)) is similar to the HAND strategy. However, this time, the haptic circle with a varying radius is always centered at the target, **T**. As before, the radius varies proportionally to the distance *d*, with a maximum of *r* = 4 cm when *d* = 10 cm. Of course, in this case, as the circle is centered on the target, users feel no stimulation if they are further than 4 cm from it.

* HAND-OFFSET (Fig. 1(c)) provides information about both distance and direction to the target. Similarly to HAND, we render a haptic circle with a varying radius on the hand. However, instead of always rendering it at the center of the palm, this time we center it on the border of the palm, to indicate the direction towards the target. Specifically, if *d* > 3 cm, the circle is centered at the intersection between a 3-cm-radius circle centered in **H** and **T** – **H**; on the other hand, if *d* ≤ 3 cm, the circle is centered on the target **T**. As before, the radius of the circle varies proportionally to the distance *d*, with a maximum of *r* = 4 cm when *d* = 10 cm.

* HAND-LINE (Fig. 1(d)) also provides information about distance and the direction to the target, but using a line instead of a circle. To do so, we render a segment of varying length along the direction **T** – **H**, starting from **H** and growing proportionally to the distance *d*, with a maximum length of *l* = 6 cm when *d* = 10 cm. We used a wider pattern to make up for the fact that a circle covers a larger area of the hand.

b) Orientation guidance: The orientation of the hand can be considered as a composition of two rotations, one defined by a flexion/extension of the wrist and one by its prona-

tion/supination. Angles α and β indicate the orientation error with respect to flexion/extension and pronation/supination movements, respectively. For the purposes of this paper, we do not consider rotations defined by radial/ulnar deviations of the wrist. We devised two guiding techniques:

* 2LINES (Fig. 1(e)) provides information about α and β errors by rendering two orthogonal segments of varying length, starting at **H**: one segment runs along the proximal-distal hand axis and one along the radial-ulnar hand axis. The length of the segments is proportional to |α| and |β|, reaching a maximum length of *l* = 4 cm when the angular error is more than 30° around the considered axis. If the first segment grows towards the distal direction of the hand (the fingers), the user should extend the wrist; if it grows towards the proximal direction of the hand (the wrist), the user should flex the wrist. Similarly, if the second segment grows towards the radial direction of the hand (the thumb), the user should supinate the wrist; if it grows towards the ulnar direction of the hand, the user should pronate the wrist.

* 1LINE (Fig. 1(f)) provides the same information in a composite way, by rendering the vector addition of the two vector segments described in the 2LINES strategy.

The threshold values of 10 cm for the positioning and 30° for the orienting strategies were chosen empirically. Higher values would provide varying information in a bigger workspace while leading to slower variations of the stimulus, requiring larger hand motion to feel any variation. If participants go beyond this value, they still receive feedback, but it will not grow any larger. Reaching the optimal configuration reduces the pattern to a point, which will no longer be perceptible, indicating that the target has been reached.

### III. EVALUATION IN A STATIC VR ENVIRONMENT

Our objective is to evaluate the proposed ultrasound haptic strategies in providing 2DoF guidance information to users in
VR. We seek to answer two main questions:

* Can we guide the user to a certain hand position and orientation using UMH feedback?

* If we can, what is the best guiding strategy among the ones we designed?

A video of the experiment is shown at https://youtu.be/GdIh0VLIGE4.

## A. Materials and Methods

1) Apparatus: The experimental setup is shown in the top of Fig. 2. Participants were seated facing the table, and wore an HTC VIVE Tracker strapped to the back of their dominant hand. They observed the virtual environment through a HTC VIVE Pro Eye VR headset, with the headphones playing pink noise to mask any audio cues from the haptic interface. Tracking errors were found to be way below the size of a focal point [20], [26], and were thus considered negligible. They were asked to place their dominant hand facing the haptic interface. They could rest the arm on an armrest between trials.

The UHM feedback was provided by an Ultraleap STRATOS Explore device. The haptic device was housed on a custom stand, keeping the array in a vertical position, perpendicular to the table supporting it.

<figure><img src="image_4.png" /><figcaption>Fig. 2. (Top) Experimental setup. Participants seat near the ultrasound haptic interface, with their dominant hand facing the device. They wear a VR HMD and a tracker. They hold a controller in their other hand to validate trials and start blocks. (Bottom) VR scene, where the interaction area is indicated by a translucent squared area. During training only, the scene also showed the rendered haptic sensations (blue circle in this case).</figcaption></figure>

The VR environment (developed in Unity3D) is illustrated at the bottom of Fig. 2. It was composed of a translucent 28 cm-edge square representing the area where the participants should interact. This square area was placed 20 cm away from the haptic device to ensure good haptic perception [27]. A small green cube, located 25 cm under the center of the interaction area, acted as a button allowing users to proceed to the next trial. A virtual hand avatar mimicked the movements

of the user's hand in the virtual environment. During training, the scene also showed a visual rendition of the haptic cues in the form of blue circles or lines, similar to that in Fig. 1.

2) Task and protocol: Participants were immersed in the VR scene described above. At the beginning of each task, the target hand pose is displayed for 1 s thanks to an additional virtual hand appearing in the scene. This gives participants the time to create a mental image of the target. The visual indication of the target then disappears and participants are asked to move their own hand to match this target pose, both in terms of position and orientation, being as accurate as possible, and relying on the mental image they constructed.

Participants performed the task using each of the six guidance strategies presented in Sec. II and under a baseline condition without haptic feedback. Participants were told whether the current strategy provided position or orientation guidance (see Sec. II), but had to match the target hand pose position and orientation in all trials. This design enables us to analyze whether providing guidance on one information component affects the other, by e.g. relieving cognitive load, an important topic in many haptics applications, e.g., robot-assisted surgery [28]. As mentioned above, we included a strategy without haptic feedback. In this case, participants relied only on the visual information provided by the target virtual hand briefly showed at the beginning of each trial.

The experimental protocol was divided in two main parts, evaluating the position and orientation guidance strategies, respectively. The order was balanced between participants. For both parts, we carried out an additional set of tasks with no haptic guidance. This design led to a total of 4 (position strategies) +1 (no haptic feedback) +3 (orientation strategies) +1 (no haptic feedback) = 8 blocks. The block order for each participant was computed using a balanced Latin square. Each block started with a training phase, where the feedback strategy under consideration was explained orally by the experimenter and textually with an illustration in the VR scene (similar to what is shown in Fig. 1). After training, participants carried out 15 guidance trials, composed of 3 repetitions of 5 predefined target hand positions and orientations. These poses covered as much of the available workspace as possible, both in terms of position and orientation.

3) Collected Data: Before the beginning of the experiment, participants were asked to fill a questionnaire about their self-identified gender, age, dominant hand, experience with VR and haptics, and whether they had previously used an UMH device. During the experiment, we recorded the trial completion times, as well as the final positioning and angular errors. After each of the two parts of the experiment (positioning and orientation guidance), participants were asked to rank the haptic strategies they just experienced based on perceived effectiveness.

4) Participants: 20 participants took part in the study (17 M, 2 F, 1 NB, age ± s.d. = 25.8 ± 5.56). 7 of them used VR weekly, 10 of them used it yearly, and 3 of them had no experience with VR at all. 8 of them used haptic interfaces weekly, 5 of them yearly, and 7 of them had no experience with haptics at all. 7 participants had already used an UMH device at least once, and 13 had not. 17 participants were right-handed, 2 were left-handed, and 1 was ambidextrous (and used their right hand).
<caption>TABLE I</caption>

RESULTS OF THE STATIC AND DYNAMIC USER STUDIES. H-O, H-L, AND N-F STAND FOR THE HAND-OFFSET, HAND-LINE, AND NO HAPTIC FEEDBACK (NO-HAPTIC) STRATEGIES, RESPECTIVELY. WE REPORT THE MEAN VALUE FOR THE RANKING AND THE MEDIAN FOR THE OTHER METRICS.

<table><thead><tr><th colspan="8">Evaluation in a static VR environment</th><th colspan="5">Evaluation in a dynamic VR environment</th></tr><tr><th>Strategy</th><th>HAND</th><th>TARGET</th><th>H-O</th><th>H-L</th><th>N-H</th><th>2LINES</th><th>1LINE</th><th>N-H</th><th>Strategy</th><th>H-O</th><th>N-F</th><th>1LINE</th><th>N-H</th></tr></thead><tbody><tr><td>Error</td><td>11 mm</td><td>10 mm</td><td>8.7 mm</td><td>16.7 mm</td><td>20 mm</td><td>9.8°</td><td>10.6°</td><td>10.6°</td><td>Error</td><td>17.8 mm</td><td>19 mm</td><td>7.2°</td><td>9.3°</td></tr><tr><td>Duration</td><td>5.9 s</td><td>5.5 s</td><td>5.4 s</td><td>3.5 s</td><td>2.1 s</td><td>5.2 s</td><td>3.8 s</td><td>2.2 s</td><td>Duration</td><td>4.88 s</td><td>5 s</td><td>4.7 s</td><td>3.6 s</td></tr><tr><td>Ranking</td><td>2.95 / 4</td><td>2.95 / 4</td><td>1.63 / 4</td><td>2.47 / 4</td><td></td><td>1.63 / 2</td><td>1.37 / 2</td><td></td><td>Difficulty</td><td>2.375 / 7</td><td></td><td>4.75 / 7</td><td></td></tr></tbody></table>

We observed little variability in the participant's hand sizes. However, if needed, all the strategies can be adapted to different hand sizes by modifying the circle radii or line length. The personalization of haptic rendering is an open and interesting topic in the field [29].

## B. Results

The left of Table I shows the main results of this experiment.

1) *Comparisons Based on Logged Data:* We gathered a total of 1500 positioning trials and 900 orienting trials. Error data were far from normal, as assessed by Shapiro-Wilk tests. We therefore applied a log transformation to the data. Fig. 3 (Top) shows the errors in position and orientation, while Fig. 3 (Bottom) shows the completion times.

<figure><img src="image_5.png" /><figcaption>Fig. 3. (Top) Box plots of the errors in reaching the target pose. Horizontal bars represent the median value. (Top-Left) Positioning errors are computed as the distance (m) of the 2D vector between the hand and the target. (Top-Right) Orienting errors are computed as the absolute angle (°) between the user's and the target's rotation. (Bottom) Box plot of the completion time (s) for each positioning (Bottom-Left) and orienting (Bottom-Right) strategy. From left to right, strategies are HAND, HAND-LINE, HAND-OFFSET, NO-HAPTICS, TARGET, 1LINE, 2LINES, NO-HAPTICS.</figcaption></figure>

a) *Position/orientation errors:* We modelled the positioning and orienting errors with mixed linear models with respect to the haptic strategies (4 and 2 degrees of freedom, df) and considering the participants as a random effect. With these models, we then performed an analysis of deviance.

The analysis of deviance showed significant differences in positioning errors between the positioning strategies ($F = $

30.623, $p < 0.001$). Post-hoc analysis using a Tukey test showed that all pairwise comparisons were significant ($p < 0.05$), except for HAND-OFFSET vs. TARGET and HAND vs. TARGET. This result shows that all the proposed haptic strategies allowed improved positioning performances compared to the NO-HAPTIC scenario. Considering the pairwise comparisons and median results, the HAND-OFFSET and TARGET strategies were considered to be the best.

We carried out the same comparisons for the orienting strategies. An ANOVA ($F = 0.4949, p > 0.05$) showed no significant effect of the haptic feedback on the user's orienting performance for this experiment.

b) *Completion time:* We followed the same methodology as in Sec. III-B1a to investigate the effects of employing different haptic strategies on the duration of the trials.

The analysis showed once again a significant difference across the positioning guidance strategies ($F = 153.838, p < 0.001$). A post-hoc analysis using a Tukey test revealed that all pairwise comparisons were significant ($p < 0.05$) except for HAND vs. HAND-OFFSET, HAND vs. TARGET, and HAND-OFFSET vs. TARGET. This result shows that the three strategies rendering a circle (HAND, TARGET, HAND-OFFSET) take a similar amount of time, while that rendering a line is significantly faster, although still significantly slower than NO-HAPTIC.

Significant differences across the orienting guidance strategies ($F = 213.367, p < 0.001$) were also detected by the analysis of deviance. A post-hoc analysis using a Tukey test revealed that all comparisons were significant ($p < 0.001$), meaning that the NO-HAPTIC is faster than the 1LINE strategy, which is turn faster than the 2LINES strategy.

2) *Subjective rankings:* We analysed the subjective rankings that users submitted at the end of each experimental part (position and orientation), shown in Table I. The NO-HAPTIC was not included in the ranking. We had to remove the ranking data from one participant who did not fill the form correctly.

A Friedman test showed a significant difference across the position guidance strategies ($F = 4.389, p = 0.004$). A post-hoc analysis using pairwise Wilcoxon tests revealed that the HAND-OFFSET strategy was ranked significantly better than the TARGET and HAND strategies ($p = 0.04$ and $p < 0.001$ respectively).

A Friedman test did not find a significant difference across the orientation guidance strategies ($F = 1.316, p > 0.05$).

3) *Discussion:* Results showed that the TARGET and HAND-OFFSET strategies offered better performances, while the HAND-OFFSET and HAND-LINE were perceived as
being the most intuitive and effective by our subjects. Thus, we can consider the HAND-OFFSET strategy as being the best strategy out of the four positioning strategies. This result is in accord with the work of Freeman *et al.* [22] and Suzuki *et al.* [24], who showed that hand guidance was possible with circular and linear patterns in a non-virtual environment. On the other hand, the only significant comparison with respect to the orienting strategies (completion time) showed that 1LINE allows comparable performance and level of intuitiveness than 2LINES, but it requires less time. Therefore, we can consider the 1LINE strategy as being better than the 2LINES one.

## IV. EVALUATION IN A DYNAMIC VR ENVIRONMENT

Following the first experiment's promising results, we extended our investigation to a dynamic/changing virtual environment.

### A. Materials and Methods

1) **Apparatus:** The experimental setup is the same as described in Sec. III-A1 and shown in the top of Fig. 2, except for the VR environment, which is composed of a ping pong table. The participants are immersed in the environment at one end of the table, and the virtual representation of their hand in VR is a paddle (see the video at https://youtu.be/GdIh0VLIGE4). Similarly to the first experiment, we placed a 28×28 cm translucent rectangle where the user-controlled paddle should move to ensure good haptic perception [27]. A “ghost” paddle represents the projection of the user-controlled paddle onto this area to encourage users to stay within it.

2) **Task and Protocol:** For this experiment, participants were provided with either the HAND-OFFSET or 1LINE strategies, as we considered them to be the best for the positioning and orienting tasks, respectively (see Sec. III-B3).

Similarly as before, the experiment is divided in two parts, one for evaluating position guidance and one for evaluating orientation guidance. For the position guidance tasks, a ping pong ball is launched towards the participant, who has to intercept it by moving the paddle around. In this case, the orientation of the paddle is not important, the only objective is to catch the ball. For the orientation guidance tasks, a ping pong ball is again launched towards the participant. However, this time, the ball always ends up in the same location, and participants need to adjust the orientation of their hand/paddle to bounce the ball towards a specific target above the net (red-circled pad in the video). In both cases, users can construct a mental image of how the ball will behave, in a similar way as in the first experiment. As the environment is dynamic, this image will be refined as the ball moves.

Each part (position and orientation) is divided into two blocks, one providing the considered haptic feedback strategy and one providing no haptic feedback. Similarly as before, each block started with a training phase, where the feedback strategy under consideration is explained. The task and block order are randomized to avoid order effects. After the training, participants carried out 15 guidance trials, considering three different speeds for the ball and random target positions and orientations. These poses are again designed to cover as much as possible the available workspace, both in terms of position

and orientation. Once participant were confident with their position or orientation, they pressed the controller trigger to speed up the ball and proceed to the next trial.

3) **Collected Data:** We gathered the same pre-experimental data as in the first experiment (see Sec. III-A3). During the experiment, at the end of each trial, we registered the trial duration as well as the position and angular errors. After each of the two parts of the experiment (positioning and orientation guidance), participants were asked to rate the perceived difficulty of the task, how well they understood the haptic stimuli, and what cues did they rely on (from visual only to haptics only) on a 7-points Likert scale.

4) **Participants:** 8 participants participated in the study (8 M, 1F, age ± s.d. = 25.8 ± 4.6). 4 of them had plenty of experience with VR and 4 of them had little experience with VR. 4 of them had plenty of experience with haptics and 4 of them had little experience with haptics. 7 participants were right-handed and 1 was ambidextrous; everyone used their right hand. All participants had already used an UMH device at least once. 5 of them also participated in the first experiment.

### B. Results

The right of Table I shows the main results of this experiment and the data distribution is illustrated in Fig. 4.

1) **Performance Analysis:** We gathered a total of 720 positioning trials and 720 orienting trials.

As the data did not follow a normal distribution (Shapiro-Wilk; *p* < 0.001), we used the same analysis protocol as in Sec. III-B1a. The ANOVA revealed a significant impact of the guiding strategy on the orienting error (*p* < 0.001). This proves that the orienting guidance feedback does indeed help participants orient their hand. The tests also revealed a significant effect on the trial duration during the orientation tasks (*p* < 0.001). We also observed a significant effect of ball speed on the orienting performance (*F* = 11.362, *p* < 0.001, less speed leads to better results, from 9.7° in a 2.5 s trial, to 7.2° in a 7 s trial), showing that participants need some time to understand and follow this cue.

On the other hand, we observed no significant effect of the feedback on the positioning error, nor on the positioning trial duration.

2) **Discussion:** Contrary to the previous study of Sec. III-B3, the positioning feedback did not improve positioning errors, while the orienting strategies did. This result shows that orientation guidance is possible with UMH technologies, in line with previous studies in UMH [19] and with other technologies [30]. This difference between the two results

<figure><img src="image_6.png" /><figcaption>Fig. 4. Results for the user study in a dynamic environment. (Left) Positioning (cm) and orienting (°) errors. (Right) Trial completion times (s).</figcaption></figure>
can be explained by the difference in task difficulty. On a 1 to 7 Likert scale, participants ranked the difficulty of the positioning task with an average of 2.375, and the orienting task with an average of 4.75. A Wilcoxon paired test showed that this difference was statistically significant (p = 0.012). Similarly, the positioning task revealed to be easy enough to be carried out without haptic feedback.

## V. CONCLUSION AND PERSPECTIVES

In this work, we presented different UMH guiding strategies designed to help users position and orient their hand in VR environments. We evaluated them in static and dynamic VR environments, revealing significant effects of the strategies on the performances, depending on task difficulty.

Position and orientation feedback were both shown to be potentially beneficial in tasks where they are pertinent. Positioning feedback reduced errors in the static task, yet orientation feedback was found to be ineffective in this task. On the other hand, orientation feedback was found to be effective in the dynamic task, where the complexity of the orientation task was greater than that of the positioning task. These results lead us to hypothesize that task complexity is determining in the effectiveness of a provided guidance feedback scheme. We also showed that haptic feedback often significantly slowed task execution, as participants are able to refine their position. This result means that these techniques can be beneficial in scenarios requiring high precision while not being extremely time sensitive.

The current study design does not allow us to draw any inference regarding what task difficulty thresholds may influence the effectiveness of UMH guidance feedback. This will be the topic of future investigations, along with the optimization of the guidance strategy designs by investigating the perception of varying shapes (including points). Notably, we will also look into extending these techniques to 3D and continuous motion guidance. To get the full 3DoF information, one may either modulate another haptic parameter or add a 1DoF visual feedback, although more studies on both topics are required.

UMH guidance in VR was found to be effective under certain circumstances. We hope this will open perspectives for improving applications that could benefit from haptic guidance, such as e.g., VR training. A “peg in hole” game is a typical example of scenario that could benefit from 2DoF positioning and orienting guidance feedback. Also, techniques exist to increase the relatively small workspace of UMH devices [27], [31], [32], which could make such feedback useful in a wider variety of virtual scenarios.

## VI. ACKNOWLEDGMENTS

The authors would like to thank P.-A. Cabaret and G. Gicquel for their help in creating the video accompanying this paper.

## REFERENCES

[1] Y. Harada et al., "Quantitative evaluation of visual guidance effects for 360-degree directions," *Virtual Reality*, vol. 26, pp. 759-770, 2022.

[2] M. Prasad et al., "HaptiMoto: turn-by-turn haptic route guidance interface for motorcyclists," in *Proc. SIGCHI Conf.*, 2014, pp. 3597-3606.

[3] P. Di Campli San Vito et al., "Haptic navigation cues on the steering wheel," in *Proc. ACM CHI Conf.*, 2019, p. 1-11.

[4] F. Abi-Farraj et al., "A haptic shared-control architecture for guided multi-target robotic grasping," *IEEE Trans. Haptics*, vol. 13, pp. 270-285, 2019.

[5] M. Cognetti et al., "Perception-aware human-assisted navigation of mobile robots on persistent trajectories," *IEEE Robot. Autom.*, vol. 5, pp. 4711-4718, 2020.

[6] M. Aggravi et al., "Design and evaluation of a wearable haptic device for skin stretch, pressure, and vibrotactile stimuli," *IEEE Robot. Autom.*, vol. 3, no. 3, pp. 2166-2173, 2018.

[7] P. Preechayasomboon et al., "Chasm: A Screw Based Expressive Compact Haptic Actuator," in *Proc. ACM CHI Conf.*, 2020, pp. 1-13.

[8] A. M. Kappers et al., "Hand-held haptic navigation devices for actual walking," *IEEE Trans. Haptics*, vol. 15, no. 4, pp. 655-666, 2022.

[9] L. Kuang et al., "Design of a 2-dof haptic device for motion guidance," in *Proc. EuroHaptics Conf.*, 2022, pp. 198-206.

[10] L. Devigne et al., "Power wheelchair navigation assistance using wearable vibrotactile haptics," *IEEE Trans. Haptics*, vol. 13, pp. 52-58, 2020.

[11] M. Sarac et al., "Design requirements of generic hand exoskeletons and survey of hand exoskeletons for rehabilitation, assistive, or haptic use," *IEEE Trans. Haptics*, vol. 12, no. 4, pp. 400-413, 2019.

[12] D. Feygin et al., "Haptic guidance: Experimental evaluation of a haptic training method for a perceptual motor skill," *Proc. HAPTICS Symp.*, pp. 40-47, 2002.

[13] C. Asque et al., "Haptic-assisted target acquisition in a visual point-and-click task for computer users with motion impairments," *IEEE Trans. Haptics*, vol. 5, pp. 120-130, 2012.

[14] E. Pezent et al., "Spatially separating haptic guidance from task dynamics through wearable devices," *IEEE Trans. Haptics*, vol. 12, pp. 581-593, 2019.

[15] E. Battaglia et al., "Skin stretch haptic feedback to convey closure information in anthropomorphic, under-actuated upper limb soft prostheses," *IEEE Trans. Haptics*, vol. 12, pp. 508-520, 2019.

[16] J. Hong et al., "Evaluating angular accuracy of wrist-based haptic directional guidance for hand movement," *Graphics Interface*, pp. 195-200, 2016.

[17] F. Nakamura et al., "Evaluation of spatial directional guidance using cheek haptic stimulation in a virtual environment," *Frontiers in Computer Science*, vol. 4, 2022.

[18] T. L. Baldi et al., "Hand guidance using grasping metaphor and wearable haptics," *IEEE Haptics Symp.*, pp. 961-967, 2020.

[19] T. Howard et al., *Ultrasound Mid-Air Tactile Feedback for Immersive Virtual Reality Interaction*, 2022, ch. Ultrasound Mid-Air Haptics for Touchless Interfaces, pp. 147-183.

[20] I. Rakkolainen et al., "A survey of mid-air ultrasound haptics and its applications," *IEEE Trans. Haptics*, vol. 14, pp. 2-19, 2021.

[21] O. Georgiou et al., *Ultrasound Mid-Air Haptics for Touchless Interfaces*. Springer, 2022.

[22] E. Freeman et al., "Haptiglow: Helping users position their hands for better mid-air gestures and ultrasound haptic feedback," *Proc. IEEE World Haptics Conf.*, pp. 289-294, 2019.

[23] A. Yoshimoto et al., "Midair haptic pursuit," *IEEE Trans. Haptics*, vol. 12, no. 4, pp. 652-657, 2019.

[24] S. Suzuki et al., "Midair hand guidance by an ultrasound virtual handrail," *Proc. IEEE World Haptics Conf.*, pp. 271-276, 2019.

[25] W. Frier et al., "Using spatiotemporal modulation to draw tactile patterns in mid-air," in *Proc. EuroHaptics Conf.*, 2018, pp. 270-281.

[26] P. Bauer et al., "Accuracy investigation of the pose determination of a vr system," *Sensors*, vol. 21, no. 5, p. 1622, 2021.

[27] T. Howard et al., "Pumah: Pan-tilt ultrasound mid-air haptics for larger interaction workspace in virtual reality," *IEEE Trans. Haptics*, vol. 13, no. 1, pp. 38-44, 2019.

[28] C. Pacchierotti et al., "Cutaneous feedback of fingertip deformation and vibration for palpation in robotic surgery," *IEEE Trans. Biomedical Engineering*, vol. 63, no. 2, pp. 278-287, 2015.

[29] E. Young et al., "Compensating for fingertip size to render tactile cues more accurately," *IEEE Trans. Haptics*, vol. 13, no. 1, pp. 144-151, 2020.

[30] K. Drewing et al., "First evaluation of a novel tactile display exerting shear force via lateral displacement," *ACM TAP*, vol. 2, no. 2, pp. 118-131, 2005.

[31] D. Brice et al., "A proof of concept integrated multi-systems approach for large scale tactile feedback in vr," *AVR*, pp. 120-137, 2019.

[32] S. Inoue et al., "Horn: the hapt-optic reconstruction," in *ACM SIGGRAPH*, 2014.