# XR-HAP-Enabling Haptic Interaction in Extended Reality: Challenges, Directions, and Opportunities

Francesco Chinello, Nikhil Deshpande, Claudio Pacchierotti, Orestis Georgiou, Guido Gioioso, Konstantinos Koumaditis

► To cite this version:

Francesco Chinello, Nikhil Deshpande, Claudio Pacchierotti, Orestis Georgiou, Guido Gioioso, et al. XR-HAP-Enabling Haptic Interaction in Extended Reality: Challenges, Directions, and Opportunities. ISMAR 2024 - 23rd IEEE International Symposium on Mixed and Augmented Reality, Oct 2024, Seattle, United States. pp.672-677, 10.1109/ISMAR-Adjunct64951.2024.00205. hal-04823125

HAL Id: hal-04823125

https://hal.science/hal-04823125v1

Submitted on 6 Dec 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# XR-HAP - Enabling Haptic Interaction in Extended Reality: Challenges, Directions, and Opportunities

Francesco Chinello *

Nikhil Deshpande †

Claudio Pacchierotti ‡

Aarhus University, Denmark.

University of Nottingham, UK.

CNRS, Univ Rennes, Inria, IRISA, Rennes, France.

Orestis Georgiou §

Guido Gioioso ¶

Konstantinos Koumaditis ‖

Ultraleap Ltd, Bristol, UK

WEART S.r.l., Milan, Italy

Aarhus University, Denmark.

## ABSTRACT

In the current digital era, virtual artefacts and immersive technologies have become integral tools in users’ daily lives. These technologies enable interaction with digital worlds through immersive wearable systems (such as VR headsets and XR solutions) or screen-based three-dimensional artefacts. Within this context, haptics plays a crucial role as the primary means of allowing users to “touch” virtual objects in digital environments. Despite its symbiotic relationship with visual XR technology in digital interfaces, haptics has followed a distinct path, presenting unique challenges, opportunities, and solutions. This tutorial aims to provide the audience with an overview of haptic technology in digital scenarios. We will explore current development directions in haptic technology in XR, existing limitations and challenges in haptic interaction, and the diverse ways in which haptic technology intersects with digital interaction. By examining these aspects, we hope to foster a deeper understanding of the potential and constraints of haptic technology in enhancing user experiences in virtual environments.

**Index Terms:** Haptics, Interaction, Multimodality, Mid-air haptics.

## 1 INTRODUCTION

Recent advancements in virtual and digital environments have significantly enhanced user interaction through virtual scenarios, augmented reality, and mixed reality. Collectively, these visual digital technologies are referred to as “Extended Reality” (XR), where the “X” includes “V” for Virtual, “A” for Augmented, or “M” for Mixed Reality. While the taxonomy of these technologies is continually evolving, the literature already offers models that represent various levels of digital abstraction to comprise these elements and their variations [33, 46].

Interaction within virtual environments typically occurs via hand-held controllers or interfaces, often accompanied by specific haptic paradigms. Haptics can be defined as the sum of knowledge and technology focused on studying and utilizing the sense of touch in user interactions with environments, digital artefacts, and objects or in teleoperated systems [42]. The reason why the usual interaction in immersive XR happens through hand-held controllers, relies on its simplicity and relatively limited cost of realization, which also delivers the greatest ease of use across different age groups [11]. However, over the years, technological advancements in headsets have led to various interaction approaches in XR, such

as using bare hands and cameras to enable more “natural” interactions with virtual scenes, directly through the use of hands. These approaches allow a more “natural” interaction without the use additional hand-held hardware but come with limitations in tracking capabilities and user embodiment [7, 51].

Interacting with virtual objects through bare hands has opened new possibilities for more direct and intuitive user engagement with virtual objects, but it also raises essential development directions. In fact, while hand-held controllers offer the possibility of intuitive vibrotactile feedback, bare-hand interactions eliminate the touch component entirely. Furthermore, users often encounter limitations when interacting with digital artefacts, such as the inability to perceive the consistency or weight of objects. In response, researchers have developed several hand-based solutions to address these limitations [20, 23, 10].

As humans, we rely heavily on the sense of touch to understand our surroundings, communicate, and regulate our actions. The sense of touch is mediated by mechanoreceptors distributed across the skin, muscles, and tendons [24]. These elements of the tactile system contribute to user information, with receptors predominantly located in the skin. Meissner corpuscles, for instance, are primarily found in the dermis and non-hairy skin, where they are particularly sensitive to low-frequency vibrations, aiding in the detection of contact or no-contact conditions [13, 14]. Pacinian corpuscles, located deeper in the subdermal layers, are fast-adapting and sensitive to higher frequencies (40-400 Hz). Two additional receptors, Merkel cells and Ruffini endings, are part of the skin's slow-adapting receptors and are sensitive to slow changes, such as variations in pressure, essential to guarantee a stable and efficient grasp during or manipulative tasks [14, 8].

Before the revolutionary changes we are facing in the digital era of XR, tactile systems already played a fundamental role in many applications. Historically, haptic feedback has been primarily employed in teleoperation and remote manipulation to provide sensory information to the operator during the interaction between a robotic system and a specific operative environment [21]. Beyond teleoperation, haptic feedback has become increasingly important in virtual and augmented reality applications [16, 32]. In these immersive environments, users found haptics as an effective middleware to interact with digital objects in a way that feels tangible and realistic, bridging the gap between the physical and virtual worlds, increasing enjoyment and immersion. The growing interest of the XR audience pushed forward the development directions for haptics with the creation of a variety of solutions, including gloves, belts, and devices for headsets and suits. Despite the momentum, the technological development for new haptic systems remains challenging, with high costs, long testing, and device complexity. Furthermore, advancements in cutaneous haptics are expanding the use of tactile feedback in everyday applications, facilitating accessibility for visual impairment subjects, and providing new forms of communication through touch [40, 35, 37]. As technology continues to evolve, the role of tactile systems in human-computer interaction is likely to grow, offering new possibilities for how we perceive and

<footnote>*e-mail: chinello@btech.au.dk</footnote>

<footnote>†e-mail: nikhil.deshpande@nottingham.ac.uk</footnote>

<footnote>‡e-mail: claudio.pacchierotti@irisa.fr</footnote>

<footnote>§e-mail: orestis.georgiou@ultraleap.com</footnote>

<footnote>¶e-mail: guido.gioioso@weart.it</footnote>

<footnote>lb-mail: kkoumaditis@btech.au.dk</footnote>
interact with the digital world.

In this context, the presented work aims to highlight the chal-
lenges, current state, and future developments of haptics in XR
across the following areas:

* Immersive VR and Wearable Haptics

* Ultrasound Haptics: the technology, the opportunities in XR

* Multimodal Haptics: applications in science and industry

* XR Interfaces and Intuitiveness

## 2 WEARABLE AND HAND-HELD HAPTICS FOR IMMERSIVE INTERACTIONS IN XR

This tutorial workshop part focuses on wearable and hand-held hap-
tics for immersive interaction in XR.

The rapid advancement of Virtual Reality (VR) and Augmented
Reality (AR) technologies has opened new opportunities in human-
computer interaction. While visual and auditory feedback have
been the primary focus of these immersive technologies for a long
time, more recently, the integration of haptic feedback has emerged
as a crucial element in enhancing the sense of presence and realism
in virtual and extended reality environments [29, 36]. Wearable
and hand-held haptics, in particular, have gained significant atten-
tion due to their potential to provide unobtrusive and continuous
tactile feedback to users in such scenarios [38].

**Content Overview:** Wearable and hand-held haptics have emerged as pivotal technologies in the domain of human-computer interaction, offering tangible interfaces enriching the user experience by simulating ungrounded touch sensations. This tutorial will commence with an exploration of the foundational principles underlying wearable and hand-held haptic devices, discussing their progression and the technological milestones that have motivated their development.

The versatility of wearable and hand-held haptics is evident in
their widespread applications across various sectors. In the auto-
motive industry, these technologies enhance driver interaction with
in-vehicle systems, providing tactile feedback that can reduce cog-
nitive load and improve safety [47]. In navigation and mobility as-
sistance, such ungrounded devices have been widely used for pro-
viding rich guidance and directional haptic cues [6, 26, 25]. In tele-
robotics, they have been employed for providing information about
remote contacts and navigation guidance [44, 1], often in combi-
nation with grounded kinesthetic interfaces [39]. As ungrounded
cutaneous feedback does not affect the safety/stability in teleopera-
tion systems, particularly promising in telemedicine and telesurgery
applications [37, 9].

Within the realms of Virtual Reality (VR) and Augmented Real-
ity (AR), wearable haptics has been proven to increase immersion
and agency. With respect to standard grounded haptic interfaces,
wearable and hand-held haptics provide a much wider workspace
and freedom of movement of the user. The successful implemen-
tation of wearable and hand-held haptics is indeed rooted in a few
important design principles and methodologies [38]. They are in-
fluenced by factors such as form factor (small, compact designs that
fit naturally on the body are preferred, with special attention to ac-
tuator size and device placement), weight (device weight should be
proportional to the strength of the body part it's worn on, to avoid
discomfort), impairment (devices should not interfere with natural
body movement and should ensure kinematic compatibility with the
body), and comfort (devices should avoid causing discomfort, such
as pressure or skin irritation, by using ergonomic designs, soft ma-
terials, and adjustable features.). This tutorial analyses critical de-
sign guidelines essential for developing effective haptic interfaces.
Emphasis will be placed on user-centric design, ensuring that haptic
devices not only function well, but also deliver enriching user ex-
periences that are personalised for one's needs and preference [28].

Looking forward, the tutorial will also address the challenges
and opportunities that define the future landscape of wearable and
hand-held haptics. Integration of these devices into daily life
poses questions related to comfort, wearability, and social accep-
tance. The potential for augmenting haptic feedback with multi-
modal stimuli, such as combining tactile sensations with auditory
and visual cues, presents exciting avenues for enhancing user en-
gagement. Additionally, advancements in material science and soft
robotics promise the development of more adaptable and respon-
sive haptic interfaces. Participants will be encouraged to contem-
plate innovative applications and design strategies that can address
current limitations and unlock the full potential of such haptic tech-
nologies.

**Expected Outcomes for the Audience:** This tutorial workshop session aims to provide attendees with an understanding of the current landscape and future potential of wearable and hand-held haptics in XR environments. Participants will get to know the fundamental principles underlying these technologies, gaining insights into the latest advancements that have shaped the field. The session will equip attendees with knowledge of critical design considerations and methodologies essential for creating effective haptic interfaces.

By exploring diverse applications across sectors such as naviga-
tion, telerobotics, and XR, participants will develop a broad per-
spective on the versatility and impact of haptic technologies. The
tutorial will address both the challenges and opportunities in in-
tegrating haptic feedback into daily life and XR experiences. At-
tendees will also gain exposure to emerging trends, including the
integration of haptics with other sensory modalities. By facilitating
discussions on innovative applications and future directions, the tu-
torial aims to inspire new research and encourage collaborations
that could lead to groundbreaking advancements in wearable and
hand-held haptics for XR.

**Scientific Contribution:** This tutorial workshop session aims to provide attendees with an understanding of the current landscape and future potential of wearable and hand-held haptics in XR environments. Participants will get to know the fundamental principles underlying these technologies, gaining insights into the latest advancements that have shaped the field. The session will equip attendees with knowledge of critical design considerations and methodologies essential for creating effective haptic interfaces.

By exploring diverse applications across sectors such as naviga-
tion, telerobotics, and XR, participants will develop a broad per-
spective on the versatility and impact of haptic technologies. The
tutorial will address both the challenges and opportunities in in-
tegrating haptic feedback into daily life and XR experiences. At-
tendees will also gain exposure to emerging trends, including the
integration of haptics with other sensory modalities. By facilitating
discussions on innovative applications and future directions, the tu-
torial aims to inspire new research and encourage collaborations
that could lead to groundbreaking advancements in wearable and
hand-held haptics for XR.

**Scientific Contribution:** This tutorial workshop session makes a contribution to the field of wearable and hand-held haptics for XR by synthesising current knowledge and highlighting interdisciplinary connections. By bringing together the latest research and developments, the session provides an overview of the state of the art, identifying trends, gaps, and opportunities in the current body of knowledge. This synthesis serves as a valuable resource for researchers and practitioners, offering a foundation for future investigations and technological innovations. The interdisciplinary nature of haptic research is emphasized throughout the session, demonstrating how insights from human-computer interaction, robotics, and psychology converge in the development of effective haptic interfaces. The integration of diverse scientific disciplines contributes to a more holistic understanding of haptic technology and its ap-
plications, potentially opening new avenues for cross-disciplinary collaboration and innovation. Ultimately, by focusing on the application of haptic technologies in XR, the session contributes to the broader goal of creating more immersive and realistic virtual experiences. This has important implications not only for entertainment but also for applications such as training, education, and therapy. The scientific insights shared in this tutorial have the potential to drive advancements in XR experiences, making them more engaging, intuitive, and effective across different domains.

## 3 ULTRASOUND HAPTICS: THE TECHNOLOGY AND OPPORTUNITIES IN XR

The focus of this tutorial workshop session is on mid-air haptics, a cutting-edge technology that enables tactile feedback in the absence of physical contact [18]. This innovative field leverages ultrasonic waves to create sensations in mid-air, enriching user experiences in various applications such as automotive interfaces, virtual and augmented reality (VR/AR), public displays, healthcare, and scientific communication.

Over the past 10-15 years [22], the technology has advanced significantly, and so has our understanding of how to apply it most effectively [19]. To that end, this talk will cover the foundational principles, practical applications, design guidelines, and future challenges of mid-air haptics.

**Content Overview:** Mid-air haptics represent a significant advancement in the realm of human-computer interaction, offering a novel way to bridge the gap between physical and digital experiences. By introducing the concept of providing cutaneous sensation via ultrasonic waves, mid-air haptics is one of the few solutions able to provide haptic feedback without the necessity of physical contact with the user's skin, providing the opportunity for new solutions, scenarios, and paradigms in XR interaction. The tutorial will commence with an introduction to the fundamental principles of mid-air haptics, tracing its historical development and the key technological breakthroughs that have shaped its evolution. By comparing mid-air haptics with other haptic technologies, we will highlight its unique advantages and capabilities.

The exploration of mid-air haptics will then transition to its diverse applications across multiple domains, from industry and human-computer interaction to automotive. In the automotive sector, mid-air haptics enhances in-vehicle infotainment systems, reduce driver distraction, and improve user interaction through touchless controls [50]. Public displays benefit from mid-air haptics by offering interactive kiosks and digital signage that engage users while providing a hygienic alternative to touch-based interactions [12]. In the immersive worlds of VR and AR, mid-air haptics significantly boosts realism and immersion, leading to better user experiences and more precise interactions [30]. Healthcare applications include training medical professionals and enhancing patient interactions with touchless interfaces and simulations. Furthermore, mid-air haptic stimulation is being utilised in scientific communication to intuitively convey complex concepts, making them more accessible and engaging. Central to the success of mid-air haptics is the application of well-defined design guidelines and methodologies. The tutorial will delve into twelve key design guidelines that are essential for creating effective mid-air haptic interfaces. Participants will learn about an iterative haptic design framework that encompasses requirement gathering, designing alternatives, prototyping, and rigorous evaluation. This session emphasises the importance of user experience in the design and implementation of mid-air haptics, ensuring that the technology not only functions effectively but also enhances the overall user interaction.

Finally, looking towards the future, the tutorial will address the challenges and opportunities that lie ahead for mid-air haptics. As the technology continues to evolve, integrating it seamlessly into various applications presents both obstacles and exciting possibilities.

The potential for further enhancing user experience through multimodal feedback and innovative design approaches will be explored, encouraging participants to think creatively about the future of this technology [17].

**Expected Outcomes for the Audience:** By the end of this tutorial, participants will have a comprehensive understanding of mid-air haptics, including an overview of the applications, benefits, and challenges. They will gain insights into the design principles and methodologies for creating effective haptic experiences and learn about the latest research and practical implementations in the field. This knowledge will empower attendees to integrate mid-air haptics into their own projects, fostering innovation and enhancing user interaction in their respective domains.

**Scientific Contribution:** This talk aims to advance the field of human-computer interaction and XR in particular by providing a deep dive into mid-air haptics, highlighting its potential to transform user experiences across various domains while also standardising some of the terminology, challenges, and best practices. In addition to the dissemination of cutting-edge research and practical insights, the tutorial will nudge innovation by encouraging the exploration of new applications that apply the design guidelines described within our structured framework. Ultimately, we aim to inspire attendees to leverage mid-air haptics in their projects and foster collaboration and innovation in computer science, virtual reality, and human-computer interaction.

## 4 MULTIMODAL HAPTICS: APPLICATIONS IN SCIENCE AND INDUSTRY

The focus of this tutorial workshop session is on multimodal haptics and its applications in science and industry, particularly in the context of extended reality (XR) environments. Human interaction through touch involves a rich variety of sensations beyond simple gestures like exploring with hands, caressing, or squeezing objects. When we interact with objects or other humans, we experience a complex array of sensations including temperature, skin stretch, strain, kinesthetic sensations, audiovisual stimuli, and more. These sensations are crucial for our cognitive processing, allowing us to understand and represent our environment in scenarios ranging from danger assessment to object manipulation and exploration. The totality of possible sensations that users can experience through touch, skin receptors, nerves, and corpuscles are commonly referred to as "modalities." This concept is closely tied to that of "sensory," typically indicating the type of sensation from the user's perception associated with a specific stimulus [45]. This area of research, pivotal in neuroscience, has crucial elements that are vital for haptic research in general.

**Content Overview:** The tutorial will begin by exploring the concept of modalities in haptics, emphasising how different types of stimuli correspond to various sensory experiences. We will discuss how modalities such as kinesthetic, proprioceptive, cutaneous, and tactile feedback contribute to our overall haptic perception [24]. We will then analyse the challenges of isolating specific modalities in real-world interactions, as well as the potential benefits of doing so in certain applications. For instance, we will examine the concept of "sensory subtraction" in teleoperation tasks, where tactile and kinesthetic feedback can be selectively "removed" from the interaction, allowing users to utilise specific modalities as needed [43]. The importance of cutaneous feedback in object manipulation within XR environments will be highlighted, demonstrating how it enables more stable and natural grip and manipulation of both physical and virtual objects [3, 41]. We will explore various stimuli associated with different haptic modalities, such as vibration (often provided through vibrotactile motors like eccentric rotating mass or linear resonant actuators) and temperature sensations [24, 15]. The tutorial will then transition to discussing the current state and challenges of haptics in everyday applications. We
will examine the common use of vibrotactile feedback in mobile and wearable devices, its applications in gaming and entertainment, and the design challenges affecting the development of haptic systems [48]. Finally, we will explore how industries and researchers are pushing the boundaries of haptic interactions in XR, showcasing advanced technological solutions such as haptic gloves for multimodal cutaneous feedback (e.g. WEART), kinesthetic feedback (e.g., Senseglove, HaptX) and devices that provide tactile stimuli with minimal physical constraints (e.g., Ultraleap) [5, 38]. An overview of relevant use cases will be given, relating them with the combination of haptic stimuli adding more value for the final user. The analysis will include industrial cases, marketing applications and entertainment scenarios, also covering scalability and market readiness aspects.

**Expected Outcomes for the Audience:** The aim of this part of the tutorial is to elucidate the participants with an understanding of multimodal haptics and their applications in science and industry, particularly within XR environments. They will gain insights into the various haptic modalities and their roles in human perception, the challenges and benefits of isolating specific haptic modalities, the importance of cutaneous feedback in object manipulation in XR, current applications and limitations of haptic technology in everyday devices, emerging trends and advanced solutions in haptic technology for XR. Attendees will be equipped with the knowledge to critically evaluate haptic interfaces and consider the integration of multimodal haptic feedback in their own XR projects or research.

**Scientific Contribution:** This session of the tutorial contributes to the field of disseminating the role of haptics in XR by synthesising current knowledge on multimodal haptics and highlighting its applications across various domains. By bringing together insights from neuroscience, human-computer interaction, and XR technology, the session provides a holistic view of the state of the art in multimodal haptics. The interdisciplinary nature of this topic is emphasized throughout the session, demonstrating how insights from various fields converge in the development of effective haptic interfaces for XR. Integrating diverse scientific disciplines contributes to a more comprehensive understanding of haptic technology and its applications, potentially opening new avenues for cross-disciplinary collaboration and innovation. By focusing on the application of multimodal haptics in XR, the session contributes to the broader goal of creating more immersive and realistic virtual experiences. This has important implications not only for entertainment but also for applications such as training, education, and therapy.

# 5 XR INTERFACES AND INTUITIVENESS

Intuitiveness and immersion are key themes in XR. The most prominent way of enhancing intuitiveness is through adding features, e.g., 3D information, overlays, etc., in the visual feedback [34]. As noted in the earlier sections, haptic technology is also a key element that can help elevate the intuitiveness and immersion of XR experiences, making virtual interactions feel more real and engaging. Incorporating the sense of touch into virtual interactions deepens immersion, making users feel as though they are physically interacting with virtual objects. For instance, feeling the texture of a virtual object or the resistance when pressing a button or opening a door can make the experience more realistic [2, 27]. This is closely linked to cognitive processes as well. Haptic feedback can help users understand and remember interactions within the XR environment. For example, in therapeutic XR applications, haptic feedback can reinforce learning by providing tactile cues that complement visual and auditory information [31]. This tutorial focuses on how haptics plays a crucial role in the user interface to enhance and improve intuitiveness and immersion in XR.

**Content Overview:** The tutorial will begin by introducing the concept of intuitiveness and immersion through haptics, developing the understanding in the context of XR interaction. The idea

of how haptic feedback can contribute to overall intuitiveness and immersion for the user in XR will be discussed [49]. The tutorial will highlight the importance of having intuitiveness and immersion in XR interactions and how such effects can be measured quantitatively and qualitatively in human-XR interactions. Then, the tutorial will introduce the importance of having haptic feedback in XR interaction, with different modalities, e.g., pressure, vibrations, temperature, tactile, etc., as well as the benefits of this in real-world applications. For instance, the concept of “encountered-type haptics” (ETH) will be examined where the haptic feedback is tied to the visual interactions with objects, e.g., vibrations when pressing virtual buttons, allowing users to have contextually relevant feedback without overwhelming the senses [49, 27]. The tutorial will then discuss the state-of-the-art of intuitiveness in XR using haptics in everyday applications, as well as the limitations thereof. The use of ETH, haptic devices, vibration, as well as temperature will be examined, with applications in different sectors, e.g., training, therapy, and gaming [31, 4]. Finally, the tutorial will showcase how the advancement in haptic technology is evolving towards enhancing intuitiveness and immersion even further in different applications, e.g., multimodal cutaneous feedback (WEART), kinesthetic feedback (e.g., Force Dimension, Haption), etc. The tutorial will include an overview of relevant use cases, relating them with the research and commercial applications in various domains.

**Expected Outcomes for the Audience:** The tutorial aims to give the participants a comprehensive understanding of intuitiveness and immersion in XR and how haptics plays a vital role in achieving that. Participants will explore:

* Role of intuitiveness in enhancing XR interaction

* Importance of haptics in improving intuitiveness and immersion in XR

* Benefits and challenges in achieving intuitiveness with haptic feedback for XR users

* Current applications and limitations of haptic technology in everyday devices

* Emerging trends and advanced solutions in haptic technology for intuitiveness in XR

Attendees will be equipped with the knowledge to understand and critically evaluate haptic feedback as a modality of enhancing intuitiveness and immersion in XR into their own projects or research.

**Scientific Contribution:** This tutorial workshop session advances the field of haptics and XR by consolidating current knowledge on intuitiveness and immersion in XR and showcasing its diverse applications. By integrating insights from human-computer interaction, haptics, and XR technology, the session offers a comprehensive overview of the latest advancements in intuitive XR with haptics. The session highlights the vital nature of this topic, illustrating how haptics contribute as a key element, along with the other senses, in experiencing intuitiveness and immersion effectively in XR. This fusion of insights from different disciplines enhances our understanding of haptic technology and its role in improving human XR experience. The tutorial highlights the use of haptics in XR as a pathway to the broader objective of creating more intuitive, immersive, and realistic XR experiences for humans. This can only help improve the use of XR in a broader range of applications including industrial training, education, rehabilitation and therapy, as well as gaming and entertainment.
# 6 CONCLUSION

The integration of haptics into XR is a rapidly advancing field, presenting exciting opportunities to enhance user experience, immersion, and realism. This tutorial highlights the key advancements in wearable and hand-held haptics, providing a deeper understanding of their design principles, applications, and limitations. With the emerging field of mid-air haptics, wearable haptics, and multimodal devices, examining their unique capabilities and potential to revolutionise user interaction in XR scenarios, the perspective in using touch for interaction in XR applications leaves space for relevant advancements in automotive, human-computer interaction and cutaneous feedback in teleoperation. Further, with the discussion about the concept of multimodal haptics, emphasising the diverse sensations involved in human touch and the importance of integrating multiple modalities to create more realistic haptic experiences, the crucial role that haptics plays in improving the intuitiveness and immersion in XR applications appear more clear. Open challenges remain the present and future work that researchers and companies are attempting to address, related to device design, user comfort, and integration into everyday life applications; all crucial aspects for the continued development of haptics in XR. Looking ahead, advancements in materials science, soft robotics, and multimodal feedback systems hold the promise of unlocking even more immersive and intuitive haptic experiences. By fostering interdisciplinary research and collaboration, we can continue to push the boundaries of haptics technology, paving the way for transformative applications in gaming, training, healthcare, and beyond.

# ACKNOWLEDGEMENTS

OG was supported from the EU Horizon 2020 research and innovation programme under grant agreement No 101017746.

# REFERENCES

[1] M. Abayazid, C. Pacchierotti, P. Moreira, R. Alterovitz, D. Prattichizzo, and S. Misra. Experimental evaluation of co-manipulated ultrasound-guided flexible needle steering. *The International Journal of Medical Robotics and Computer Assisted Surgery*, 12(2):219-230, 2016. 2

[2] B. Araujo, R. Jota, V. Perumal, J. X. Yao, K. Singh, and D. Wigdor. Snake charmer: Physically enabling virtual objects. In *Proceedings of the TEI '16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction*, TEI '16, page 218-226, New York, NY, USA, 2016. Association for Computing Machinery. 4

[3] A.-S. Augurelle, A. M. Smith, T. Lejeune, and J.-L. Thonnard. Importance of cutaneous feedback in maintaining a secure grip during manipulation of hand-held objects. *Journal of neurophysiology*, 89(2):665-671, 2003. 3

[4] D. E. Benrachou, M. Masmoudi, O. Djekoune, N. Zenati, and M. Ousmer. Avatar-facilitated therapy and virtual reality: next-generation of functional rehabilitation methods. In *2020 1st International Conference on Communications, Control Systems and Signal Processing (CCSSP)*, pages 298-304. IEEE, 2020. 4

[5] C. Bermejo and P. Hui. A survey on haptic technologies for mobile augmented reality. *ACM Computing Surveys (CSUR)*, 54(9):1-35, 2021. 4

[6] E. Bouzbib, L. Kuang, P. Robuffo Giordano, A. Lécuyer, and C. Pacchierotti. Survey of Wearable Haptic Technologies for Navigation Guidance. 2023. 2

[7] G. Buckingham. Hand tracking for immersive virtual reality: opportunities and challenges. *Frontiers in Virtual Reality*, 2:728461, 2021. 1

[8] D. G. Caldwell, N. Tsagarakis, and A. Wardle. Mechano thermo and proprioceptor feedback for integrated haptic feedback. In *Proceedings of International Conference on Robotics and Automation*, volume 3, pages 2491-2496. IEEE, 1997. 1

[9] M. Chauhan, N. Deshpande, C. Pacchierotti, L. Meli, D. Prattichizzo, D. G. Caldwell, and L. S. Mattos. A robotic microsurgical forceps

for transoral laser microsurgery. *International Journal of Computer Assisted Radiology and Surgery*, 14:321-333, 2019. 2

[10] F. Chinello, M. Malvezzi, D. Prattichizzo, and C. Pacchierotti. A modular wearable finger interface for cutaneous and kinesthetic interaction: control and evaluation. *IEEE Trans. Industrial Electronics*, 67(1):706-716, 2019. 1

[11] D. M. Cook, D. Dissanayake, and K. Kaur. Virtual reality and older hands: Dexterity and accessibility in hand-held vr control. In *Proceedings of the 5th international acm in-cooperation hci and ux conference*, pages 147-151, 2019. 1

[12] L. Corenthy, M. Giordano, R. Hayden, D. Griffiths, C. Jeffrey, H. Limerick, O. Georgiou, T. Carter, J. Müller, and S. Subramanian. Touchless tactile displays for digital signage: mid-air haptics meets large screens. In *Extended abstracts of the 2018 CHI conference on human factors in computing systems*, pages 1-4, 2018. 3

[13] I. Darian-Smith. The sense of touch: performance and peripheral neural processes. *Sensory Processes*, 1984. 1

[14] J. Feher. Cutaneous sensory systems. *Quantitative human physiology*, pages 389-399, 2012. 1

[15] D. Filingeri. Neurophysiology of skin thermal sensations. 2016. 3

[16] R. Galdieri, C. Camardella, M. Carrozzino, and A. Frisoli. Designing multi-purpose devices to enhance users' perception of haptics. *Journal on Multimodal User Interfaces*, 16(3):335-342, 2022. 1

[17] O. Georgiou, W. Frier, E. Freeman, C. Pacchierotti, and T. Hoshi. Mid-air haptics: Future challenges and opportunities. *Ultrasound Mid-Air Haptics for Touchless Interfaces*, pages 385-397, 2022. 3

[18] O. Georgiou, W. Frier, E. Freeman, C. Pacchierotti, and T. Hoshi. *Ultrasound mid-air haptics for touchless interfaces*. Springer, 2022. 3

[19] O. Georgiou, W. Frier, and O. Schneider. User experience and mid-air haptics: Applications, methods, and challenges. *Ultrasound Mid-Air Haptics for Touchless Interfaces*, pages 21-69, 2022. 3

[20] D. Hajas, D. Pittera, A. Nasce, O. Georgiou, and M. Obrist. Mid-air haptic rendering of 2d geometric shapes with a dynamic tactile pointer. *IEEE Trans. haptics*, 13(4):806-817, 2020. 1

[21] S. Hirche and M. Buss. Human-oriented control for haptic teleoperation. *Proceedings of the IEEE*, 100(3):623-647, 2012. 1

[22] T. Hoshi, M. Takahashi, T. Iwamoto, and H. Shinoda. Noncontact tactile display based on radiation pressure of airborne ultrasound. *IEEE Trans. Haptics*, 3(3):155-165, 2010. 3

[23] T. Howard, M. Marchal, A. Lécuyer, and C. Pacchierotti. Pumah: pan-tilt ultrasound mid-air haptics for larger interaction workspace in virtual reality. *IEEE Trans. haptics*, 13(1):38-44, 2019. 1

[24] L. Jones. *Haptics*. MIT press, 2018. 1, 3

[25] A. M. Kappers, R. J. Holt, T. J. Junggeburth, M. F. S. Oen, B. J. van de Wetering, and M. A. Plaisier. Hands-free haptic navigation devices for actual walking. *IEEE Trans. Haptics*, 2024. 2

[26] A. M. Kappers, M. F. S. Oen, T. J. Junggeburth, and M. A. Plaisier. Hand-held haptic navigation devices for actual walking. *IEEE Trans. Haptics*, 15(4):655-666, 2022. 2

[27] Y. Kim, M. C. C. Silva, S. Anastasi, and N. Deshpande. Towards immersive bilateral teleoperation using encountered-type haptic interface. In *2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC)*, pages 1354-1359. IEEE, 2023. 4

[28] M. Malvezzi, F. Chinello, D. Prattichizzo, and C. Pacchierotti. Design of personalized wearable haptic interfaces to account for fingertip size and shape. *IEEE Trans. Haptics*, 14(2):266-272, 2021. 2

[29] M. Marchal and C. Pacchierotti. Virtual reality and haptics. In B. Siciliano, editor, *Robotics Goes MOOC*. Springer Nature, 2024. 2

[30] J. Martinez, D. Griffiths, V. Biscione, O. Georgiou, and T. Carter. Touchless haptic feedback for supernatural vr experiences. In *2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)*, pages 629-630. IEEE, 2018. 3

[31] M. Masmoudi, N. Zenati, S. Benbelkacem, Z. Hadjadj, O. Djekoune, M. A. Guerroudji, and Y. Izountar. Low-cost haptic glove for grasp precision improvement in virtual reality-based post-stroke hand rehabilitation. In *2021 International Conference on Artificial Intelligence for Cyber Security Systems and Privacy (AI-CSP)*, pages 1-3. IEEE, 2021. 4

[32] L. Meli, C. Pacchierotti, G. Salvietti, F. Chinello, M. Maisto, A. De Luca, and D. Prattichizzo. Combining wearable finger hap-
tics and augmented reality: User evaluation using an external camera and the microsoft hololens. *IEEE Robotics and Automation Letters*, 3(4):4297-4304, 2018. 1

[33] P. Milgram and F. Kishino. A taxonomy of mixed reality visual displays. *IEICE Trans. Information and Systems*, 77(12):1321-1329, 1994. 1

[34] A. Naceri, D. Mazzanti, J. Bimbo, Y. T. Tefera, D. Prattichizzo, D. G. Caldwell, L. S. Mattos, and N. Deshpande. The vicarios virtual reality interface for remote robotic teleoperation: Teleporting for intuitive tele-manipulation. *Journal of Intelligent & Robotic Systems*, 101:1-16, 2021. 4

[35] C. Pacchierotti. Cutaneous and kinesthetic cues to improve transparency in teleoperation. *Cutaneous haptic feedback in robotic teleoperation*, pages 93-120, 2015. 1

[36] C. Pacchierotti, F. Chinello, K. Koumaditis, M. Di Luca, E. Ofek, and O. Georgiou. Haptics in the metaverse: Haptic feedback for virtual, augmented, mixed, and extended realities. *IEEE Trans. Haptics*, 17(2):122-128, 2024. 2

[37] C. Pacchierotti and D. Prattichizzo. Cutaneous/tactile haptic feedback in robotic teleoperation: Motivation, survey, and perspectives. *IEEE Trans. Robotics*, 2023. 1, 2

[38] C. Pacchierotti, S. Sinclair, M. Solazzi, A. Frisoli, V. Hayward, and D. Prattichizzo. Wearable haptic systems for the fingertip and the hand: taxonomy, review, and perspectives. *IEEE Trans. haptics*, 10(4):580-600, 2017. 2, 4

[39] C. Pacchierotti, A. Tirmizi, G. Bianchini, and D. Prattichizzo. Enhancing the performance of passive teleoperation systems via cutaneous feedback. *IEEE Trans. haptics*, 8(4):397-409, 2015. 2

[40] C. Pacchierotti, A. Tirmizi, and D. Prattichizzo. Improving transparency in teleoperation by means of cutaneous tactile force feedback. *ACM Trans. Applied Perception (TAP)*, 11(1):1-16, 2014. 1

[41] J. Park, B. Son, I. Han, and W. Lee. Effect of cutaneous feedback on the perception of virtual object weight during manipulation. *Scientific Reports*, 10(1):1357, 2020. 3

[42] R. V. Patel, S. F. Atashzar, and M. Tavakoli. Haptic feedback and force-based teleoperation in surgical robotics. *Proceedings of the IEEE*, 110(7):1012-1027, 2022. 1

[43] D. Prattichizzo, C. Pacchierotti, and G. Rosati. Cutaneous force feedback as a sensory subtraction technique in haptics. *IEEE Trans. Haptics*, 5(4):289-300, 2012. 3

[44] M. Selvaggio, J. Cacace, C. Pacchierotti, F. Ruggiero, and P. R. Giordano. A shared-control teleoperation architecture for nonprehensile object transportation. *IEEE Trans. Robotics*, 38(1):569-583, 2021. 2

[45] S. Shimojo and L. Shams. Sensory modalities are not separate modalities: plasticity and interactions. *Current opinion in neurobiology*, 11(4):505-509, 2001. 3

[46] R. Skarbez, M. Smith, and M. C. Whitton. Revisiting milgram and kishino's reality-virtuality continuum. *Frontiers in Virtual Reality*, 2:647997, 2021. 1

[47] J. Szczerba, R. Hersberger, and R. Mathieu. A wearable vibrotactile display for automotive route guidance: evaluating usability, workload, performance and preference. In *Proceedings of the Human Factors and Ergonomics Society Annual Meeting*, volume 59, pages 1027-1031, 2015. 2

[48] D. Valkov and L. Linsen. Vibro-tactile feedback for real-world awareness in immersive virtual environments. In *2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)*, pages 340-349. IEEE, 2019. 4

[49] Y. Yokokohji, R. Hollis, and T. Kanade. What you can see is what you can feel-development of a visual/haptic interface to virtual environment. In *Proceedings of the IEEE 1996 Virtual Reality Annual International Symposium*, pages 46-53, 1996. 4

[50] G. Young, H. Milne, D. Griffiths, E. Padfield, R. Blenkinsopp, and O. Georgiou. Designing mid-air haptic gesture controlled user interfaces for cars. *Proceedings of the ACM on human-computer interaction*, 4(EICS):1-23, 2020. 3

[51] T. Zhang, Z. McCarthy, O. Jow, D. Lee, X. Chen, K. Goldberg, and P. Abbeel. Deep imitation learning for complex manipulation tasks from virtual reality teleoperation. In *2018 IEEE international conference on robotics and automation (ICRA)*, pages 5628-5635. IEEE,