# Tutorial in “Frontiers in Haptic Technology and Interaction Design: the Challenges, the Technology, the Perspectives”

Francesco Chinello, Claudio Pacchierotti, Hasti Seifi, Cheng Fang

<figure><img src="image_1.png" /></figure>

## To cite this version:

Francesco Chinello, Claudio Pacchierotti, Hasti Seifi, Cheng Fang. Tutorial in “Frontiers in Haptic Technology and Interaction Design: the Challenges, the Technology, the Perspectives”. NordiCHI 2022 - Nordic Human-Computer Interaction Conference, Oct 2022, Aarhus, Denmark. ACM, pp.1-3, 10.1145/3547522.3558898. hal-03807745

HAL Id: hal-03807745

https://inria.hal.science/hal-03807745

Submitted on 10 Oct 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
# Tutorial in "Frontiers in Haptic Technology and Interaction Design: the Challenges, the Technology, the Perspectives"

FRANCESCO CHINELLO, Aarhus University, Denmark

CLAUDIO PACCHIEROTTI, CNRS, Univ. Rennes, Inria, IRISA, France

CHENG FANG, University of Southern Denmark, Denmark

HASTI SEIFI, University of Copenhagen, Denmark

Technological advancement provides an increasing number and variety of solutions to interact with digital content. However, the complexity of the devices we use to interact with such content grows according to the users' needs as well as the complexity of the target interactions. This also includes all those tools designed to mediate touch interactions with virtual and/or remote environments, i.e., haptic interfaces and rendering techniques. We propose three hours of tutorials to discuss the technology, challenges, and perspective of haptic systems and rendering techniques for immersive human-computer interaction.

CCS Concepts: • **Human-centered computing** → **Haptic devices**.

Additional Key Words and Phrases: Human-Computer-Interaction, Haptics, Wearable Devices, Human-Robot-Interaction

## ACM Reference Format:

Francesco Chinello, Claudio Pacchierotti, Cheng Fang, and Hasti Seifi. 2018. Tutorial in "Frontiers in Haptic Technology and Interaction Design: the Challenges, the Technology, the Perspectives". In . ACM, New York, NY, USA, 3 pages. https://doi.org/XXXXXXX.XXXXXXX

## 1 INTRODUCTION

We face an era in which digital contents and virtual reality applications are growing in terms of complexity and market distribution [21]. From computer screens to smartphones, from tablets to wearables, user interaction with virtual and augmented reality environments provides a rich and immersive experience to interact with digital contents. In this respect, touch actuation and sensing technologies are analogous to video cameras and display screens, which register moving images and feed them to our eyes, or microphones and speakers, which register sounds and play them to our ears, respectively [12, 19]. In the scenario of interaction with immersive environments, the sense of touch provided by haptic devices can influence the user experience, ergonomics, and learning ability. In this tutorial, where participation is physical, target audience will involve ICT/HCI professionals, students, and researchers interested in exploring more about haptics in computer and immersive interaction. (Link to the tutorial https://sites.google.com/view/xrr-nordichi22/home).

## 2 HAPTICS AND WEARABLE SOLUTIONS

In this scenario, haptic research contributes with novel and advanced solutions designed to meet interactivity requirements and the simplicity of feedback interpretation for the interaction. Novel devices and engineering solutions involving wearable, portable, grounded, and mid-air haptics proposes several possible interaction directions [19]. The relevance of the contact interaction passes through the need to touch virtual objects to enhance the interaction dynamics

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org.

© 2018 Association for Computing Machinery.

Manuscript submitted to ACM
to the necessity of providing better control of specific interfaces. This can involve scenarios where haptic feedback, for example, plays a crucial role in assembly [10], for the simulation and training of stitching procedures [8], palpation [11], various dental procedures [23, 24], as well as endoscopy [5], laparoscopy [20], and biopsy [15] procedures.

The learning goal of this part of the tutorial deals with providing an overview of haptic technologies and rendering techniques for interaction with virtual and distant environments, specifically focusing on wearable and tactile solutions. The audience will be introduced to the methods, technologies and challenges for providing haptic feedback to the users.

## 3 HUMAN-COMPUTER INTERACTION DESIGN AND NEW HAPTIC TECHNOLOGIES

Haptic feedback is essential to guarantee a comprehensive experience using digital interfaces. This interests immersive virtual reality users, whose interaction with the virtual environment is counterbalanced by the presence of a wearable headset that limits the external world view. In this part of the tutorial, the presentation and the demonstration will guide the audience toward the latest research results connecting human-computer interaction design and the sense of touch. Furthermore, this session will describe various stimuli provided by robotic and cutaneous technologies. Finally, a demonstration of possible applications related to the possibility, for the users, of bridging the gap between Virtual Reality and haptic stimuli will be presented [13]. The learning goal in this part is to provide a picture of how common graphical environments, including, for example, immersive virtual reality scenarios, can be enriched by the haptic interaction derived from cutaneous and kinesthetic feedback.

## 4 THE ROLE OF HAPTICS IN VIRTUAL AND PHYSICAL INTERACTION IN INDUSTRY

Regardless of the variety of studies and solutions that enable haptic feedback, the touch sense also plays a crucial role in robotics. Several works are oriented to improve training performance in robotic surgical scenarios [6] and, in general, robotic teleoperation. Indeed, research shows that haptic feedback plays an important role in improving robotic teleoperation at large. It has been proven to enhance operators' performance in a large range of robotic applications, including micro-robotics [4, 18], needle and catheter insertion [1, 16, 22], surgical knot tying [3, 14], waste management [2]. The benefits of haptic feedback in such scenarios include increased manipulation and perception accuracy, decreased completion time, and decreased the peak and mean force applied to the remote environment. The inclusion of haptic feedback is also usually supported by the majority of users, both experts and novices [9, 17]. A growing interest is also provided by the possibility of using industrial robots arms in ergonomic tasks connected to human manufacturing operations [7]. It is the case, for example, of the possibility to control a set of robots to provide force feedback to the users' arms. With that said, it is essential to create a scenario where the users can integrate the beneficial aspects connecting the force feedback in robotic tasks for the industry.

In this part, the audience is guided towards a preliminary explanation on how haptics can be used in research studies involving safety and ergonomics, with special attention to human-robot interaction applications.

## 5 CONCLUSION AND EXPECTED OUTCOME

Open to all attendees, connected to industry, academia, and enthusiasts, the proposed tutorial focuses on providing the audience with current technological challenges and research methods in haptics. Another expected outcome is to provide an initial picture of the existing solutions and limitations of haptic technology. The tutorial illustrates these aspects with presentations involving the latest results on haptic solutions, human-computer interaction and haptics. In addition, there will be space dedicated to touch and force feedback in industrial scenarios, the latter involving robotics.
During the morning, interactive demonstrations will be presented to the audience, whose physical participation is crucial for the realization of the tutorial.

## REFERENCES

[1] M. Abayazid, C. Pacchierotti, P. Moreira, R. Alterovitz, D. Prattichizzo, and S. Misra. 2016. Experimental evaluation of co-manipulated ultrasound-guided flexible needle steering. *The International Journal of Medical Robotics and Computer Assisted Surgery* 12, 2 (2016), 219-230.

[2] Firas Abi-Farraj, Claudio Pacchierotti, Oleg Arenz, Gerhard Neumann, and Paolo Robuffo Giordano. 2019. A Haptic Shared-Control Architecture for Guided Multi-Target Robotic Grasping. *IEEE Trans. Haptics* (2019).

[3] Brian T Bethea, Allison M Okamura, Masaya Kitagawa, Torin P Fitton, Stephen M Cattaneo, Vincent L Gott, William A Baumgartner, and David D Yuh. 2004. Application of haptic feedback to robotic surgery. *Journal of Laparoendoscopic & Advanced Surgical Techniques* 14, 3 (2004), 191-195.

[4] Aude Bolopion and Stéphane Régnier. 2013. A review of haptic feedback teleoperation systems for micromanipulation and microassembly. *IEEE Trans. Automation Science and Engineering* 10, 3 (2013), 496-502.

[5] Sébastien Delorme, Denis Laroche, Robert DiRaddo, and Rolando F Del Maestro. 2012. NeuroTouch: a physics-based virtual simulator for cranial microneurosurgery training. *Operative Neurosurgery* 71, suppl_1 (2012), ons32-ons42.

[6] Marzieh Ershad, Robert Rege, and Ann Majewicz Fey. 2021. Adaptive Surgical Robotic Training Using Real-Time Stylistic Behavior Feedback Through Haptic Cues. *IEEE Transactions on Medical Robotics and Bionics* 3, 4 (2021), 959-969.

[7] Cheng Fang, Navvab Kashiri, Giuseppe F. Rigano, Arash Ajoudani, and Nikos G. Tsagarakis. 2019. Exploitation of Environment Support Contacts for Manipulation Effort Reduction of a Robot Arm. In *2019 International Conference on Robotics and Automation (ICRA)*. 9502-9508.

[8] Shiyu Jia and Zhenkuan Pan. 2010. A preliminary study of suture simulation in virtual surgery. In *Proc.International Conference on Audio, Language and Image Processing*. 1340-1345.

[9] Jacqueline K Koehn and Katherine J Kuchenbecker. 2015. Surgeons and non-surgeons prefer haptic feedback of instrument vibrations during robotic surgery. *Surgical endoscopy* 29, 10 (2015), 2970-2983.

[10] Konstantinos Koumaditis, Francesco Chinello, Panagiotis Mitkidis, and Simon Karg. 2020. Effectiveness of Virtual Versus Physical Training: The Case of Assembly Tasks, Trainer's Verbal Assistance, and Task Complexity. *IEEE Computer Graphics and Applications* 40, 5 (2020), 41-56.

[11] Min Li, Angela Faragasso, Jelizavata Konstantinova, Vahid Aminzadeh, Lakmal D Seneviratne, Prokar Dasgupta, and Kaspar Althoefer. 2014. A novel tumor localization method using haptic palpation based on soft tissue probing data. In *2014 IEEE International Conference on Robotics and Automation (ICRA)*. 4188-4193.

[12] Maud Marchal and Claudio Pacchierotti. 2022. *Haptics and Virtual Reality (VR)*. Springer International Publishing.

[13] Martin Maunsbach, Kasper Hornbæk, and Hasti Seifi. 2022. Whole-Hand Haptics for Mid-air Buttons. In *International Conference on Human Haptic Sensing and Touch Enabled Computer Applications*. Springer, 292-300.

[14] L. Moody, C. Baber, and T. N. Arvanitis. 2002. Objective surgical performance evaluation based on haptic feedback. *Studies in health technology and informatics* (2002), 304-310.

[15] Dong Ni, Wing Yin Chan, Jing Qin, Yim-Pan Chui, Ingrid Qu, Simon SM Ho, and Pheng-Ann Heng. 2009. A virtual reality simulator for ultrasound-guided biopsy training. *IEEE computer graphics and applications* 31, 2 (2009), 36-48.

[16] C. Pacchierotti, M. Abayazid, S. Misra, and D. Prattichizzo. 2014. Teleoperation of steerable flexible needles by combining kinesthetic and vibratory feedback. *IEEE Trans. Haptics* 7, 4 (2014), 551-556.

[17] C. Pacchierotti, D. Prattichizzo, and K. J. Kuchenbecker. 2016. Cutaneous feedback of fingertip deformation and vibration for palpation in robotic surgery. *IEEE Transactions on Biomedical Engineering* 63 (2016), 278-287.

[18] C. Pacchierotti, S. Scheggi, D. Prattichizzo, and S. Misra. 2016. Haptic feedback for microrobotics applications: a review. *Frontiers in Robotics and AI* 3, 53 (2016).

[19] Claudio Pacchierotti, Stephen Sinclair, Massimiliano Solazzi, Antonio Frisoli, Vincent Hayward, and Domenico Prattichizzo. 2017. Wearable haptic systems for the fingertip and the hand: taxonomy, review, and perspectives. *IEEE Transactions on Haptics* 10, 4 (2017), 580-600.

[20] Chung Hyuk Park, Kenneth L Wilson, and Ayanna M Howard. 2013. Examining the learning effects of a low-cost haptic-based virtual reality simulator on laparoscopic cholecystectomy. In *Proc. IEEE international symposium on computer-based medical systems*. 233-238.

[21] Unnikrishnan Radhakrishnan, Konstantinos Koumaditis, and Francesco Chinello. 2021. A systematic review of immersive virtual reality for industrial skills training. *Behaviour & Information Technology* 40, 12 (2021), 1310-1339.

[22] S. E. Salcudean, S. Ku, and G. Bell. 1997. Performance measurement in scaled teleoperation for microsurgery. In *Proc. First Joint Conference on Computer Vision, Virtual Reality and Robotics in Medicine and Medial Robotics and Computer-Assisted Surgery*. 789-798.

[23] Brian Tse, William Harwin, Alastair Barrow, Barry Quinn, Margaret Cox, et al. 2010. Design and development of a haptic dental training system-haptel. In *International conference on human haptic sensing and touch enabled computer applications*. 101-108.

[24] Dangxiao Wang, Yuru Zhang, Jianxia Hou, Yong Wang, Peijun Lv, Yonggang Chen, and Hui Zhao. 2011. iDental: a haptic-based dental simulator and its preliminary user evaluation. *IEEE Trans. Haptics* 5, 4 (2011), 332-343.