# A collaborative learning plateform combining co-embodiement and kinesthetic co-manipulation for technical Skill Acquisition in Virtual Reality

Flavien Lebrun, Ludovic Saint-Bauzel, Amine Chellali

► To cite this version:

Flavien Lebrun, Ludovic Saint-Bauzel, Amine Chellali. A collaborative learning plateform combining co-embodiement and kinesthetic co-manipulation for technical Skill Acquisition in Virtual Reality. IHM'25 - 36e Conférence Internationale Francophone sur l'Interaction Humain-Machine, Nov 2025, Toulouse, France. hal-05312171

HAL Id: hal-05312171

https://hal.science/hal-05312171v1

Submitted on 13 Oct 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# A collaborative learning plateform combining co-embodiement and kinesthetic co-manipulation for technical Skill Acquisition in Virtual Reality
## Une plateforme d'apprentissage collaboratif combinant le co-embodiment et la co-manipulation kinesthésique pour l'acquisition de compétences techniques en réalité virtuelle.

FLAVIEN LEBRUN, IBISC Lab, Université Evry Paris-Saclay, France

LUDOVIC SAINT-BAUZEL, ISIR Lab, Université Sorbonne, France

AMINE CHELLALI, IBISC Lab, Université Evry Paris-Saclay, France

<figure><img src="image_2.png" /><figcaption>Fig. 1. Illustration of the learning platform. Here, an expert guides a learner in virtual reality by monitoring their progress on a screen.</figcaption></figure>

We propose a demonstration of a virtual reality platform for learning technical gestures. With this platform, an expert can guide a learner during their training. The guidance is visual through the co-embodiment of a single virtual avatar shared between the two users and also a kinesthetic co-manipulation link thanks to two haptic interfaces. The system will be used to explore the impact of the connection between the learner and the teacher on the long-term retention of a technical skill.

Nous proposons la démonstration d'une plateforme d'apprentissage de gestes techniques en réalité virtuelle. Avec cette plateforme, un expert peut guider un apprenant dans lors de son entrainement. Le guidage est visuel au travers de la co-incarnation d'un seul avatar virtuel partagé entre les deux utilisateurs et également kinesthésique grâce à deux interfaces à retour d'effort. Le dispositif servira à explorer l'impact du lien entre apprenant et expert pour la rétention à long terme d'un geste technique.

Authors' Contact Information: Flavien Lebrun, IBISC Lab, Université Evry Paris-Saclay, Evry, France, flavien.lebrun@univ-evry.fr; Ludovic Saint-Bauzel, ISIR Lab, Université Sorbonne, Paris, France, saintbauzel@isir.upmc.fr; Amine Chellali, IBISC Lab, Université Evry Paris-Saclay, Evry, France, amine.chellali@univ-evry.fr.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org.

© 2018 Copyright held by the owner/author(s). Publication rights licensed to ACM.

Manuscript submitted to ACM
53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

Additional Key Words and Phrases: Virtual Reality, Technical Skill Learning, Collaborative Learning, Co-embodiment, Co-Manipulation

Mots Clés et Phrases Supplémentaires: Réalité Virtuelle, Apprentissage de Gestes Techniques, Apprentissage collaboratif, Co-incarnation, Co-manipulation.

**ACM Reference Format:**

Flavien Lebrun, Ludovic Saint-Bauzel, and Amine Chellali. 2018. A collaborative learning plateform combining co-embodiement and kinesthetic co-manipulation for technical Skill Acquisition in Virtual Reality. In *Proceedings of Make sure to enter the correct conference title from your rights confirmation email (Conference acronym 'XX')*. ACM, New York, NY, USA, 4 pages. https://doi.org/XXXXXXX.XXXXXXX

# 1 Context

Traditionally, learners are trained based on the mentorship model when learning a technical gesture: the expert ensures the transfer of skills to the novice through collaborative actions and demonstrations. During the initial stages of learning, the expert's instructions and corrections contribute to improving learning [1]. To avoid risky situations during training (especially in the medical field), the utility of immersive virtual environments (IVE) has been demonstrated [4]. However, in current IVEs, learners often work independently and thus lack expert guidance and feedback. To stay within the mentorship model, it is necessary to use Collaborative Virtual Environments (CVE) where experts can also be present in the IVE.

# 2 Challenges

Within IVE, partners can communicate using several modalities [15]. Studies show that guidance using a combination of visual and haptic modalities is best suited to improve learners' performance [14] and reduce mental load [8]. However, the effect of VR and multimodal approaches on learning, particularly for acquiring technical skills, requires further research to understand better the underlying skill transfer mechanisms [10]. Notably, overly frequent corrections or obvious guidance can hinder learning and, above all, long-term retention of the technical skill [12] (this difference between expected and actual results is now known as the Guidance Hypothesis). The central nervous system (CNS) relies on the coding and storage of control loops between proprioceptive and visual inputs and motor outputs to perform dynamic tasks. To update this control loop and master the required skill, a strong sense of agency (SoA) toward the movement is required. SoA is the subjective feeling of initiating and controlling an action, which arises when the predicted outcome is perceived as consistent with the actual outcome. In the presence of guidance, the SoA is weak; the CNS cannot code and retain the loop as it would during active practice [13]. Moreover, the learner will make fewer errors by being constrained to an optimal trajectory, but these errors will stimulate learning [16]. Finding a guidance approach that accelerates learning while preserving long-term skill retention is necessary [2].

# 3 Approach and Originality

Going beyond common interactions in CVEs, researchers have recently explored the concept of virtual co-embodiment [5], where two users can be immersed in a virtual environment in first-person view and can have complementary levels of control (total, partial, or none) over a shared avatar (see Figure 2). With this method, users, despite a good understanding of their level of control, overestimate their sense of agency (SoA). It has been shown that learning while accompanied by an expert using co-embodiment improves the efficiency of learning technical skills and increases their short-term retention compared to learning alone or passively following an expert's gestures [7]. We propose
<figure><img src="image_3.png" /><figcaption>Fig. 2. Illustration of the co-embodiment technique where the avatar's movements are a combination of 25% of User A's movements and 75% of User B's movements.</figcaption></figure>

to use co-embodiment and explore the benefits of this technique for the long-term retention of technical skills. Co-embodiment is a **visual** interaction technique between the expert and the learner. We have seen that multiplying interaction modalities is beneficial for learning and is also helpful in reinforcing the SoA. A recently defended hypothesis is that if sensory cues are reliable, SoA can emerge even for unintended actions [9]. We, therefore, want to extend the co-embodiment paradigm by adding a kinesthetic channel between the expert and the learner. In fact, haptically coupled human-human dyads make faster group decisions and can implicitly share confidence [11]. It also allows better learning and performance in tracking tasks, even in conflicts [6].

## 4 Objectives

The first objective of this research is to investigate the benefit of co-embodiment for long-term retention of technical skills. We also want to compare the effectiveness of skill transfer between an expert and a novice using the co-embodiment alone, a kinesthetic link alone, and a combination of these two methods, especially the effect on long-term retention. This project's broader objective is to understand better the links between the agency and the joint agency in the long-term retention of technical skills. A strong SoA is often associated with good skill retention; however, a characterization of the agency role in skill learning is still lacking [3]. Moreover, within the mentorship model, we will also investigate the effect of joint agency on learning.

## 5 Learning Platform

To achieve these goals, we have designed a learning platform. It consists of two haptic interfaces (Geomatic Touch or Omega) and at least one virtual reality headset. The expert and the learner are kinesthetically linked using the haptic interfaces. They are also co-embodied in the same virtual avatar. The learner and the expert perform the technical gesture at the same time. The movements of the virtual avatar are a weighted sum of the movements of the two users. Additionally, a force similar to that of a spring is exerted using the haptic interfaces. The intensity of this force is calculated based on the distance between the two users' hands. Using this platform, the expert can guide the learner in the learning of a complex trajectory, for example. The Kinesthetic force and the co-embodiment weight can be adjusted to change the nature of the link between the two users.

## 6 acknowledgment

This work is supported by two grants from the French National Research Agency (ANR): the France 2030 program (PEPR eNSEMBLE / CATS, ANR-22-EXEN-0002) and the Show-me project (ANR-20-CE33-0010).
References

[1] Michael J Barrington, Daniel M Wong, Ben Slater, Jason J Ivanusic, and Matthew Ovens. 2012. Ultrasound-guided regional anesthesia: how much practice do novices require before achieving competency in ultrasound needle visualization using a cadaver model. *Regional Anesthesia & Pain Medicine* 37, 3 (2012), 334–339. doi:10.1097/AAP.0b013e3182475fba

[2] Niclas Braun, Stefan Debener, Nadine Spychala, Edith Bongartz, Peter Sörös, Helge HO Müller, and Alexandra Philipsen. 2018. The senses of agency and ownership: a review. *Frontiers in psychology* 9 (2018), 535.

[3] Valentina Cesari, Sveva D'Aversa, Andrea Piarulli, Franca Melfi, Angelo Gemignani, and Danilo Menicucci. 2024. Sense of agency and skills learning in virtual-mediated environment: a systematic review. *Brain Sciences* 14, 4 (2024), 350.

[4] Amine Chellali, Woojin Ahn, Ganesh Sankaranarayanan, JT Flinn, Steven D Schwaitzberg, Daniel B Jones, Suvranu De, and Caroline GL Cao. 2015. Preliminary evaluation of the pattern cutting and the ligating loop virtual laparoscopic trainers. *Surgical endoscopy* 29 (2015), 815–821.

[5] Rebecca Fribourg, Nami Ogawa, Ludovic Hoyet, Ferran Argelaguet, Takuji Narumi, Michitaka Hirose, and Anatole Lécuyer. 2020. Virtual co-embodiment: evaluation of the sense of agency while sharing the control of a virtual body among two individuals. *IEEE Transactions on Visualization and Computer graphics* 27, 10 (2020), 4023–4038.

[6] Gowrishankar Ganesh, Atsushi Takagi, Rieko Osu, Toshinori Yoshioka, Mitsuo Kawato, and Etienne Burdet. 2014. Two is better than one: Physical interactions improve motor performance in humans. *Scientific reports* 4, 1 (2014), 3824.

[7] Daiki Kodama, Takato Mizuho, Yuji Hatada, Takuji Narumi, and Michitaka Hirose. 2023. Effects of collaborative training using virtual co-embodiment on motor skill learning. *IEEE Transactions on Visualization and Computer Graphics* 29, 5 (2023), 2304–2314.

[8] Flavien Lebrun, Cassandre Simon, Assia Boukezzi, Samir Otmane, and Amine Chellali. 2025. Mentor-Guided Learning in Immersive Virtual Environments: The Impact of Visual and Haptic Feedback on Skill Acquisition. In *32nd IEEE Conference On Virtual Reality And 3d User Interfaces (VR 2024)*.

[9] Roberto Legaspi and Taro Toyoizumi. 2019. A Bayesian psychophysics model of sense of agency. *Nature communications* 10, 1 (2019), 4250.

[10] Daniel Martin, Sandra Malpica, Diego Gutierrez, Belen Masia, and Ana Serrano. 2022. Multimodality in VR: A survey. *ACM Computing Surveys (CSUR)* 54, 10s (2022), 1–36.

[11] Giovanni Pezzulo, Lucas Roche, and Ludovic Saint-Bauzel. 2021. Haptic communication optimises joint decisions and affords implicit confidence sharing. *Scientific Reports* 11, 1 (2021), 1051.

[12] Richard A Schmidt and Robert A Bjork. 1992. New conceptualizations of practice: Common principles in three paradigms suggest new concepts for training. *Psychological science* 3, 4 (1992), 207–218.

[13] Reza Shadmehr and Ferdinando A Mussa-Ivaldi. 1994. Adaptive representation of dynamics during learning of a motor task. *Journal of neuroscience* 14, 5 (1994), 3208–3224.

[14] Cassandre Simon, Manel Boukli-Hacene, Flavien Lebrun, Samir Otmane, and Amine Chellali. 2024. Impact of Multimodal Instructions for Tool Manipulation Skills on Performance and User Experience in an Immersive Environment. In *2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)*. IEEE, 670–680.

[15] Cassandre Simon, Manel Boukli-Hacene, Samir Otmane, and Amine Chellali. 2023. Study of communication modalities to support teaching tool manipulation skills in a shared immersive environment. *Computers & Graphics* 117 (2023), 31–41.

[16] Kurt A Thoroughman and Reza Shadmehr. 2000. Learning of action through adaptive combination of motor primitives. *Nature* 407, 6805 (2000), 742–747.

A Demonstration details

The demonstration setup will look like the Figure 1. During the demonstration, participants will take the place of the learner and attempt to perform the technical gesture while being guided by the expert. We will need a table of similar dimensions to the one presented in the Figure 1 and a power strip.

Received 20 February 2007; revised 12 March 2009; accepted 5 June 2009