# Mentor-Guided Learning in Immersive Virtual Environments: The Impact of Visual and Haptic Feedback on Skill Acquisition

Flavien Lebrun, Cassandre Simon, Assia Boukezzi, Samir Otmane, Amine Chellali

► To cite this version:

Flavien Lebrun, Cassandre Simon, Assia Boukezzi, Samir Otmane, Amine Chellali. Mentor-Guided Learning in Immersive Virtual Environments: The Impact of Visual and Haptic Feedback on Skill Acquisition. IEEE Transactions on Visualization and Computer Graphics, 2025, 31 (5), pp.3547–3557. 10.1109/TVCG.2025.3549547. hal-04956417

HAL Id: hal-04956417

https://hal.science/hal-04956417v1

Submitted on 13 Mar 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# Mentor-Guided Learning in Immersive Virtual Environments: The Impact of Visual and Haptic Feedback on Skill Acquisition

Flavien Lebrun ID, Cassandre Simon ID, Assia Boukezzi ID, Samir Otmane ID, Amine Chellali ID

<figure><img src="image_2.png" /><figcaption>Fig. 1: Illustration of the user study. During a bimanual task, the expert supervises the learner's performance on a personalized view. In case of an error, the expert provides corrections to the learner using real-time augmented feedback. The feedback can be visual: the expert's virtual hand appears in the learner's view to show the correct path to follow. The feedback can also be visual and haptic: the learner sees the expert's hand and feels a corrective force toward the correct path.</figcaption></figure>

**Abstract**—In the early stages of learning a technical skill, trainees require guidance from a mentor through augmented feedback to develop higher expertise. However, the impact of such feedback and the different modalities used to communicate it remain underexplored in immersive virtual environments (IVE). This paper presents a study in which 27 participants were divided into three groups to learn a tool manipulation trajectory in an IVE. Two experimental groups received guidance from an expert using visual and/or haptic augmented feedback, while the control group received no feedback. The results indicate that both experimental groups showed significantly greater improvement in tool trajectory performance than the control group from pre- to post-test, with no significant differences between them. Analysis of their learning curves revealed similar performance improvements in tool trajectory across trials, outperforming the control group. Additionally, the visual-haptic feedback condition was linked to lower task load in three out of six dimensions of the NASA-TLX and a higher perceived interdependence with the expert's actions. These findings suggest that augmented feedback from an expert enhances the learning of tool manipulation skills. Although adding haptic feedback did not lead to better learning outcomes compared to visual feedback alone, it did enhance the overall user experience. These results offer valuable insights for designing IVEs that support mentor-trainee interactions through augmented feedback.

**Index Terms**—Multimodal interactions, Augmented feedback, Mentorship, Remote collaboration, Immersive learning

## 1 INTRODUCTION

Traditionally, learners are trained in many skill acquisition tasks based on the mentorship model, where experts guide and accompany them. During early learning stages, expert's corrections contribute to improving skill acquisition [1]. This involves using the visual modality by demonstrating the movement, using the auditory modality with alerts

* Flavien Lebrun is with the IBISC Lab, Université Evry Paris-Saclay. E-mail: flavien.lebrun@univ-evry.fr.

* Cassandre Simon is with the IBISC Lab, Université Evry Paris-Saclay. E-mail: cassandre.simon@univ-evry.fr.

* Assia Boukezzi is with IBISC, Université Evry Paris-Saclay and the Algerian Higher National School of Computer Science. E-mail: ja_boukezzi@esi.dz.

* Samir Otmane is with the IBISC Lab, Université Evry Paris-Saclay. E-mail: samir.otmane@univ-evry.fr.

* Amine Chellali is with the IBISC Lab, Université Evry Paris-Saclay. E-mail: amine.chellali@univ-evry.fr.

Manuscript received xx xxx. 201x; accepted xx xxx. 201x. Date of Publication xx xxx. 201x; date of current version xx xxx. 201x. For information on obtaining reprints of this article, please send e-mail to: reprints@ieee.org. Digital Object Identifier: xx.xxxx/TVCG.201x.xxxxxxx

or verbal advice, or using the haptic modality by physically guiding the learner's hand [23]. This is referred to as augmented feedback to distinguish all the information augmented by the expert from that arising directly from the learner's perception of task completion [64].

In motor learning, feedback is movement-related information that is provided to the student before, during (concurrent), and after (terminal) an attempt to perform a task, enabling modifications for the next action [28, 41]. Its role is to reduce the gap between what is understood and what is expected, helping students to correct errors and improve their performance. It also directs students' attention toward relevant task details [73], and it keeps them motivated by reassuring them about the relevance of their movements.

These augmented feedback mechanisms have proven to be particularly useful, for instance, in the acquisition of medical skills [38]. However, training novices on actual patients is questionable from both ethical and patient safety perspectives [61, 77]. Indeed, involving novices, who may struggle to manage complex tasks, increases the risk of errors and injuries [77]. Moreover, patients may not always be fully informed, may feel pressured to consent, or may be unable to refuse due to sedation or vulnerability [77]. Finally, supervisors often face a conflict between teaching and maintaining service efficiency, which can compromise trainee's learning opportunities and patient safety [30].

To avoid risky situations, the value of immersive virtual environ-
ments (IVE) for training medical technical skills has been demonstrated [6,76]. However, in current IVE, the learners find themselves working independently, and thus deprived of experts' guidance and, notably, their augmented feedback. In this context, Collaborative Virtual Environments (CVE) offer valuable tools that enable learner and expert co-presence within a shared IVE, thus facilitating their interactions and enhancing skills transfer. Visual, auditory, and haptic modalities can be used to convey information in such spaces [21,59]. However, the impact of IVE and communication modalities on learning, particularly for developing technical skills under the supervision of an expert, remains an area that needs further exploration [40].

Augmented multimodal feedback [57], particularly the combination of visual and haptic feedback [19], has proven effective in real-world skill acquisition. However, communication in CVEs differs significantly from face-to-face interactions in the physical world [46,62]. For example, accurately replicating actions like visually demonstrating a movement or physically guiding a learner's hand can be challenging in immersive CVEs. This highlights the need for further research into the impact of these communication modalities within CVEs, particularly for acquiring technical skills.

The main contribution of this work is to investigate the effectiveness of augmented feedback in an immersive CVE for transferring technical skills from an expert to a learner. Particularly, we compare the absence of augmented feedback during learning to two augmented feedback forms, the visual modality, and the combination of visual and haptic modalities, and study their impact on learning outcomes and user experience. We focus only on the visual and haptic modalities because they have been shown to be optimal sources of sensory information integrated for motor learning [39,47]. On the other hand, the verbal modality is less relevant in motor control and can increase student's cognitive load while having limited impact on skill acquisition [58].

A key challenge to address is designing appropriate interactions that enable an expert to provide augmented feedback to a learner through visual and haptic modalities within a CVE. By better understanding the role of these feedback modalities, we aim to enhance the learning experience and improve the design of IVE that supports skills transfer from experts to learners.

The main findings of this study are: 1) training in VR with augmented feedback, whether visual or visual-haptic, enhances motor performance and learning compared to training without feedback; 2) Combining haptic and visual feedback modalities does not significantly improve participants' performance and learning compared to the visual modality alone; 3) Visual-haptic feedback significantly reduces cognitive load and enhanced perceived collaboration.

# 2 RELATED WORK

## 2.1 Augmented Feedback for Motor Learning

In motor learning, feedback is movement-related information that is provided to the student before, during (concurrent), and after (terminal) an attempt to perform a task, enabling modifications for the next action [41].

The optimal frequency, timing, and duration of augmented feedback remain unclear and are related to the complexity of the skill [41]. The more complex the skill is, the more time, practice, and feedback will be required to master it. Moreover, too much feedback can impact the retention of skills when students have to perform alone [48,63]. In this case, feedback may become incorporated into memories of the movements [53]. This phenomenon is known as the guidance hypothesis: permanent feedback during acquisition leads to a dependency on the feedback. With gradually diminishing concurrent feedback, one can overcome this issue [32].

### 2.1.1 Visual Augmented Feedback

The visual modality is the most natural modality to provide augmented feedback due to the critical role played by vision as a sensory modality [41]. Indeed, the specificity of the learning hypothesis posits that learning entails the integration of the most optimal sources of afferent information for executing a given task, thereby outperforming other sources of afferent information, such as proprioception [47]. Moreover,

visual feedback provides immediate information about the learners' performance [41]. The design aspects of visual feedback can be an abstract visualization (for example, a line or a point on a screen) [16] or a more natural representation of the user's movements. Abstract visualizations seem to be efficient since they can represent a key feature of a movement in an unambiguous way [13]. However, visual feedback designs are more effective when they allow for the simultaneous processing of visual and kinesthetic information relevant to movement generation [70]. Students can adjust their movements by visualizing their actions, progressively improving their technical skills. Overall, concurrent visual feedback has proven useful at least in the early stages of complex skill acquisition [57]. However, the effectiveness of visual communication is not uniform for everyone. As Conati and Maclaren [10] point out, individual differences exist in the ability to interpret and assimilate this visual information. To address these variations, research recommends combining visual communication with other forms of communication, particularly the verbal and kinesthetic modalities [15].

### 2.1.2 Multimodal Augmented Feedback

Concurrent auditory feedback has proven useful for motor learning [57]. It is indeed possible, for instance, to use an alarm to signal an error [66] or to represent characteristics of the movement with a sound [3]. However, audio cues are less relevant sensory information for movement completion and may interfere with information from other sensory modalities [58].

On the other hand, haptic guidance theoretically offers numerous advantages, such as its potential ability to promote motor plasticity and somatosensory stimulation, inducing brain plasticity [39]. Most of the time, this guidance is in the form of an applied force to correct the student's movement [75] or vibrotactile information to alarm about a wrong movement [31]. It has been shown that haptic augmented feedback can enhance motor learning of a complex task, reduce the perceived workload [57], and enhance cooperation [44]. For certain tasks, augmented haptic feedback can enable the learning of temporal and spatial aspects similar to visual feedback [49]. For the learning of a 3D movement, haptic feedback favors timing-related performance, whereas visual feedback enhances the learning of position and shape. However, it is the combination of modalities that produces the most effective learning [19]. Indeed, learning that involves multiple senses reinforces the multimodal representation of the task [55]. In particular, the combination of visuo-tactile feedback seems to be more effective than visuo-auditory feedback for improving learning performance [58] and reducing mental load [4]. In general, most studies point out the superiority of visuo-haptic feedback over unimodal feedback, particularly for the learning of movement timing [57] and for the retention of learned skills [41].

In summary, augmented sensory feedback helps students correct errors and improve performance in motor skill acquisition. Visual feedback is particularly effective due to the critical role of vision in learning, but its effectiveness varies among individuals. Combining visual feedback with other modalities, such as verbal and kinesthetic, can address these variations. Audio (verbal) feedback seems to be the less optimal modality for learning motor tasks than haptic and visual information. Thus, based on the learning hypothesis, we choose to limit our scope to visual and haptic feedback in this study. Haptic feedback offers advantages like promoting motor plasticity and reducing workload, and it can enhance cooperation. The combination of visual and haptic feedback is generally more effective than unimodal feedback, particularly for learning movement timing and retaining skills. Moreover, it has been shown that augmented feedback provided to a student by an expert instead of automated feedback was best for skill acquisition and favors retention of information [5]. This motivates us to include experts with learning CVE following the mentorship model.

## 2.2 Augmented Feedback in Virtual Environments

CVEs allow multiple users to interact together in the same virtual space [9]. Within these environments, partners can communicate using several modalities [21,59]. Thus, CVEs seem to be a relevant medium
for complex skill acquisition under the supervision of a mentor. However, they have certain limitations, particularly regarding the quality of communication between users [11]. For instance, accurately reproducing partners' facial expressions, body movements, or gaze direction can be challenging and impede effective communication and collaborative interactions. Additionally, partners may not share the same perspective and, consequently, may not be aware of the ongoing activities of their partners [8]. These limitations can hinder skill transmission in teaching interactions and should be considered when designing related systems.

Communication modalities in CVEs have been the subject of numerous studies. It has been shown that combining modalities can enhance communication and collaboration within a CVE [21, 58]. Several studies propose various communication modalities between an expert and a learner. Combining verbal and visual modalities enhances communication [72] and collaboration [24]. Studies also show that combining visual and haptic modalities improves performance [7] and increases the sense of presence [52]. Completing collaborative tasks is faster with verbal-visual and verbal-haptic combinations than with visual-haptic combinations [42]. Furthermore, using all three modalities (haptic, verbal, and visual) results in better performance in collaborative tasks than using one modality or a combination of two [42]. These studies suggest that each modality has its strengths and limitations, and multi-modal communication enhances collaboration and user experience in CVEs. Another study found that haptic instructions led to faster task completion times, while visual instructions resulted in greater accuracy. Verbal instructions increased the sense of copresence with the teacher [59]. A follow-up study found that combining haptic and visual instructions was best suited for enhancing learners' performance [58]. However, the effect of VR and multimodal approaches on learning, especially for acquiring technical skills, is still an area that needs further exploration [40].

In this paper, we explore this issue within the framework of augmented feedback. Research in CVEs has examined the impact of various forms of augmented feedback, including verbal cues [36], visual guidance like pointers [18], and haptic force feedback [69], on learning tasks. Similar to results in non-CVE settings, the inclusion of augmented feedback has consistently been shown to enhance participants' performance. Recently, Liu et al. [37] demonstrated the benefits of automated augmented visual and audio feedback for promoting the rehabilitation in VR of participants who have suffered a traumatic brain injury. Similarly, Shi et al. [56] investigated the impact of automated visual, haptic, and visuo-haptic feedback on rehabilitation in virtual reality. They show that augmented visual feedback significantly improved movement performance and decreased neural loading compared to visuo-haptic feedback. However, both studies focused on rehabilitation and did not consider the contribution of an expert and the impact of multimodal feedback on the learning of a technical skill.

In summary, in CVE, multiple users can interact and communicate using various modalities, making CVEs relevant for complex skill acquisition under mentor supervision. Combining modalities like verbal, visual, and haptic feedback enhances communication, collaboration, and performance in CVEs. Unimodal augmented feedback in CVEs has been shown to enhance participants' performance and rehabilitation outcomes. However, the impact of augmented feedback, and especially multimodal augmented feedback, has, to our knowledge, been little studied. We aim to address this issue in this study.

# 3 USER STUDY

## 3.1 Study Objectives

This study investigates the impact of augmented feedback provided by an instructor to a trainee during the learning of technical skills in an IVE. We assess the impact of the modalities used to provide the augmented feedback on the trainee's ability to replicate the movement alone and on user experience. The investigated forms of augmented feedback are the visual modality and the visual-haptic modality. We also compare these two mechanisms to a control group of trainees who received no feedback during the learning. The task involves replicating a reference tool trajectory as accurately as possible. The differences between the augmented feedback forms are assessed based on their impact on the

trainees' performance (accuracy and speed), mental workload, and a subjective evaluation of the quality of their collaboration with the instructor during training.

## 3.2 Hypotheses

Our general hypothesis is that the form of the augmented feedback will affect participants' performance, cognitive load, and subjective learning experience differently. More particularly, we expect that:

* **H1.** The augmented feedback (either uni or multimodal) received during the training improves the participants' learning curve in terms of performance measures compared to the control group.

* **H2.** The multimodal augmented feedback (visual-haptic) received during the training improves the participants' learning curve in terms of performance measures compared to the uni-modal (visual-only) augmented feedback.

* **H3.** The augmented feedback (either uni or multimodal) received during the training decreases the participants' perceived mental workload during the training phase compared to the control group.

* **H4.** The multimodal augmented feedback (visual-haptic) received during the training decreases the participants' perceived mental workload during the training phase compared to the uni-modal (visual-only) augmented feedback.

* **H5.** The multimodal augmented feedback (visual-haptic) received during the training improves the perceived quality of collaboration between the participants and the instructor during the training phase compared to the uni-modal (visual-only) augmented feedback.

* **H6.** The improvement in performance between the pre- and post-tests is higher for the augmented feedback (either uni or multimodal) groups compared to the control group.

* **H7.** The improvement in performance between the pre- and post-tests is higher for the multimodal augmented feedback (visual-haptic) group compared to the uni-modal (visual-only) augmented feedback group.

## 3.3 Participants

Twenty-seven participants took part in this study (8 women and 19 men), recruited among students, university staff, and external participants. No specific criteria for expertise or prior experience with VR were required. The mean age of the participants was $30.52 \pm 11.26$ years (min = 18, max = 62). Twenty-five of them were right-handed, and two were left-handed. All participants had normal or corrected-to-normal vision, with 10 of them wearing corrective glasses during the experiment. All participants had previous experience with VR headsets, of which nine were regular users (at least once a week). Fourteen had prior experience with haptic devices, mainly used during demonstrations or other experimental studies. The Research Ethics Committee of (CER) of Université Paris-Saclay validated the experimental protocol, and all participants signed an informed consent form before being included in the study.

## 3.4 Experimental Task

The experimental task is inspired by basic medical skills that require precise hand-eye coordination during instrument manipulation. Participants are tasked with using one hand to accurately guide an instrument along a predetermined reference path while the other hand manipulates a visualization tool (similar to an ultrasound probe). This setup is inspired by real-world scenarios such as ultrasound-guided regional anesthesia or biopsy procedures and can be generalized to other medical interventions. For instance, ultrasound-guided regional anesthesia consists of inserting a needle following a specific path (from a needle entry point to an injection position) on a 2D plane while avoiding some anatomical structures (eg. veins and arteries) [60]. These structures are visualized using an ultrasound probe manipulated with one hand, while inserting the needle using the other hand. Similarly, our experimental
<figure><img src="image_3.png" /><figcaption>Fig. 2: Experimental apparatus including a VR headset, a controller, two haptic devices, and an armrest</figcaption></figure>

task consists of following a reference path from start to finish using a drawing instrument while adjusting the visualization tool to display the optimal view of obstacles along the trajectory, ensuring they are avoided. To do so, the participant must frequently move the visualization tool to the left or right to find the optimal view that maximizes the size of each obstacle. Although multiple optimal trajectories may exist, we chose a single reference trajectory to simplify the evaluation of participants' performance. This approach has been used in previous studies on the learning of calligraphy and drawing [34], as well as on the acquisition of basic surgical skills [51].

Accuracy was the primary criterion for evaluating the appropriate completion of the task. Therefore, the participant was instructed to follow the reference path as accurately as possible while avoiding the obstacles. The participant was also instructed to perform the task as quickly as possible. The task is completed once the instrument reaches the final position.

## 3.5 Apparatus

The equipment (Fig. 2) includes a Meta Quest 2 HMD to display the immersive VE to the participant. It has a resolution of 1832 × 1920 pixels per eye, with a refresh rate of 120 Hz and a field of view of 90 degrees (horizontal and vertical). Additionally, two Geomagic Touch haptic devices, with a workspace of 160 mm x120 mm x70 mm were used. One was operated by the participant's right hand to manipulate the virtual drawing instrument and receive the haptic augmented feedback. On the other hand, the instructor's right hand controlled the other device to provide the augmented feedback. The terminal component of this interface is a stylus featuring a button that the instructor presses to activate the augmented feedback and release to stop it. In addition, the participant's left hand manipulated an upside-down Meta Quest controller to control the virtual visualization tool. The participants used the armrest to rest their left arm while manipulating the controller. In addition, the instructor used a display monitor (29 inches with a resolution of 2560x1080) to visualize a personalized view of the VE.

The VE was created using Unity 3D (version 2020.3.15f2) and the SteamVR plugin (version 1.17.3). It consists of a room with a table on which a display screen was positioned at 45° (Fig. 1). The virtual representations of both participant's hands were displayed during the experiment. The right hand held the virtual drawing instrument, while the left hand held a virtual ultrasound probe.

The virtual display screen showed the working space. The latter included the starting position displayed as a green sphere and the final position as a blue sphere. The obstacles to avoid were displayed as red ellipses whose size varies based on the position of the ultrasound probe. Finally, the path appeared gradually on the display screen as the participant had drawn it.

Depending on the experimental condition, the instructor's virtual hand holding a similar instrument appeared intermittently in the partici-

<figure><img src="image_4.png" /><figcaption>Fig. 3: Expert's view during the experiment. Participants did not see the reference trajectory (green) or error margins (blue). They only saw the expert's hand when visual feedback was provided.</figcaption></figure>

pant's view to display the visual augmented feedback. It was colored dark grey and semi-transparent to differentiate it from the participant's hand and was controlled by the instructor's haptic device (Fig. 1).

The personalized instructor's view (Fig. 3) was displayed during the training phase (c.f. section 3.7). It included the reference path with the error margins and the obstacles. He was also able to visualize his own hand and instrument, the participant's hand manipulating the instrument, and the current path drawn by the participant. This helped him supervise the task's progress and provide augmented feedback when necessary.

## 3.6 Experimental Design and Conditions

The experiment followed a mixed design with one between-subjects factor (Feedback modality) with three conditions: visual feedback, visual-haptic feedback, and the control condition, and one within-subjects factor Trial with a different number of trials depending on the experimental phase. Indeed, each participant performed 10 training trials, 2 pre-test trials (averaged), and 2 post-test trials (averaged). This resulted in a total of 378 recorded trials (14 trials x 27 participants).

Following this experimental design, the participants were randomly divided into three groups of nine participants, each assigned to one of the between-subjects factor conditions. In the control condition, the participants received no feedback during the training phase. Each of the other two groups received a different form of augmented feedback:

* **Visual feedback:**

While the participant was performing the task, the instructor followed the reference trajectory simultaneously while supervising the participant's progress on his personalized view. When the participant deviated from the reference path and exceeded the allowed error margin (Fig. 3), the instructor pressed the button on his haptic device, making his virtual hand visible to the participant. This allowed the participants to visually match their hand with the instructor's hand, helping them correct their movement and move closer to the reference trajectory. Once the participant was back again on the reference path, the instructor released the button, and his hand disappeared from the participant's view.

* **Visual and Haptic feedback:** The haptic augmented feedback required a master-slave connection between the two haptic devices. The instructor's device was run as a master to control the slave device manipulated by the participant. Hence, along with the visual feedback described above, the instructor's haptic device applied a force to the participant's haptic device, guiding the participant's hand to align with the instructor's virtual hand. This elastic force $F(F_x, F_y)$ illustrated here in 2 dimensions, is applied to both hands:

$$ \begin{bmatrix} F_x \\ F_y \end{bmatrix} = K \begin{bmatrix} x_p - x_e \\ y_p - y_e \end{bmatrix} + D \begin{bmatrix} \dot{x}_p - \dot{x}_e \\ \dot{y}_p - \dot{y}_e \end{bmatrix} \quad (1) $$
Where $x_p, y_p$ and $x_e, y_e$ are the positions of the participant's and expert's hands, respectively, K is the stiffness coefficient, and D is the damping coefficient. This corrective force remained active as long as the instructor held the button. Once the participant was back again on the reference path, the instructor released the button, and his hand disappeared from the participant's view, and the master-slave connection was deactivated.

## 3.7 Experimental Phases and Virtual Scenes

A series of virtual scenes were presented to the participants depending on the experimental phase. At the beginning of the experiment, participants were placed in a familiarization scene where they could test the manipulations of the haptic device and the controller. They particularly experienced friction forces when drawing the path on the virtual display and manipulating the virtual ultrasound probe to control the display of the obstacles.

Subsequently, the participants were placed in the main experimental scene. They were asked to put the instrument in the starting position. Once this position was reached, the instruction phase began. This phase aimed to demonstrate the reference trajectory to the participants before they were asked to replicate it. The instruction was provided based on prerecorded data using haptic and visual modalities, as suggested by previous studies [58]. For that purpose, the participant's haptic device moved automatically along the reference path while the virtual instrument and the participant's hand moved on the display screen according to this path. A transition screen appeared at the end of the instruction phase, informing the participants that they could proceed to the pre-test phase.

During the pre-test phase, the participants had to bring the instrument to the starting position and attempt to replicate the instructed trajectory on their own while avoiding the obstacles. To do so, they had to move the instrument to draw the path while adjusting the display of obstacles to avoid them. If the obstacle size was not maximized, participants risked crossing it without realizing it, as the contact area with the obstacle was at its maximum size. Participants had to reach the final position to complete the trial. At the end of each trial, a transition screen appeared, indicating the trial number, the time taken to complete the task, and a score measuring the quality of the trajectory.

After that, the participants moved to the training phase. Similarly, they reproduced the trajectory while minimizing each trial's time and trajectory score. However, the participants in the two experimental groups received augmented feedback from the instructor (according to their experimental condition), while the control group participants did not. After ten training trials, participants were asked to complete two trials for the post-test phase. Similar to the pre-test, these trials were performed without feedback from the instructor.

## 3.8 Experimental Procedure

The average experiment duration was 30 minutes per participant. The first step was introducing the study's objective to the participants and the equipment they would use with the help of a prerecorded video. Next, they were asked to read and sign a consent form to participate in the study and to complete a demographic questionnaire, after which participants were asked to put on the HMD, to begin the experiment. Once the pre-test and training trials were completed, participants were instructed to remove the HMD to fill in the NASA-TLX questionnaire and a questionnaire evaluating the quality of collaboration with the instructor. Once this step was completed, they were required to put on the HMD again to perform the two post-test trials.

## 3.9 Measurements

The objective measurements assessed the participants' ability to replicate the reference trajectory. They included three metrics: **the number of crossed obstacles**, **the quality of trajectory**, and **the completion time**. The number of crossed obstacles indicated the number of times the drawn path crossed an obstacle at its maximum size. Lower values indicate better performance. The trajectory quality was assessed using Dynamic Time Warping (DTW) [74], calculating the similarity score between the reference trajectories and those performed by the

participants. The lower the DTW distance, the more similar the two trajectories are. Finally, the manipulation time calculation for each trial started when the participants positioned the instrument in the starting position and ended when they placed it in the final position. Lower values indicate better performance.

The subjective measurements consisted of participants' responses to two questionnaires. These included the NASA-TLX [27] to measure the participants' mental workload after the training phase and quality of collaboration questionnaire assessing the sense of social presence, copresence, perceived comprehension, perceived behavioral interdependence, and perceived learning experience on a five-point Likert scale.

The questions were based on questionnaires used in peer-reviewed international publications and adapted for our study. The sense of social presence was measured using three questions from the social presence dimension of Nowak's and Biocca's questionnaire [43]. The sense of copresence was measured using six questions. Two were extracted from the copresence dimension of Nowak's and Biocca's questionnaire [43], and four from the questionnaire of Basogan et al. [2]. The perceived comprehension and the perceived behavioral interdependence questions were extracted from the Networked Minds Social Presence Measure [25]. Finally, the learning experience was measured using two questions from the questionnaire of Simon et al. [58].

## 3.10 Data Analyses

Statistical data analysis was performed using the SPSS software, with a confidence level of 95%. The Shapiro-Wilk tests were used to check the normal distributions of the data. Levene's test was used to assess the homogeneity of variances, while Mauchly's test was applied to assess the sphericity of the data. Based on these preliminary verifications, two-way mixed analyses of variance (ANOVA) were used. The Greenhouse-Geisser correction was used in case of non-compliance with the sphericity assumption. Pairwise comparisons with Bonferroni adjustments were used when necessary. Finally, for the subjective data collected, the non-parametric Mann-Whitney U pairwise tests were used to compare the mean scores of each dimension of the quality of collaboration questionnaire. A one-Way Between-Subjects ANOVA was used to compare the scores on each dimension and the overall NASA-TLX score.

# 4 RESULTS

## 4.1 Training Phase

A 10x3 mixed design with two factors was used for the training phase. The within-subjects factor was Trial with 10 levels: T1-T10, and the between-subjects factor was the feedback modality with 3 levels: Visual feedback (V-F), Visual-Haptic feedback (VH-F), and the control group with no feedback (control).

### 4.1.1 Number of Crossed Obstacles

The results of the 10x3 mixed-design ANOVA (sphericity not assumed, Greenhouse-Geisser correction used) do not show a main effect of the Trial ($F_{(4.50,108.10)} = .29$, p=.89, partial $\eta^2=.01$) or an interaction effect between the Trial and the Feedback modality ($F_{(9.00,108.10)} = 1.60$, p=.12, partial $\eta^2=.12$) on the number of crossed obstacles (Fig. 4). On the other hand, the results show a main effect of the Feedback modality ($F_{(2,24)} = 5.73$, p=.009, partial $\eta^2=.32$).

The pairwise comparisons with the Bonferroni correction show that the participants in the control group crossed the obstacles significantly more frequently than the participants in the V-F group (adjusted-p=.04) and the VH-F group (adjusted-p=.01). No significant difference was observed between the V-F and the VH-F groups.

### 4.1.2 Quality of the Trajectory

The results of the 10x3 mixed-design ANOVA (sphericity not assumed, Greenhouse-Geisser correction used) show a main effect of the Trial ($F_{(2.56,61.52)}=3.43$, p=.02, partial $\eta^2=.12$) and a main effect of the Feedback modality ($F_{(2,24)} = 9.06$, p=.001, partial $\eta^2=.43$) on the DTW
<figure><img src="image_5.png" /><figcaption>Fig. 4: The number of crossed obstacles during the training phase</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>Fig. 5: Quality of the tool trajectories based on the DTW scores between the participants' trajectories and the reference trajectory during the training phase</figcaption></figure>

scores (Fig. 5). On the other hand, the results show no interaction effect between the Trial and the Feedback modality ($F_{(5,12,61,52)} = 1.65, p=.15$, partial $\eta^2=.12$).

The pairwise comparisons with Bonferroni correction show that the participants' trajectories were significantly closer to the reference trajectories in the V-F (adjusted-p=.005) and the VH-F (adjusted-p=.003) groups than in the control group. No significant difference was observed between the V-F and the VH-F groups. On the other hand, the pairwise comparisons with the Bonferroni correction show no significant difference between trials, indicating that the performance improvement over time was not significant.

### 4.1.3 Time to Complete the Task

The results of the 10x3 mixed-design ANOVA (sphericity not assumed, Greenhouse-Geisser correction used) show a main effect of the Trial ($F_{(3,40,81,68)}=13.55, p<.0001$, partial $\eta^2=.36$) and a main effect of the Feedback modality ($F_{(2,24)}=15.98, p<.0001$, partial $\eta^2=.57$) on the time to complete the task (Fig. 6). On the other hand, the results show only a marginal interaction effect between the Trial and the Feedback modality ($F_{(6,80,104,75)}=2.01, p=.06$, partial $\eta^2=.14$).

The pairwise comparisons with the Bonferroni correction show that the participants in the VH-F group performed the task significantly slower than in the control (adjusted-p<.0001) and the V-F (adjusted-p=.01) groups. No significant difference was observed between the V-F and the control groups. In addition, the pairwise comparisons with the Bonferroni correction show a significant difference between T1 and T4-T10, between T2 and T5-T10, and between T3 and T10 (all adjusted-p values were <.05), with the completion times decreasing over the trials. No other significant differences were observed.

### 4.1.4 Number of Expert Corrections

Because the expert did not provide feedback during the control group training, only the two feedback groups were compared.

The results of the 10x2 mixed-design ANOVA (sphericity not assumed, Greenhouse-Geisser correction used) show a main effect of the Trial ($F_{(3,56,56,96)}=13.55, p<.0001$, partial $\eta^2=.47$) on the number of the trajectory corrections provided by the expert (Fig. 7). On the other

<figure><img src="image_7.png" /><figcaption>Fig. 6: Time to complete the task during the training phase</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>Fig. 7: Number of trajectory corrections provided by the expert during the training phase</figcaption></figure>

hand, the results do not show a main effect of the Feedback modality ($F_{(1,16)}=.05, p=.81$, partial $\eta^2=.03$) or an interaction effect between the Trial and the Feedback modality ($F_{(3,56,56,96)}=1.52, p=.21$, partial $\eta^2=.08$).

The pairwise comparisons with the Bonferroni correction show a significant difference between T1 and T4-T10, T2 and T5, T9 and T10, T3 and T10, and T4 and T10, with the number of corrections decreasing over the trials. No other significant differences were observed.

### 4.1.5 Quality of Collaboration Questionnaire

The Mann-Whitney U tests for independent samples Fig. 8 show no significant effect of the feedback modality on the scores of four of the five Quality of Collaboration questionnaire dimensions. On the other hand, a significant effect of the feedback modality is observed on the "Perceived behavioral interdependence" dimension Fig. 8 (Mann-Whitney U=16.5, p=.03). The mean score for this dimension was significantly higher for the VH-F condition than for the V-F condition.

### 4.1.6 Workload

We compared both the raw NASA scores and the individual sub-scales, which is a common practice [26].

The mean raw NASA-TLX scores were lower in the visual-haptic feedback group (38.61±8.40) than those in the visual feedback group (50.83±15.84) and the control group (48.51±8.18), indicating a lower workload. The one-way between-subjects ANOVA suggests that the difference is marginal ($F_{(2,24)}=2.92, p=.07$, partial $\eta^2=.19$). The analyses by dimension (Fig. 9) show a significant effect of the Feedback modality on the Physical demand ($F_{(2,24)}=4.14, p=.02$, partial $\eta^2=.25$), the own Performance ($F_{(2,24)}=6.48, p=.006$, partial $\eta^2=.35$), and the Effort dimensions ($F_{(2,24)}=3.41, p=.04$, partial $\eta^2=.22$). Post hoc comparisons show that the participants in the visual-haptic feedback group reported significantly lower physical demand compared to participants in the visual feedback group (p=.008). They also reported significantly higher satisfaction with their own performance than participants in the visual feedback (p=.04) and the control (p=.002) groups. Finally, they reported lower levels of effort than participants in the visual feedback (p=.02) and the control (p=.04) groups. No other significant differences were observed.
<figure><img src="image_9.png" /><figcaption>Fig. 8: Boxplots with medians showing the scores (higher is better) for the subdimensions of the quality of collaboration questionnaire (scale from 1 to 7): Social Presence (SP), Copresence (CoP), Perceived Comprehension – (PC), Perceived Behavioral Interdependence (PBI), Perceived Learning Experience (PLE) (*=p < .05)</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>Fig. 9: Boxplots with medians showing the scores for the subdimensions of the NASA-TLX (scale from 0 to 100, lower is better): Mental demand, Physical demand, Temporal demand, Own Performance, Effort, and Frustration level (*=p < .05, *** =p < .001)</figcaption></figure>

## 4.2 Pre-post Tests

For the pre-post-test comparisons, a two-factor 2x3 mixed design was used. The within-subjects factor was Test with two levels: Pre-test and Post-test, and the between-subjects factor was Feedback modality with three levels: the visual-haptic feedback group (VH-F), the Visual feedback group (V-F), and the control group with no feedback (control). Analyses for each measure are presented in the following.

### 4.2.1 Number of Crossed Obstacles

The 2x3 two-way mixed ANOVA results show a main effect of Test $F_{(1,24)}=16.66, p<.0001$, partial $\eta^2=.41$), and a main effect of the feedback modality $F_{(2,24)}=3.84, p=.03$, partial $\eta^2=.24$) on the number of crossed obstacles. On the other hand, no interaction effect between the two variables $F_{(2,24)}=1.00, p=.38$, partial $\eta^2=.07$) was observed. The results show an improvement in performance in the post-test compared to the pre-test for all groups (Fig. 10).

In addition, pairwise comparisons with the Bonferroni correction show that the difference between the control and visual-haptic groups was marginal (adjusted-p=.05). No other significant differences were observed.

### 4.2.2 Quality of the Trajectory

The 2x3 two-way mixed ANOVA results show a main effect of Test $F_{(1,24)}=43.13, p<.0001$, partial $\eta^2=.64$), and an interaction effect between Test and Feedback modality variables $F_{(2,24)}=10.64, p<.0001$, partial $\eta^2=.47$) on the DTW scores (Fig. 10). On the other hand, no main effect of the Feedback modality $F_{(2,24)}=1.91, p=.16$, partial $\eta^2=.13$) was observed.

To identify where the interaction resides along the levels of the two independent variables, a series of follow-up ANOVAs was conducted (as suggested by [22]). Consequently, three 2x2 ANOVAs were conducted to test all the possible interactions between the three levels of the feedback modality and the two levels of the Test variables. Bonferroni adjustments were used to reduce the Type I error risks.

The results indicate that only two interactions were statistically significant. Thus, the effect of feedback modality over time occurred only between the control group and the two other groups. This suggests that the two groups who received expert feedback during training improved their trajectories from the pre-test to the post-test significantly more than the control group. In contrast, the results do not show evidence of a difference between the two expert feedback groups regarding performance improvement over time.

### 4.2.3 Completion Times

The 2x3 two-way mixed ANOVA results show a main effect of Test $F_{(1,24)}=11.05, p=.003$, partial $\eta^2=.31$), and a main effect of the feedback modality $F_{(2,24)}=7.98, p=.002$, partial $\eta^2=.40$) on the completion times. On the other hand, no interaction effect was observed between the two variables $F_{(2,24)}=1.24, p=.30$, partial $\eta^2=.09$). The results show that all the groups performed the task significantly faster during the post-test than the pre-test (Fig. 10).

In addition, pairwise comparisons with the Bonferroni correction show that the participants in the control group performed the task significantly faster than the visual-haptic feedback group (adjusted-p=.002). On the other hand, the difference between the visual feedback and the visual-haptic feedback groups was marginal (adjusted-p=.05).

# 5 DISCUSSION

## 5.1 Trainees Performances

### 5.1.1 Training Phase

Analyzing the training phase results can help understand the learning curve dynamics with and without the augmented feedback in the IVE. While none of the three groups significantly improved their performance in the quality of trajectory and the number of crossed obstacles during the training phase, participants who received feedback performed better than the control group throughout the session. These results partially validate hypothesis H1 (see Sec. 3.2) and corroborate existing studies that highlight the crucial role of augmented feedback in improving motor performance by allowing learners to correct their errors, optimize their movements, and know if the goal is achieved to adjust their strategy accordingly [35, 41]. The observed effect size for the main effect of feedback modality on trajectory quality ($\eta^2=.43$) underscores the practical significance of providing augmented feedback, which allowed participants to perform closer to the reference trajectory.

Since the feedback was provided from the beginning of the training phase, participants were able to adjust their movements immediately, retaining this improved performance even as expert corrections decreased significantly over time ($\eta^2=.47$ for the effect of trial on expert corrections). By the end of the training phase, the need for expert intervention was almost null, demonstrating the effectiveness of augmented feedback in promoting independent learning. In contrast, participants in the control group struggled to improve their performance, showing no notable changes in trajectory quality or the number of crossed obstacles even after ten training sessions.

On the other hand, participants in the visual-haptic group took significantly longer to complete the task compared to those in the Visual and Control groups ($\eta^2=.57$). This difference may be primarily related to an initial difference in completion times performance between the
<figure><img src="image_11.png" /><figcaption>Fig. 10: Participants' performance during the pre-tests and post-tests. (A) The two-way ANOVA shows a main effect of Test and a main effect of Feedback Modality on the number of crossed obstacles. (B) The two-way ANOVA shows a main effect of Test and an interaction effect between Test and Feedback Modality on the quality of the trajectory. The follow-up ANOVAs show that this interaction resides only between the control and the visual feedback groups, and between the control and the visual-haptic groups: the participants of the visual feedback and visual-haptic feedback groups performed significantly better than the participants of the control group during the post test. (C) The two-way ANOVA shows a main effect of Test and a main effect of Feedback Modality on the completion time (\*\* = p<.01, \*\*\* = p<.001).</figcaption></figure>

visual-haptic group and the two other groups. This is further confirmed by the analysis of the pre-test post-test results. Although randomization was used for group assignment, such differences can occasionally occur, as noted in prior methodological discussions [22].

Despite this initial disparity, all three groups exhibited a significant improvement in completion times over the trials ($\eta^2$=.36), suggesting a similar learning effect across groups. The marginal interaction effect ($\eta^2$=.14) indicates that the feedback modality did not substantially alter the rate of improvement. These findings suggest that improvements in completion times primarily reflect habituation to the task and apparatus, since accuracy was the primary evaluation criterion.

Interestingly, no significant difference was observed between the visual and the visual-haptic groups, contradicting hypothesis H2 (see Sec. 3.2). This non-significant difference may be related to the visuospatial nature of the task, where visual cues alone appeared sufficient to guide participants, rendering the addition of haptic feedback less impactful.

The large main effect of feedback modality on trajectory quality ($\eta^2$=.43) highlights the utility of visual feedback, while the additional haptic feedback provided in the visual-haptic group did not yield significant benefits. The visual system often dominates other modalities, such as haptics in spatial tasks [29]. This visual dominance is also confirmed by the fact that sensorimotor learning relies on continuous adaptation to visual cues, especially in spatial tasks where precision is crucial [33]. Participants seemed to primarily rely on visual obstacles and the drawn path to memorize and execute the trajectory, without gaining additional benefits from haptic feedback. Although haptic feedback can sometimes be useful, it does not necessarily provide significant advantages in tasks dominated by visual cues [14]. In addition, when visual cues are available and sufficient to guide the user, haptic feedback can sometimes slow down task execution [50]. The simultaneous integration of information from multiple sensory channels during motor control can lead to sensory conflicts when signals are incongruent or redundant [65]. This may also partly explain why the haptic feedback led to higher completion times.

In summary, for tasks requiring spatial precision, users tend to rely primarily on visual cues to guide their actions rather than haptic cues. Finally, motor learning relies heavily on repetition and self-correction [71]. This repetition of trials, combined with the augmented feedback, allowed participants to refine their motor schemas without constantly depending on expert feedback, explaining the gradual decrease in expert interventions, regardless of the type of feedback received.

### 5.1.2 Pre-test and Post-test Phases

The pre- and post-test results confirm the observations made during the learning phase. The significant interaction in the quality of trajectory ($\eta^2$=.47) indicates that the participants who received feedback (visual or visual-haptic) showed a significant improvement in the quality of

their trajectory, while the control group did not. This partially confirms H6 (see Sec. 3.2) and highlights the role of feedback in facilitating motor learning. The large effect size ($\eta^2$=.64) for the main effect of the test further underscores the significant improvement in participants' performance from the pre-test to the post-test. Feedback helped participants refine their trajectories more effectively, aligning with previous research showing that expert feedback improves motor performance by enabling error correction and strategy optimization [20, 35, 45].

In contrast, all groups reduced the number of crossed obstacles between the pre-test and post-test ($\eta^2$=.41), despite no specific feedback being provided on probe manipulation. This improvement is likely due to task habituation and increased confidence over time. Initially, the participants focused more on the trajectory, with less attention to avoiding obstacles. After some training and gaining more confidence in performing the task, they freed up more cognitive resources to control the probe and avoid obstacles, which were displayed and visible by all participants. Completion times also decreased significantly across all groups from the pre-test to the post-test ($\eta^2$=.31), reflecting habituation to the task and apparatus. However, participants in the visual-haptic group consistently completed the task slower than the control group ($\eta^2$=.40), with this difference persisting across both sessions. The absence of a significant interaction effect ($\eta^2$=.09) suggests that this disparity reflects an initial difference in performance rather than an effect of the experimental intervention. This finding supports the interpretation that accuracy was the primary focus of participants, with improvements in task time reflecting secondary gains from practice.

Finally, as observed during the training phase, no significant difference was found between the visual and visual-haptic groups, leading to the rejection of H7 (see Sec. 3.2). Once again, the visual nature of the task seems to reduce the influence of the haptic feedback. This highlights the efficacy of visual feedback, while combining it with haptic feedback did not contribute additional benefits. The effectiveness of haptic feedback can vary depending on the task, and in some cases, it might even hinder performance [14]. In our study, while haptic feedback did not impair the performance, it also did not enhance it, potentially due to redundancy with visual information. It would be interesting, in future studies, to investigate the impact of haptic feedback on tasks with fewer visual cues, for instance, with fewer obstacles and without tracing the path during the execution of the trajectory.

## 5.2 Subjective Measurements

The NASA-TLX results show that combining the visual and haptic modalities significantly reduces the perceived workload compared to visual feedback alone or the control group, validating H4 and partially validating H3 (see Sec. 3.2). Although the overall difference in raw scores was marginal ($\eta^2$=.19), the significant effects observed in specific dimensions—Physical Demand ($\eta^2$=.25), Performance ($\eta^2$=.35), and Effort ($\eta^2$=.22)—highlight the practical significance of multimodal
feedback. These results align with existing literature, which reports that integrating different modalities in collaborative learning environments can decrease cognitive load [17]. Moreover, combining multiple sensory modalities, such as visual and haptic, can alleviate mental load during the acquisition of motor skills [57].

In our study, the difference is mainly related to physical demand, performance, and effort dimensions. It seems that haptic feedback, in addition to visual aids, provides physical assistance that lightens the learner's load, giving them the impression of exerting less effort. This perceived reduction in effort appears to positively impact their perception of their own performance. These results are consistent with previous research on the effects of multimodal feedback on motor learning and cognitive load. Indeed, participants exposed to visual-haptic feedback have been shown to report significantly lower physical demand and perceived effort, as well as higher perceived performance [68].

Interestingly, no significant differences were observed in the Temporal Demand dimension, contrasting with the completion time results, where the visual-haptic feedback group was slower. This discrepancy may stem from an initial performance difference between groups, as discussed earlier. Since accuracy was the primary evaluation criterion, participants appeared to prioritize trajectory optimization over speed, which likely explains the lack of significant differences in temporal demand.

Overall, these findings reinforce the value of multimodal feedback in reducing workload and enhancing perceived performance, especially in tasks requiring precise motor control. Future research could explore whether these benefits extend to more complex or time-sensitive tasks, where cognitive and temporal demands may play a larger role.

The quality of the collaboration questionnaire results shows higher scores on the Perceived Behavioral Interdependence (PBI) dimension for the visual-haptic group than the visual group, partially validating H5 (see Sec. 3.2). Research shows that adding haptic feedback in learning environments can strengthen the sense of behavioral interdependence between participants. This is because haptic feedback creates a form of contact that enhances the perception of joint action and collaboration between individuals [67].

However, the acceptance and appreciation of the visual-haptic feedback vary from one individual to another, as shown by the high individual variations between the participants for the other dimensions of the questionnaire. This large variability is also observed in some dimensions of the NASA-TLX. These measurements are subjective and may suggest that the haptic feedback is perceived differently by different individuals. It has been shown that despite the advantages of haptic feedback for learning, all learners do not perceive haptic systems uniformly [12]. Indeed, some of them find these haptic systems disruptive or difficult to use due to personal preference or learning style. In addition, some participants may have difficulty adapting to this modality, especially when it is added to other sensory stimuli [54]. In addition, in a recent study, it has been [59] found that the haptic modality was the most disturbing when interacting with an instructor in an IVE. These three studies highlight the need to adapt haptic devices to avoid overwhelming the student with sensory overload while optimizing the learning experience based on the needs and preferences of each individual. Therefore, it would be interesting in future studies to include personalized approaches that would allow students to control the level of haptic feedback or choose learning modalities that best suit their learning style.

# 6 LIMITATIONS AND FUTURE WORKS

Our study has some limitations. Firstly, the experimental task was highly dependent on visual cues. We introduced visual obstacles to increase task complexity, mimicking medical procedures that require bimanual dexterity and hand-eye coordination. Additionally, the trajectory was shown to participants during the instruction phase, offering extra visual guidance. We did not foresee that this would enable participants to memorize the path more quickly, as indicated by the learning curves, which show rapid improvement in trajectory accuracy, with less need for expert intervention after just five trials. Although these visual

aids alone were insufficient (as seen in the control group, which did not improve solely with them), they may have reduced the opportunity to fully assess the benefits of combining visual and haptic feedback. Future studies in scenarios with fewer visual cues would be valuable in understanding the distinct role of haptic feedback in motor learning.

In addition, no augmented feedback was provided for correcting the probe manipulation, which led to less improvement in obstacle avoidance during the training phase. Participants seem to have focused more on the augmented feedback provided by the expert and improving their trajectory rather than obstacle avoidance. However, once they practiced independently (post-test), their performance in avoiding obstacles improved (compared to pre-test). This raises questions about the effectiveness of using multiple augmented feedback sources for parallel tasks. Future research could explore whether providing simultaneous augmented feedback through different modalities tailored to the nature of the task could enhance learning outcomes. However, the chosen task is inspired by a medical ultrasound probe manipulation task. The results found here are, therefore likely to be generalizable to other medical training tasks and, by extension, to all technical skill training tasks in VR. One interesting question for future work is the generalization of these results into augmented reality (AR) applications. In AR, the mentor can be visible, which can reinforce the sense of copresence. However, the interactions and especially the feedback will be similar to those in a real case, thus limiting the possibilities of exploring new interaction techniques. It would be interesting to investigate whether haptic feedback in AR also provides little improvement in motor performance.

Moreover, the interindividual variability in the perception and acceptance of haptic feedback highlights the need to personalize these approaches to adapt to each learner's preferences. Thus, exploring personalized strategies that allow learners to regulate the level of feedback could help overcome individual variations in the acceptance and effectiveness of feedback. Conducting a study examining the best way to combine visual and haptic modalities to minimize cognitive overload while maximizing learning outcomes would be particularly relevant.

Finally, it would be crucial to explore how these modalities affect not only learners but also instructors, by evaluating how technology can facilitate the effective transmission of skills. These studies will help us develop more effective user interfaces to support the mentorship model and facilitate the transfer of motor skills from the instructor to the learner within IVE dedicated to learning.

# 7 CONCLUSION

This study evaluated the impact of visual and visual-haptic feedback on the transfer of motor information during tool manipulation in a bimanual task within a collaborative virtual environment. The findings highlight the effectiveness of augmented feedback in enhancing motor performance and learning.

Visual feedback proved sufficient for tasks with dominant spatial components, underscoring its critical role in motor learning. While visual-haptic feedback did not improve trajectory quality or obstacle avoidance, it significantly reduced cognitive load and enhanced learners' perceived collaboration with the expert. These insights suggest that augmented feedback can efficiently support visuomotor skill acquisition, particularly when tailored to task demands.

The results have broader implications for designing collaborative training systems, emphasizing the importance of matching feedback modalities to task-specific requirements. This work contributes to advancing the development of immersive virtual environments for skill acquisition and highlights the potential of multimodal feedback to enrich the learning experience.

# ACKNOWLEDGMENTS

The authors wish to thank all volunteers who participated in the study. This work was supported by a grant from the French National Research Agency (ANR-20-CE33-0010 Show-me).

# REFERENCES

[1] M. J. Barrington, D. M. Wong, B. Slater, J. J. Ivanusic, and M. Ovens. Ultrasound-guided regional anesthesia: how much practice do novices
require before achieving competency in ultrasound needle visualization using a cadaver model. *Regional Anesthesia & Pain Medicine*, 37(3):334-339, 2012. doi: 10.1097/AAP.0b013e3182475fba 1

[2] C. Basdogan, C. Ho, M. A. Srinivasan, and M. Slater. An experimental study on the role of touch in shared virtual environments. *ACM Transactions on Computer-Human Interaction (TOCHI)*, 7:443–460, 2000. 5

[3] J. Boyd and A. Godbout. Corrective sonic feedback for speed skating: A case study. *Georgia Institute of Technology*, 2010. 2

[4] J. L. Burke, M. S. Prewett, A. A. Gray, L. Yang, F. R. Stilson, M. D. Covert, L. R. Elliot, and E. Redden. Comparing the effects of visual-auditory and visual-tactile feedback on user performance: a meta-analysis. In *Proceedings of the 8th international conference on Multimodal interfaces*, pp. 108–117, 2006. 2

[5] D. Cecilio-Fernandes, F. Cnossen, J. Coster, A. D. C. Jaarsma, and R. A. Tio. The effects of expert and augmented feedback on learning a complex medical skill. *Perceptual and motor skills*, 127(4):766–784, 2020. 2

[6] A. Chellali, W. Ahn, G. Sankaranarayanan, J. Flinn, S. D. Schwaitzberg, D. B. Jones, S. De, and C. G. Cao. Preliminary evaluation of the pattern cutting and the ligating loop virtual laparoscopic trainers. *Surgical endoscopy*, 29:815–821, 2015. 2

[7] A. Chellali, C. Dumas, and I. Milleville-Pennel. Haptic communication to support biopsy procedures learning in virtual environments. *Presence: Teleoperators and Virtual Environments*, 21(4):470–479, 2012. 3

[8] A. Chellali, I. Milleville-Pennel, and C. Dumas. Influence of contextual objects on spatial interactions and viewpoints sharing in virtual environments. *Virtual Reality*, 17(1):1–15, 2013. 3

[9] E. F. Churchill, D. N. Snowdon, and A. J. Munro. *Collaborative virtual environments: digital places and spaces for interaction*. Springer Science & Business Media, 2012. 2

[10] C. Conati and H. Maclaren. Exploring the role of individual differences in information visualization. In *Proceedings of the working conference on Advanced visual interfaces*, pp. 199–206, 2008. 2

[11] G. Convertino, H. M. Mentis, A. Slavkovic, M. B. Rosson, and J. M. Carroll. Supporting common ground and awareness in emergency management planning: A design research project. *ACM Transactions on Computer-Human Interaction (TOCHI)*, 18(4):1–34, 2011. 3

[12] R. Crandall and E. Karadoğan. Designing pedagogically effective haptic systems for learning: a review. *Applied Sciences*, 11(14):6245, 2021. 9

[13] D. L. Eaves, G. Breslin, P. Van Schaik, E. Robinson, and I. R. Spears. The short-term effects of real-time virtual reality feedback on motor learning in dance. *Presence: Teleoperators and Virtual Environments*, 20(1):62–77, 2011. 2

[14] D. L. Eaves, M. Riach, P. S. Holmes, and D. J. Wright. Motor imagery during action observation: a brief review of evidence, theory and future research opportunities. *Frontiers in neuroscience*, 10:514, 2016. 8

[15] O. Ehmer and G. Brône. Instructing embodied knowledge: multimodal approaches to interactive practices for knowledge constitution. *Linguistics Vanguard*, 7(s4):20210012, 2021. 2

[16] M. Eriksson, K. A. Halvorsen, and L. Gullstrand. Immediate effect of visual and auditory feedback to control the running mechanics of well-trained athletes. *Journal of sports sciences*, 29(3):253–262, 2011. 2

[17] B. E. Erlandson, B. C. Nelson, and W. C. Savenye. Collaboration modality, cognitive load, and science inquiry learning in virtual inquiry environments. *Educational Technology Research and Development*, 58:693–710, 2010. 9

[18] Y. Feng, H. McGowan, A. Semsar, H. R. Zahiri, I. M. George, A. Park, A. Kleinsmith, and H. Mentis. Virtual pointer for gaze guidance in laparoscopic surgery. *Surgical endoscopy*, 34:3533–3539, 2020. 3

[19] D. Feygin, M. Keehner, and R. Tendick. Haptic guidance: Experimental evaluation of a haptic training method for a perceptual motor skill. In *Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. HAPTICS 2002*, pp. 40–47. IEEE, 2002. 2

[20] N. S. Fong, W. F. A. W. Mansor, M. H. Zakaria, N. H. M. Sharif, and N. A. Nordin. The roles of mentors in a collaborative virtual learning environment (cvle) project. *Procedia-Social and behavioral sciences*, 66:302–311, 2012. 8

[21] R. K. Ghamandi, R. K. Kattoju, Y. Hmaiti, M. Maslych, E. M. Taranta, R. P. McMahan, and J. LaViola. Unlocking understanding: An investigation of multimodal communication in virtual reality collaboration. In *Proceedings of the CHI Conference on Human Factors in Computing Systems*, pp. 1–16, 2024. 2, 3

[22] G. Gignac. How2statsbook (online 2023 edition), chapter 11. *Perth, Australia*, 2023. 7, 8

[23] R. B. Gillespie, M. S. O'Modhrain, P. Tang, D. Zaretzky, and C. Pham. The

virtual teacher. In *ASME International Mechanical Engineering Congress and Exposition*, vol. 15861, pp. 171–178. American Society of Mechanical Engineers, 1998. 1

[24] C. Gutwin, O. Schneider, R. Xiao, and S. Brewster. Chalk sounds: the effects of dynamic synthesized audio on workspace awareness in distributed groupware. In *Proceedings of the ACM 2011 conference on Computer supported cooperative work*, pp. 85–94, 2011. 3

[25] C. Harms and F. Biocca. Internal consistency and reliability of the networked minds measure of social presence. In *Seventh Annual International Workshop: Presence 2004*, 2004. 5

[26] S. G. Hart. Nasa-task load index (nasa-tlx): 20 years later. In *Proceedings of the human factors and ergonomics society annual meeting*, vol. 50, pp. 904–908. Sage publications Sage CA: Los Angeles, CA, 2006. 6

[27] S. G. Hart and L. E. Staveland. Development of NASA-TLX (task load index): Results of empirical and theoretical research. In *Advances in psychology*, vol. 52, pp. 139–183. Elsevier, 1988. doi: 10.1016/s0166-4115(08)62386-9 5

[28] J. Hattie and H. Timperley. The power of feedback. *Review of educational research*, 77(1):81–112, 2007. 1

[29] D. Hecht and M. Reiner. Sensory dominance in combinations of audio, visual and haptic stimuli. *Experimental brain research*, 193:307–314, 2009. 8

[30] N. Houghton, A. Prionas, R. Kneebone, and V. Papalois. Ethics of training surgeons. *British Journal of Surgery*, 111(12):znae252, 2024. 1

[31] M. S. Islam and S. Lim. Vibrotactile feedback in virtual motor learning: A systematic review. *Applied Ergonomics*, 101:103694, 2022. 2

[32] A. J. Kovacs and C. H. Shea. The learning of 90 continuous relative phase with and without lissajous feedback: external and internally generated bimanual coordination. *Acta psychologica*, 136(3):311–320, 2011. 2

[33] J. W. Krakauer and P. Mazzoni. Human sensorimotor learning: adaptation, skill, and beyond. *Current opinion in neurobiology*, 21(4):636–644, 2011. 8

[34] T. Langerak, J. J. Zárate, V. Vechev, D. Lindlbauer, D. Panozzo, and O. Hilliges. Optimal control for electromagnetic haptic guidance systems. In *Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology*, pp. 951–965, 2020. 4

[35] B. Lauber and M. Keller. Improving motor performance: Selected aspects of augmented feedback in exercise and health. *European journal of sport science*, 14(1):36–43, 2014. 7, 8

[36] C. Lin, D. Andersen, V. Popescu, E. Rojas-Munoz, M. E. Cabrera, B. Mullis, B. Zarzaur, K. Anderson, S. Marley, and J. Wachs. A first-person mentee second-person mentor ar interface for surgical telementoring. In *2018 IEEE international symposium on mixed and augmented reality adjunct (ISMAR-Adjunct)*, pp. 3–8. IEEE, 2018. 3

[37] M. Liu, S. Wilder, S. Sanford, M. Glassen, S. Dewil, S. Saleh, and R. Nataraj. Augmented feedback modes during functional grasp training with an intelligent glove and virtual reality for persons with traumatic brain injury. *Frontiers in Robotics and AI*, 10:1230086, 2023. 3

[38] C. MacKenzie, T. M. Chan, and S. Mondoux. Clinical improvement interventions for residents and practicing physicians: a scoping review of coaching and mentoring for practice improvement. *AEM Education and Training*, 3(4):353–364, 2019. 1

[39] L. Marchal-Crespo and D. J. Reinkensmeyer. Review of control strategies for robotic movement training after neurologic injury. *Journal of neuroengineering and rehabilitation*, 6:1–15, 2009. 2

[40] D. Martin, S. Malpica, D. Gutierrez, B. Masia, and A. Serrano. Multimodality in vr: A survey. *ACM Computing Surveys (CSUR)*, 54(10s):1–36, 2022. 2, 3

[41] A. Moinuddin, A. Goel, and Y. Sethi. The role of augmented feedback on motor learning: a systematic review. *Cureus*, 13(11), 2021. 1, 2, 7

[42] J. Moll, E.-L. S. Pysander, K. S. Eklundh, and S.-O. Hellström. The effects of audio and haptic feedback on collaborative scanning and placing. *Interacting with computers*, 26(3):177–195, 2014. 3

[43] K. L. Nowak and F. Biocca. The effect of the agency and anthropomorphism of users' sense of telepresence, copresence, and social presence in virtual environments. *Presence: Teleoper. Virtual Environ.*, 12:481–494, 2003. doi: 10.1162/105474603322761289 5

[44] I. Oakley, S. Brewster, and P. Gray. Can you feel the force? an investigation of haptic collaboration in shared editors. In *Proceedings of EuroHaptics*, vol. 2001, pp. 54–59, 2001. 2

[45] S. Ojala, J. Sirola, T. Nykopp, H. Kröger, and H. Nuutinen. The impact of teacher's presence on learning basic surgical tasks with virtual reality headset among medical students. *Medical education online*, 27(1):2050345, 2017. doi: 10.1177/1477731717020503 8
[46] A. Peña Pérez Negrón, N. E. Rangel Bernal, and G. Lara López. Nonverbal interaction contextualized in collaborative virtual environments. *Journal on Multimodal User Interfaces*, 9:253–260, 2015. 2

[47] L. Proteau and D. Elliott. *Vision and motor control*. Elsevier, 1992. 2

[48] R. Ranganathan and K. M. Newell. Influence of augmented feedback on coordination strategies. *Journal of motor behavior*, 41(4):317–330, 2009. 2

[49] G. Rauter, R. Sigrist, R. Riener, and P. Wolf. Learning of temporal and spatial movement aspects: A comparison of four types of haptic control and concurrent visual feedback. *IEEE transactions on haptics*, 8(4):421–433, 2015. 2

[50] R. Riener, M. Harders, R. Riener, and M. Harders. Vr for medical training. *Virtual reality in medicine*, pp. 181–210, 2012. 8

[51] E. M. Ritter and D. J. Scott. Design of a proficiency-based skills training curriculum for the fundamentals of laparoscopic surgery. *Surgical innovation*, 14(2):107–112, 2007. 4

[52] E.-L. Sallnäs, K. Rassmus-Gröhn, and C. Sjöström. Supporting presence in collaborative environments by haptic force feedback. *ACM Transactions on Computer-Human Interaction (TOCHI)*, 7(4):461–476, 2000. 3

[53] R. A. Schmidt. Frequent augmented feedback can degrade learning: Evidence and interpretations. In *Tutorials in motor neuroscience*, pp. 59–75. Springer, 1991. 2

[54] U. A. Shaikh, A. J. Magana, L. Neri, D. Escobar-Castillejos, J. Noguez, and B. Benes. Undergraduate students' conceptual interpretation and perceptions of haptic-enabled learning experiences. *International Journal of Educational Technology in Higher Education*, 14:1–21, 2017. 9

[55] L. Shams and A. R. Seitz. Benefits of multisensory learning. *Trends in cognitive sciences*, 12(11):411–417, 2008. 2

[56] Y. Shi, M. Liu, S. Dewil, N. Y. Harel, S. Sanford, and R. Nataraj. Augmented sensory feedback during training of upper extremity function in virtual reality. In *2024 IEEE 37th International Symposium on Computer-Based Medical Systems (CBMS)*, pp. 231–236. IEEE, 2024. 3

[57] R. Sigrist, G. Rauter, R. Riener, and P. Wolf. Augmented visual, auditory, haptic, and multimodal feedback in motor learning: a review. *Psychonomic bulletin & review*, 20:21–53, 2013. 2, 9

[58] C. Simon, M. Boukli-Hacene, F. Lebrun, S. Otmane, and A. Chellali. Impact of multimodal instructions for tool manipulation skills on performance and user experience in an immersive environment. In *2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)*, pp. 670–680. IEEE, 2024. 2, 3, 5

[59] C. Simon, M. Boukli-Hacene, S. Otmane, and A. Chellali. Study of communication modalities to support teaching tool manipulation skills in a shared immersive environment. *Computers & Graphics*, 117:31–41, 2023. 2, 3, 9

[60] C. Simon, L. Herfort, F. Lebrun, E. Brocas, S. Otmane, and A. Chellali. Design and evaluation of ultrasim: An immersive simulator for learning ultrasound-guided regional anesthesia basic skills. *Computers & Graphics*, 119:103878, 2024. 3

[61] H. G. Stassen, H. J. Bonjer, C. A. Grimbergen, and J. Dankelman. The future of minimally invasive surgery and training. *Engineering for patient safety: issues in minimally invasive procedures*, pp. 272–282, 2005. 1

[62] A. Steed and R. Schroeder. Collaboration in immersive and non-immersive virtual environments. *Immersed in media: Telepresence theory, measurement & technology*, pp. 263–282, 2015. 2

[63] S. Sülzenbrück and H. Heuer. Type of visual feedback during practice influences the precision of the acquired internal model of a complex visuo-motor transformation. *Ergonomics*, 54(1):34–46, 2011. 2

[64] Y. Sunaryadi. The role of augmented feedback on motor skill learning. In *6th International Conference on Educational, Management, Administration and Leadership*, pp. 271–275. Atlantis Press, 2016. 1

[65] M. Tagliabue and J. McIntyre. A modular theory of multisensory integration for motor control. *Frontiers in computational neuroscience*, 8:1, 2014. 8

[66] S. M. Underwood. Effects of augmented real-time auditory feedback on top-level precision shooting performance. Master's thesis, University of Kentucky, 2009. 2

[67] J. B. Van Erp and A. Toet. Social touch in human–computer interaction. *Frontiers in digital humanities*, 2:2, 2015. 9

[68] H. S. Vitense, J. A. Jacko, and V. K. Emery. Multimodal feedback: an assessment of performance and mental workload. *Ergonomics*, 46(1-3):68–87, 2003. 9

[69] J. Wang, A. Chellali, and C. G. Cao. Haptic communication in collabora-

tive virtual environments. *Human factors*, 58(3):496–508, 2016. 3

[70] K. Wei and K. Kording. Relevance of error: what drives motor adaptation? *Journal of neurophysiology*, 101(2):655–664, 2009. 2

[71] D. M. Wolpert and J. R. Flanagan. Motor learning. *Current biology*, 20(11):R467–R472, 2010. 8

[72] F. Wu, J. Thomas, S. Chinnola, and E. S. Rosenberg. Exploring communication modalities to support collaborative guidance in virtual reality. In *2020 IEEE conference on virtual reality and 3d user interfaces abstracts and workshops (VRW)*, pp. 79–86. IEEE, 2020. 3

[73] G. Wulf and C. H. Shea. Principles derived from the study of simple skills do not generalize to complex skill learning. *Psychonomic bulletin & review*, 9(2):185–211, 2002. 1

[74] G. Wyvill, C. McPheeters, and B. Wyvill. Data structure for soft objects. *The Visual Computer*, 2(4):227–234, Aug. 1986. doi: 10.1007/BF01900346 5

[75] X.-D. Yang, W. F. Bischof, and P. Boulanger. Validating the performance of haptic motor skill training. In *2008 Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems*, pp. 129–135. IEEE, 2008. 2

[76] E. Yiannakopoulou, N. Nikiteas, D. Perrea, and C. Tsigris. Virtual reality simulators and training in laparoscopic surgery. *International Journal of Surgery*, 13:60–64, 2015. 2

[77] A. Ziv, P. R. Wolpe, S. D. Small, and S. Glick. Simulation-based medical education: an ethical imperative. *Simulation in Healthcare*, 1(4):252–256, 2006. 1