###### Abstract  
In this paper, we studied the effects of using Microsoft HoloLens 2 in a Metaverse-based collaborative mixed reality environment on the driver's social presence while using an autonomous driving system. In (semi-) autonomous vehicles the driver is the system's monitor, and the driving process becomes a secondary task. Our approach is motivated by the advent of Microsoft Mesh XR technology that enables immersion in multi-person, shared mixed reality environments. We conducted a user study comparing the effects on social presence in two scenarios: baseline and mixed reality collaboration. During the baseline condition, participants communicated and interacted with another person using Skype/Meet which was installed on a mobile tablet. In the second scenario the participants used the Microsoft Mesh application installed on HoloLens 2 to collaborate in a mixed reality environment where each user is represented by an augmented 3D avatar. During the experiment, the participant had to perform a social interaction tell-a-lie task and a remote collaborative tic-tac-toe game, while also monitoring the vehicle's behavior. The social presence was measured using the Harms and Bioca questionnaire, one of the most widely used tools for evaluating the user's experience. We found that there are significant statistical differences for Co-presence, Perceived Emotional Interdependence, and Perceived Behavioral Interdependence, and participants were able to easily interact with the avatar in the mixed reality scenario. The proposed study procedure could be taken further to assess the driver's performance during handover procedures, especially when the autonomous driving system encounters a critical situation. social presence; HoloLens 2; driver immersion; mixed reality; human computer interaction; Metaverse +
Footnote †: journal: Medical  
also during these secondary tasks, which pose a significant risk to the safety of the driver and to other road participants. Cognitive distractions [3] were found to have a negative effect on hazard anticipation. A literature review on driver behavior was presented in [4]. The driver is the decision-maker and is combining two types of behavior: the automatic one, which is fast and effortless, with the slower one, which is more deliberate. Driving is a safety-critical task and the driver is also responsible for risk management [5]. In the decision-making process, the information flow is activating mental models, as the internal representations of the current state. To perform a task, the mental model is going through a cycle of stages like perception, prediction, evaluation, and action [6]. The Society of Automotive Engineers (SAE) has defined six levels of driving automation in their J3016 standard, from SAE level 0 (without any automation) to SAE level 5 (full driving autonomy). The driving process becomes a secondary task in (semi-)autonomous driving systems, as the human operator switches to supervising the process in SAE level 3, or can be completely out-of-the-loop in SAE level 4 (if certain road conditions are met) and 5 (in all driving conditions) [7]. The driving time, which is now rather a monitoring time, can be used for other tasks that are non-driving related [8]. One type of activity in which the driver can be engaged during driving is communicating and collaborating through a digital platform. This is commonly performed using conventional 2D video conferencing systems (Zoom, Google Meet, Microsoft Skype, Meta Messenger, WhatsApp). Recently, Nissan [9] presented the concept Invisible-to-Visible (I2V) that uses mixed reality to connect drivers and other participants by displaying them as 3D augmented reality (AR) avatars inside the vehicle. The I2V concept is likely to gain more traction with the progress of the Metaverse; however, there are certain technological and social challenges ahead. One issue could be related to the acceptance of the new technology, which can be prevented with proper user studies.