<figure><img src="image_1.png" /></figure>

HAL
open science

# The Triangle of Misunderstanding in Interactive Virtual Narratives: Gulfs Between System, Designers and Players

Florent Alain Sauveur Robert, Hui-Yin Wu, Lucile Sassatelli, Marco Winckler

<figure><img src="image_2.png" /></figure>

## To cite this version:

Florent Alain Sauveur Robert, Hui-Yin Wu, Lucile Sassatelli, Marco Winckler. The Triangle of Misunderstanding in Interactive Virtual Narratives: Gulfs Between System, Designers and Players. CHI 2025 - CHI Conference on Human Factors in Computing Systems, Apr 2025, Yokohama, Japan. pp.1 - 16, 10.1145/3706598.3713700. hal-05241510

HAL Id: hal-05241510

https://inria.hal.science/hal-05241510v1

Submitted on 5 Sep 2025

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_4.png" /></figure>

# The Triangle of Misunderstanding in Interactive Virtual Narratives: Gulfs Between System, Designers and Players

Florent Robert

I3S Lab, SPARKS/wimmics team

Université Côte d'Azur

Sophia Antipolis, France

Biovision

Centre Inria d'Université Côte d'Azur

Sophia-Antipolis, France

robertflorent11@gmail.com

Lucile Sassatelli

I3S Lab

Université Côte d'Azur

Sophia Antipolis, France

sassatelli@i3s.unice.fr

Hui-Yin Wu

Biovision

Centre Inria d'Université Côte d'Azur

Sophia-Antipolis, France

hui-yin.wu@inria.fr

Marco Winckler

I3S Lab, SPARKS/wimmics team

Université Côte d'Azur

Sophia Antipolis, France

marco.winckler@univ-cotedazur.fr

## Abstract

Designers of storytelling experiences in virtual reality (VR) can take advantage of the medium's realism and immersion to communicate their intentions. However, interaction freedom comes with unpredictability, raising the risk of miscommunication between the experience sought by the designer and the player's interpretation. To better understand such miscommunications, we revisit Don Norman's work on stages of action to propose a model of designer-player gulfs in VR that incorporates eight classes of communication gulfs. We designed a two-phase study where 10 participants designed VR scenarios and then played scenarios created by previous participants. Through coupled structured interviews, we identified 127 issues in VR-mediated communication that were mapped to our model to understand their impact on the player's interpretation of the narrative experience. Our work provides a roadmap to identifying sources of miscommunication in VR, a first step to conceiving principles and guidelines for achieving effective communication in storytelling experiences.

## CCS Concepts

* Human-centered computing → User centered design; User studies; User models; * Computing methodologies → Virtual reality.

## Keywords

Usability, Computer mediated communication, Qualitative method, Structured interviews, Virtual narratives

## ACM Reference Format:

Florent Robert, Hui-Yin Wu, Lucile Sassatelli, and Marco Winckler. 2025. The Triangle of Misunderstanding in Interactive Virtual Narratives: Gulfs

Between System, Designers and Players. In *CHI Conference on Human Factors in Computing Systems (CHI '25)*, April 26-May 01, 2025, Yokohama, Japan. ACM, New York, NY, USA, 16 pages. https://doi.org/10.1145/3706598.3713700

## 1 Introduction

Virtual reality (VR) media is seeing a growing usage to tell stories and communicate ideas, offering the possibility to use embodied interactions (i.e., natural interaction gestures to interact with the system), allowing the player to actively participate in the narrative [1], to simulate more life-like narrative experiences. The advantages of VR storytelling have been seen in many domains, opening possibilities to create meaningful daily experiences in edutainment [53, 55], rehabilitation [23, 46], therapy [11, 25], or professional training [6, 27, 29]. However, gulfs of comprehension between the designer's intention and the final player interpretation can happen on multiple levels: when choosing a strategy to communicate their intention, when building an experience, when the designer tests their own creation, and when players interact with the VR experience.

Designer-Player miscommunication is a well-documented phenomenon in the Human-Computer Integration (HCI) literature. The seminal work of Don Norman [32] illustrated what happens when players have trouble interacting and interpreting the signs produced by systems. He coined *interaction gulfs* measured in terms of *articulatory and semantic distances*, allowing a characterization of the distance between the player and the system. The work of Souza et al. [42] on semiotic engineering provides a picture of communication issues happening during the design of interactive systems by proposing a triplet *designer-system-player communication*. In this triplet, the system is an object that mediates the communication between the designer and the player. While widely relevant for understanding communication issues, most of these HCI works focus on traditional media, based on 2D interfaces and affordances. They do not address the specificities of VR applications such as the immersion, the embodied interaction possibilities offered when the body is used as a communication device [2, 3], or the physical fatigue and eye strain generated by this technology usage [14, 39].

<figure><img src="image_5.png" /></figure>

This work is licensed under a Creative Commons Attribution 4.0 International License.

CHI '25, Yokohama, Japan

© 2025 Copyright held by the owner/author(s).

ACM ISBN 979-8-4007-1394-1/25/04

https://doi.org/10.1145/3706598.3713700
Due to these specificities, on the one hand, immersive technologies like VR are likely to require alternative methods to observe and comprehend the lived experience. On the other hand, we hope to gain new insight by exploring the cause-effect relation between a designer creating a life-like narrative experience, asking for life-like embodied interactions, and the player living this experience.

In the context of creating meaningful immersive narrative experiences, where immersion [35, 41] and the sense of presence [40, 43] come into play, we investigate the designer-player relationship, mediated by the VR system. Bad communication on the scenario's narrative or on how to interact with 3D objects – such as coming from personal biases, system limitations, forgetfulness, or lack of guidelines – can impact the player's immersion or the enjoyment of the experience. On the other hand, an incomplete comprehension of the player's lived experience can impact the designer's knowledge on how to improve an experience. On this topic, some works and models have been proposed but rarely explore the complete designer-system-player relationship for immersive narrative experiences. VR is used in this context to represent the complex communication issues one encounters with highly immersive media. Designer-Player miscommunication is far more likely to occur due to the embodied interactions with the System as well as the freedom of exploration VR offers.

To more accurately observe and characterize the nature of the communication gulfs existing between the designer's intention and the player's lived experience in complex immersive narrative experiences such as ones afforded by VR, we conceived a communication model that supports the identification and mapping of eight classes of issues causing miscommunication or misinterpretation in the design-interpretation process. Our model is inspired by the seven stages of action model built by Norman [32], which refers to the way a user interacts with a product or a system in seven steps, separated into two phases: the execution phase, during which the user will form an intention, construct a plan, and execute it, and the evaluation phase, during which the user will observe the result of their action, make an interpretation, and compare the consequences with their expectations. As a result, two key gulfs emerge from the usage of a system: the execution gulf (i.e., the gap between the initial intention and the performed actions) and the evaluation gulf (i.e., the gap between what the system shows and what the user expected).

Our paper presents three contributions :

* A communication model to investigate the designer-player communication gulfs, which highlights the different steps composing the creation process between the designer and the system, and the usage process between the player and the system.

* A qualitative method in the form of a structured interview, to observe and collect communication issues happening during the usage and design of VR narrative experiences. This interview was applied during a two-phase VR study, allowing the collection of 127 design and usage communication issues.

* A taxonomy of communication issues to supports the identification and mapping of all encountered communication issues among eight classes of issues causing miscommunication or misinterpretation, to analyze their individual impact on the proper narrative interpretation of the scenario.

By revisiting Norman's model in the context of VR narrative experiences, including both the design and the usage process, we can support the identification, and eventually the reduction, of designer-player communication gulfs in VR, by proposing means to treat each class of communication issues. Throughout this paper, we refer to two user roles: those who create scenarios are *Designers*, and those who test the scenarios are *Players*.

In the following sections, we present the related work on intention communication focusing on VR domains (Section 2). We describe in section 4 our communication model and the structured interview built to collect and classify issues. Then, in section 4.2 we present the study design and the technical setup. In section 5 we present in detail three case studies, a correlation analysis between the issues, alongside some post-usage insight from participants. Finally, in section 6 we discuss the limitations and present the next steps of this work.

## 2 Related work

Understanding the gulf between the designer's expected usage of a product and its actual usage in real-life situations is key to the design of consumer products, human-computer interfaces, or the creation of virtual reality experiences. Such gulf has multiple impacts on the product depending on the usage context: using the product the wrong way can reduce its usability, directly affecting its efficiency and user satisfaction [12]. In research domains, it can be valuable for experiences and user studies to be interpreted and lived the way the designer intended them to be to ensure a precise investigation of the research question. Diverse research domains have investigated ways to formalize and reduce this designer-user gulf by defining models, taxonomies of communication issues, or exploring means to more efficiently communicate an intention to a user through a product, a system, or an experience.

### 2.1 Communication in semiotic

Communication through a product has been a key subject in the industry for a long time, to maximize a product's efficiency or attractiveness. Norman [31] described a product as a service about the discovery, the anticipation, and the first usage. He presents a product as more than a product; it's an experience built by the designer and lived by the consumer. Crilly et al. [7] investigated the nature of the designer-user relation around a product. They present the design of a product as a way to communicate an intention, which is then interpreted and used by the consumer based on their own comprehension. Norman [30] highlighted how good designers can efficiently communicate with their users, using visible clues, hints, and affordances, to maximize their chances for the user to understand and use the product as intended. These works set the basis for the importance of intention communication when designing a product. Souza and Leitão [42] presented how semiotic engineering describes human-computer interaction (HCI) as a computer-mediated communication between designers and users, where the system speaks for its designer, to communicate an intention by using diverse affordances. This work presents methods
<figure><img src="image_6.png" /><figcaption>Figure 1: Our model was inspired by Norman [32] to characterize the designer-system-player communication gulfs, allowing the definition of a taxonomy of eight issues sources of gulfs happening during the design, usage, and interpretation phases.</figcaption></figure>

such as the semiotic inspection method to appreciate the quality of the meta communication and the communicability evaluation method, which serves to improve the designer-user communication by analyzing the quality of the communication done through the user interface. Norman [32] presented a categorization of user interaction issues with a product as slips and mistakes: slips represent a conscious issue while performing the correct action, while mistakes represent an unconscious issue in understanding the correct goal. Laubheimer [21] applied this categorization to website and application design to present strategies and guidelines for preventing slips and mistakes. These works ultimately aim at reducing the designer-user gulfs, highlighting two ways to proceed: evaluate the quality of the intention communication mediated by the system, or categorize and propose guidelines to prevent the communication issues in the design or usage process source of potential gulfs.

## 2.2 Designer intention communication

A common way to investigate issues is through a taxonomy on the human errors while interacting with a system. Bolton [4] established such a taxonomy to classify the different divergences in user behavior from a normative task, and how to identify the information the user failed to properly understand or attend to. Paek's taxonomy [34] on types of communication errors in linguistic or conversation analysis highlights the different types of failures to present guidelines to handle such errors for automated systems. Gibson et al. [13] presented a taxonomy on communication issues related to railway track maintenance, classifying the observed issues according to the communication topic and the type of error in question. Tian and Oviatt [48] presented a taxonomy focused on human-robot social interaction errors and investigated their impact on the user's perception of the system's capability to achieve a task or have socio-affective interaction. The use of taxonomies can be valuable, to prevent or avoid erroneous user actions - a main concern for system designers. However, none of these works explore issues in the context of interactive environments for VR, where the

specificity of embodied interaction and immersive environment can require new ways in order to observe and understand experiences.

To reduce the designer-user gulf, another solution is to explore efficient ways to communicate an intention to the user or relevant users if they deviate from the intended experience. Matthews et al. [26] investigated how to communicate an intention to let a pedestrian cross the road when using autonomous cars without drivers. By showing a textual message to explicitly communicate the intent, users reacted more predictably compared to situations without explicit communication. This serves to enable both better communication and a better understanding of the user's behavior. Li and Zhang [22] looked at ways in robot-human interactions to implicitly communicate a user intention to a robot by proactively analyzing user gaze. They found positive results on the usage of gaze data to improve user-system communication, opening possibilities for the system to better capture the user's intention. These works highlight interesting ways to communicate intentions and communicate with users. While these works weren't built for VR, we believe that comparable application could be proposed and applied for the VR context.

In game design literature, the designer-system-player relationship has been explored to build a bridge between how games are designed and how to formalize these approaches to address research questions. In 2003, Tekinbas et al. [47] conceived a theoretical framework to analyze and design games through a clear vocabulary of components. They define for example groups of "rules" that formally structure game mechanics, to control and guide players. Robin et al. [15] presented the MDA Framework to formalize the consumption and creation of games by breaking these two processes down into multiple components. This framework describes games as artifacts, possessing their own behavior to interact with the player. Rogério et al. [16] redefined the MDA Framework to build an ontology aiming at enhancing the understanding of the game design domain that would support the conception of a structured methodology to design games. However, the focus of these works is primarily on the
formalization of game design to support the creation of structured
methodologies that can subsequently be used to establish guidelines
and good practices that support game design research. Communi-
cation issues that arise between the designer and player mediated
by a system were not the objects of research and observation.

## 2.3 Communication in virtual reality

In virtual reality, communicating an intention can be complex, due
to the offered freedom and interactivity. The work of Larusdot-
tir et al. [20] highlights this listing numerous issues encountered
during a formal user testing session where users had to climb a
mountain in VR. Speicher et al. [44] presented a prototyping ap-
plication for VR to simulate interactions in a minimalist version
of the 3D environment captured by a camera. This prototyping
methodology increased realism and reduced the gap between the
prototype tested by the designer and the actual experience pro-
posed by the system, to support designers on the refinement of
their built experiences. Wolfartsberger and Niedermayr [52] ex-
plored ways in VR to reduce the deviation of the player from the
intended path in a learning tool. They presented a virtual "ghost
user" performing the tasks to help the player learn. David-John
et al. [8] used gaze data in order to investigate the prediction of
the player's interactions with the 3D environment, which could
open possibilities for using passive, biological responses to reorient
players at the right moment who are deviating from the intended
experience. To reduce the designer-player gulf in VR, Oh and Kim
tried to [33] use onomatopoeia and sound effect feedback to empha-
size or draw attention to a precise element. Through a study with
and without onomatopoeia, onomatopoeia showed a higher success
in communicating the designer's intention and offer more object
affordance. All the presented works are helpful to assist designers
in communicating their intentions efficiently in VR, however none
of them explore the designer-player relationship of an experience,
mostly looking at one side of the relation.

All these works highlight the importance for VR research to
investigate the designer-system-player relationship. A reduction
of the different communication gulfs in VR could be valuable to
better align the designer's intention with the player's lived expe-
rience. Vignais et al. [50] for example, investigated gulfs existing
between experimental protocols built by designers and the subject
behavior in real conditions in the case of sport science. Virtual real-
ity was used to reproduce real-world situations with total control
over the environment and the elements inside it. Vuarnesson et al.
[51] investigated multiple players communicating their intention
in VR, and ways to improve this communication using minimalist
representation of players outside of VR to make them co-present,
showing a positive impact on relational dynamics. Cavalcanti et al.
[5] investigated ways in VR to teach participants about risk and
to evaluate the impact of their choices. In a gamified environment,
players had to cross a maze while avoiding dangers. We observe
here a key gulf between the designer's intention and the player's
lived experience: while designers expected the player to avoid the
danger, the players, who knew they were in a virtual environment,
had the tendency to test the consequences of their actions rather
than maintaining safety. As the designer's intention doesn't align
with the player's experience, this could impact the conclusion the

designers can draw out of such studies, especially in a situation
where this communication gulf was not noticed by the designer.
Means to detect and prevent these types of communication issues
could support the design and the comprehension of immersive expe-
riences. For example, a structured interview could be built to collect
the player's interpretations and feelings, and guidelines could then
be provided to players to ask them to act as if the danger was
real. While previously presented models ([42],[32]) already address
the observation and improvement of user-system communication,
none explored the designer-system-player communication. These
models were built to observe user-system interactions, the commu-
nication between two type of users mediated by a system was not
considered by these works. A model to observe the designer-system-
player communication is particularly important in the context of
immersive environments, which afford the use of the body as a com-
munication device, multiplying the possibilities of communication
issues compared to traditional media.

## 3 VR communication model

We present our model of VR communication shown in Figure 1. The model was inspired by Norman's seven stages of action, well-known in the HCI research community, to represent the communication between a *player* and a *system*. In our case, we need to add a third actor: the *designer* who uses the system to create the experience for the player, who will then live the experience through the system. Both the designer and player interact with the system following Norman's seven stages of action: develop and act on their intention to interact with the system, and compare the changes against the initial intention. The system is thus a medium through which the designer communicates with the player, from which also emerge designer-player gulfs, notably between the initial design intention and the final player interpretation. For these reasons, our model is the concatenation of two models of the seven stages of action, with the left side representing the design process and the right side representing the usage process.

Our model specifies four designer-system-player gulfs and eight
classes of issues that are sources of these gulfs. The first four issues
occur on the designer-system side of Figure 1, potentially creating
a design gulf (i.e., the gap between what the designer wanted to
create, and what they actually create) and an evaluation gulf (i.e.,
the gap between what the system shows and what the designer
understands). During this phase, the designers define their design
intention and build a VR scenario to communicate the intention
through the system. They then test their scenario, reiterating the
design process until they can validate that it corresponds to their
initial intention. The four classes of issues are:

(1) **Design limitations issues:** limitations on the system to implement a desired scenario, potentially limiting the designer's creativity. For example: a designer who wants to build a scenario with fireworks would rely on the system having an adapted particle system, the lack of which would force the designer to change or suppress this idea.

(2) **Design issues:** actions that are present in the designer's detailed plan, but not performed, or performed incorrectly. For example: the designer forgets to create a task or associates the
wrong interaction to an object resulting in a mismatch between the intention and final scenario.

**(3) Evaluation perception issues:** gaps identified by the designer between what they wanted to create during the design process and what they experienced while testing the scenario. For example, a designer expects an object to be clearly visible, but it is unexpectedly occluded due to the camera perspective.

**(4) Evaluation comprehension issues:** issues that the designer did not notice during the testing. For example, if the designer forgot to create a task and did not realize it during the test.

The second set of four issues occur on the system-player side in Figure 1, creating the scenario comprehension gulf (that is, the gap between what the system communicates and what the player understands) and the system interaction gulf (i.e., the gap between the expected and actual interactions of the player with the system). The four classes of issues in this phase are the following:

**(5) Task comprehension issues:** The player misunderstands the task to perform when communicated by the system. For example, the player takes the wrong object or performs the wrong interaction.

**(6) Situation comprehension issues:** the lack of understanding on the player's side of the purpose of a task. For example: not understanding the goal of the task and/or the higher message of the scenario. As a result, the player may believe that the task serves no purpose in the story.

**(7) Environment comprehension issues:** lack of comprehension by the player due to the manner in which the system communicates (e.g. visuals, audio, text). This can occur, for example, when the player confuses the target object of the task with similar objects in the scene.

**(8) Performance issues:** issues in executing the task by the player. For example, a player is unable to find the target object or fumbles with the controllers to perform a desired action.

This specification of issues in eight classes opens up the possibility of investigating in depth the link between issues, gulfs, and narrative interpretation. To detect and classify communication issues among these categories, we built a qualitative methodology presented in the next section.

# 4 Methodology

We apply the model to a user study in VR where participants had to play both the roles of a designer and a player, paired with a structured interview based on our model. The following sections first detail the design of the structured interview, then describe the player study.

## 4.1 Structured interview

The structured interview is composed of a set of questions for each phase of the design-usage process specified in our model, and was posed after each phase by the experimenter and recorded in a questionnaire. The list of questions are shown in Table 1. The reasoning for question in each of the eight categories were:

**Design limitation issue:** a designer's conscious gap between the initial design intention (Q1-3) and the scenario they were able to implement using the system (Q6,7).

**Design issue:** a designer's unconscious gaps between what they believe they created (Q1-7), and what they actually conceived (Q16-18), identified during the interview with the designer after player testing.

**Evaluation perception issue:** a designer's conscious gaps between what they expected to create and communicate (Q5,6,7) and what they observed while testing their own scenario (Q8-11), identified during the designer evaluation phase.

**Evaluation comprehension issue:** a designer's unconscious gaps between what they believe their scenario communicates (Q6,8-11), and what the players understood (Q12-18), identified by discussing the elements players didn't understand with the designer.

**Task comprehension issue:** a player's unconscious gap between the action the system asks them to perform (Q5,6), and the action the player understood (Q12-15).

**Situation comprehension issue:** a player's unconscious gap between the meaning of the task presented by the system (Q5,6) and the comprehension of the purpose of this task by the player (Q12-15).

**Environment comprehension issue:** a player's unconscious gap between the interaction expected by the system to validate a task (Q5,6) and what the player actually performed (Q12-14).

**Performance issue:** a player's gap between the expected performance of the designer (Q5,6), and the observed performance of the player during the scenario (Q16,17).

Finally, we included a category on **narrative interpretation**, corresponding to the high-level gap between the player's understanding of the meaning of the scenario they tested (Q15,19-21) and the scenario the designer actually wanted to communicate (Q1-3,22,23). While all previous presented issues aim at characterizing the eight designer-system and system-player gulfs, the narrative interpretation falls outside of this model. This enables the investigation of the potential correlation between the eight issues encountered and the quality of the narrative interpretation observed.

## 4.2 User study

We present here a qualitative study protocol performed to observe and collect information regarding the eight classes of issues. The study objective was to investigate (1) the impact of an issue on the creation of other issues in a later process, and (2) the impact of a given issue on the final comprehension of the experience by the user. The findings on these objectives are presented in Section 5.

We recruited six participants for the pilot studies (5 women, 1 man) and ten participants for the main study (5 women, 5 men) through 5 university and laboratory mailing lists. Participants needed to have normal or corrected-to-normal vision. Among the ten participants recruited for the main study, five had no prior experience with 3D scenario design (Participants 3,4,6,9,10), and five were expert (Participants 1,2,5,7,8), of using the Unity Game Engine to build 3D experiences for professional activities. The 10 participants had a variety of previous experience with VR headsets, detailed in Table 2. The study was approved by the institute ethics committee. The study was approved by the Université Côte d'Azur ethics committee (Nº 2022-057). Upon recruitment, participants received a message with their booked time slot, as well as
<table><caption>Table 1: The structure interview built for the study, based on our model, to collect the initial intention, the design issues, usage issues, and interpretation issues. These questions allows us to characterize the gulf between the designer initial intention and the player narration interpretation, by collecting the through process on both sides and comparing them.</caption><thead><tr><th>Issues</th><th>Questions</th></tr></thead><tbody><tr><td>Intention evaluation</td><td>(1) What scenario do you intend to create ?<br>(2) What reaction, emotion, or experience do you want to generate through this scenario ?<br>(3) What do you want players to experience through your scenario ?</td></tr><tr><td>Designer - System issues</td><td>Before the design :<br>(3) What do you want players to experience through your scenario ?<br>(4) What do you plan to do in the scenario to generate these reactions, emotions, or experiences ?<br>(5) What sequence of tasks would you like to create to achieve this scenario ?<br>After the design :<br>(6) What scenario did you create ?<br>(7.1) Are there differences between your scenario and your initial intention ?<br>(7.2) What are the reasons for the differences between your scenario and your initial intention ?<br>After the test :<br>(8) What did you observe during your test ?<br>(9) What did you understand about the scenario during the test ?<br>(10) Do you notice any differences between what you tested and what you wanted to create ?<br>(11) If you could change anything, what would you change ?</td></tr><tr><td>Player - System issues</td><td>To the scenario player :<br>(12) In order, what actions did you perform to achieve the scenario ?<br>(13) Did the observed consequences of your actions correspond to your expectations ?<br>(14) How successful were you in accomplishing what you were asked to do ?<br>(15) What did you understand from the scenario ?<br>To the scenario designer :<br>(16) Was the scenario played as you expected ?<br>(17) The lived experience correspond to your expectations ?<br>(18) In your opinion, what experience did the player live ?</td></tr><tr><td>Narrative interpretation</td><td>To the scenario player :<br>(19) What did you understand from the scenario ?<br>(20) In your opinion, what did the designer want to communicate through this scenario ?<br>(21) What is the designer's intention : What points correspond or not to your experience ?<br>To the scenario designer :<br>(22) In your opinion, what experience did the player live ?<br>(23) What did the player understand : Does that correspond to what you expected ?</td></tr></tbody></table>

<figure><img src="image_7.png" /><figcaption>Figure 2: The study protocol consists of three stages: (1) design phase, during which participants create and evaluate their own experience; (2) usage phase, during which each participants experience two previously created scenarios; (3) and the post-usage phase during which each participant watch record of two persons playing the created scenario. The entire study lasted roughly two hours.</figcaption></figure>

the informed consent form. The study lasted up to two hours. A compensation of twenty euros was given in the form of a check at the end of the post-usage phase. On arrival, participants were first invited to sign the informed consent form and answer a short pre-experience survey. Participants were informed of the risks of nausea

and motion sickness during the usage phase and were encouraged to ask for a pause or request the end of the study if desired, without impact on their compensation. A time slot was defined with the participant at the end of the study to perform the post-usage phase of the study.
<table><caption>**Table 2:** The profile of the 10 participants was assessed using two preliminary questions. Question 1: "Have you ever used a virtual reality headset?", Question 2: "Have you ever used a game engine to create 3D scenarios?". Participants rated their experience on a scale from 0 to 4, where 0 = Never, 1 = Once or twice, 2 = Sometimes, 3 = Regularly, 4 = I develop applications in [virtual reality / 3D with a game engine].</caption><thead><tr><th></th><th>Gender</th><th>Age</th><th>Question 1: VR headset experience</th><th>Question 2: 3D design experience</th></tr></thead><tbody><tr><td>D1</td><td>M</td><td>26-30</td><td>2</td><td>4</td></tr><tr><td>D2</td><td>F</td><td>18-25</td><td>4</td><td>4</td></tr><tr><td>D3</td><td>M</td><td>26-30</td><td>2</td><td>0</td></tr><tr><td>D4</td><td>M</td><td>18-25</td><td>1</td><td>0</td></tr><tr><td>D5</td><td>M</td><td>18-25</td><td>1</td><td>4</td></tr><tr><td>D6</td><td>F</td><td>36-40</td><td>1</td><td>0</td></tr><tr><td>D7</td><td>F</td><td>26-30</td><td>4</td><td>4</td></tr><tr><td>D8</td><td>F</td><td>18-25</td><td>4</td><td>4</td></tr><tr><td>D9</td><td>M</td><td>26-30</td><td>2</td><td>0</td></tr><tr><td>D10</td><td>F</td><td>51-60</td><td>1</td><td>0</td></tr></tbody></table>

During this two hour study, participants played the roles of both the designer and the player. Through structured interviews, they explained as designers their intentions and implementation plans, and as players their perceptions and interpretation of the scenario. A series of questions were used to determine if there were issues, which were recorded and classified post-usage. The study is composed of three phases: (1) **the design phase** during which participants, as designers, create a VR scenario; (2) **the usage phase** during which participants, as players, experience two scenarios created by other designers; and (3) **the post-usage phase** at a later date, during which designers watch the recordings of two players who played their scenario. Two additional participants were recruited for only phase two, to ensure that all scenarios had two players. The questions for each phase are shown in Table 1 to identify for each phase the issues and their type, as well as the initial designer intention and the narrative interpretation of the scenario by the player.

In Phase 1, the design phase, participants created their VR experience on a laptop using the GUSt-3D<fnref n="1" /> open toolkit which is built on Unity. This toolkit allows the definition of a vocabulary to name objects and locations, and to designate a series of interactive user tasks such as finding and placing objects in specific locations, going somewhere, or using an object to interact with another. Text, audio, and visual hints can also be added to the tasks. The toolkit has the advantage of allowing the design of interactive VR scenarios without the need for prior training in Unity nor programming. Before starting the study, participants followed a 10-15 minutes long tutorial on how to build a 3D experience using GUSt-3D. The VR setup of the usage phase consisted of an HTC Vive Pro Eye with a wireless module, two HTC Vive controllers to interact with the environment, and four HTC base stations to track a 5 meters by 5 meters space. This VR setup allows participants to physically

walk and interact in the virtual environment freely without cables restraining their movement. Logs are generated from the user's position and interactions at 10Hz.

**Design phase**: participants as designers were asked to create a 3D scenario composed of at most 10 tasks including the interactions: "Take an object", "Place an object", "Go somewhere", "Interact with an object". Each designer was assigned one of two 3D environments of a size of 5 meters by 5 meters to build their scenario:

(1) Interior environment (Figure 3 top) with an office desk, a shelf, and some objects around the room. Designers assigned to this environment were given the constraints to use the objects "Golf ball", "Golf club", and "An object fallen on the ground" in their scenario.

(2) Exterior environment (Figure 3 bottom), comprised of a small entry hall with some furniture, and a crosswalk behind a door leading to the other side of the road, with some additional props. Designers assigned to this environment were given the constraint to use the objects "Key", "Trashcan", and "A sweet object".

The scenario's tasks appear in the form of a text prompt in the field of view of the player, and are automatically updated as tasks are completed. Designers were additionally allowed to add at most two textual hints, appearing below the task prompt, to either guide the player or give additional narrative information. Designers were allowed to move and rename objects in the scene to fit the scenario they wanted to communicate. While creating this scenario, designers were able to make as many tests and modifications as they wanted. At the end of the scenario design, designers tested their scenario in VR, but they were not allowed to make further modifications. Questions were asked during this phase to collect the initial design intention and characterize the observed issues, as shown in Table 1 rows "Intention evaluation" and "Designer - System issues". This phase took a maximum of 1 hour and 15 minutes: 1 hour for the experience design and 15 minutes to test the scenario in VR, including the structured interview questions.

**Usage phase**: during the second phase of the study, participants take the player role and experience two scenarios created by the two preceding designers, one interior scene and one exterior. For example Designer 5 is Player 2 for Designer 3 (which we denote as D3P2) and Player 1 for Designer 4 (D4P1). The player then responded to questions regarding the issues encountered in this phase, and the interpreted the meaning of the scenario in rows "Player - System issues" and "Narrative interpretation" of Table 1 respectively. This phase takes up to 30 minutes.

**Post-usage phase**: the third phase was programmed at a later date with the designer after two players experienced the designer's scenario. During this phase, a recording from the viewpoint of the two players is shown to the designer. Following each recording, questions shown in the Table 1 under the rows "Player - System issues" and "Narrative interpretation" were posed to determine if the scenario interpretation and experience corresponded to what the designer intended. This phase takes up to 15 minutes. With the data collected from this study, we present in the section 5 our analysis and classification of gulfs, showing correlations between issues and the proper narrative interpretation. We present feedback from the designers regarding issues in their scenario during the post-usage, and the impact it had on the communicated and lived experience.

<footnote n="1">¹https://project.inria.fr/creattive3d/gust-3d/</footnote>
<table><caption>Table 3: The correlation matrix on the eight issues and additionally the narrative interpretation. We highlight (in green) the issues that are potentially correlated with another one in a later part of the design-usage process, or in correlation with the score observed of narrative interpretation of the scenario by the player.</caption><thead><tr><th rowspan="2"></th><th colspan="4">Designer - System issues</th><th colspan="4">Player - System issues</th><th rowspan="2">Narrative interpretation</th></tr><tr><th>System design limitation</th><th>Design issues</th><th>Evaluation perception issue</th><th>Evaluation comprehension issues</th><th>Task comprehension issues</th><th>Situation comprehension issues</th><th>Environment comprehension issues</th><th>Performance issues</th></tr></thead><tbody><tr><td>Design limitation issues</td><td>1</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Design issues</td><td>-0,3</td><td>1</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Evaluation perception issue</td><td>0,2</td><td>-0,2</td><td>1</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Evaluation comprehension issues</td><td>-0,1</td><td>0,2</td><td>-0,7</td><td>1</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Task comprehension issues</td><td>0,2</td><td>-0,1</td><td>-0,1</td><td>-0,1</td><td>1</td><td></td><td></td><td></td><td></td></tr><tr><td>Situation comprehension issues</td><td>0,2</td><td>0,2</td><td>0,1</td><td>0,0</td><td>-0,3</td><td>1</td><td></td><td></td><td></td></tr><tr><td>Environment comprehension issues</td><td>0,2</td><td>0,0</td><td>0,3</td><td>-0,1</td><td>0,0</td><td>0,2</td><td>1</td><td></td><td></td></tr><tr><td>Performance issues</td><td>0,2</td><td>0,3</td><td>0,4</td><td>-0,1</td><td>0,0</td><td>0,2</td><td>0,6</td><td>1</td><td></td></tr><tr><td>Narrative interpretation</td><td>-0,4</td><td>0,6</td><td>0,1</td><td>0,0</td><td>-0,1</td><td>0,4</td><td>0,2</td><td>0,5</td><td>1</td></tr><tr><td>Total number of issue</td><td>0,3</td><td>0,5</td><td>0,3</td><td>0,0</td><td>0,1</td><td>0,5</td><td>0,6</td><td>0,8</td><td>0,5</td></tr></tbody></table>

<table><caption>Table 4: All the issues observed during the 10 studies, classified under the eight issues specified by our model. Player Y of the designed experience X is referred as DXPY, for example Player 2 of the experience 7 is D7P2. The narrative interpretation row displays the score given for the accuracy of the narrative interpretation done by the user. A score of 0 means the narrative was interpreted as intended, 1 means partially correct, and 2 means an incorrect interpretation.</caption><thead><tr><th rowspan="2"></th><th colspan="4">Designer - System issues</th><th colspan="4">Player - System issues</th><th rowspan="2">Narrative interpretation</th></tr><tr><th>Design limitation</th><th>Design issues</th><th>Evaluation perception issue</th><th>Evaluation comprehension issues</th><th>Task comprehension issues</th><th>Situation comprehension issues</th><th>Environment comprehension issues</th><th>Performance issues</th></tr></thead><tbody><tr><td>D1P1</td><td>0</td><td>2</td><td>1</td><td>0</td><td>1</td><td>0</td><td>0</td><td>1</td><td>2</td></tr><tr><td>D1P2</td><td>0</td><td>2</td><td>1</td><td>0</td><td>0</td><td>0</td><td>1</td><td>2</td><td>2</td></tr><tr><td>D2P1</td><td>1</td><td>0</td><td>1</td><td>0</td><td>0</td><td>1</td><td>1</td><td>1</td><td>0</td></tr><tr><td>D2P2</td><td>1</td><td>0</td><td>1</td><td>0</td><td>0</td><td>1</td><td>0</td><td>0</td><td>0</td></tr><tr><td>D3P1</td><td>1</td><td>3</td><td>2</td><td>0</td><td>1</td><td>1</td><td>1</td><td>2</td><td>2</td></tr><tr><td>D3P2</td><td>1</td><td>3</td><td>2</td><td>0</td><td>0</td><td>1</td><td>2</td><td>2</td><td>2</td></tr><tr><td>D4P1</td><td>1</td><td>2</td><td>1</td><td>1</td><td>0</td><td>0</td><td>0</td><td>1</td><td>0</td></tr><tr><td>D4P2</td><td>1</td><td>2</td><td>1</td><td>1</td><td>0</td><td>2</td><td>1</td><td>2</td><td>2</td></tr><tr><td>D5P1</td><td>0</td><td>3</td><td>0</td><td>1</td><td>0</td><td>1</td><td>0</td><td>0</td><td>2</td></tr><tr><td>D5P2</td><td>0</td><td>3</td><td>0</td><td>1</td><td>0</td><td>1</td><td>0</td><td>1</td><td>2</td></tr><tr><td>D6P1</td><td>0</td><td>1</td><td>1</td><td>1</td><td>0</td><td>0</td><td>1</td><td>1</td><td>1</td></tr><tr><td>D6P2</td><td>0</td><td>1</td><td>1</td><td>1</td><td>0</td><td>0</td><td>1</td><td>1</td><td>1</td></tr><tr><td>D7P1</td><td>1</td><td>0</td><td>2</td><td>0</td><td>1</td><td>0</td><td>0</td><td>1</td><td>1</td></tr><tr><td>D7P2</td><td>1</td><td>0</td><td>2</td><td>0</td><td>0</td><td>0</td><td>1</td><td>1</td><td>1</td></tr><tr><td>D8P1</td><td>1</td><td>1</td><td>1</td><td>1</td><td>0</td><td>1</td><td>1</td><td>1</td><td>2</td></tr><tr><td>D8P2</td><td>1</td><td>1</td><td>1</td><td>1</td><td>0</td><td>1</td><td>1</td><td>2</td><td>2</td></tr><tr><td>D9P1</td><td>1</td><td>2</td><td>0</td><td>1</td><td>0</td><td>0</td><td>1</td><td>1</td><td>1</td></tr><tr><td>D9P2</td><td>1</td><td>2</td><td>0</td><td>1</td><td>0</td><td>0</td><td>0</td><td>1</td><td>1</td></tr><tr><td>D10P1</td><td>1</td><td>1</td><td>0</td><td>1</td><td>1</td><td>0</td><td>1</td><td>1</td><td>0</td></tr><tr><td>D10P2</td><td>1</td><td>1</td><td>0</td><td>1</td><td>2</td><td>0</td><td>1</td><td>1</td><td>1</td></tr><tr><td>Total</td><td>14</td><td>30</td><td>18</td><td>12</td><td>6</td><td>10</td><td>14</td><td>23</td><td>25</td></tr><tr><td>Average</td><td>0,7</td><td>1,5</td><td>0,9</td><td>0,6</td><td>0,3</td><td>0,5</td><td>0,7</td><td>1,15</td><td>1,25</td></tr></tbody></table>

# 5 Results

The user study guided through structured interviews allowed us to construct a rich dataset that contains gulfs of communication occurring in interactive narrative experiences mediated by VR. Using our classification methodology, we detected and successfully classified a total of 127 issues throughout the experiments performed with our 10 participants. A total of 74 of them were done on the designer side and 53 on the player side. The design issues and the performance issues were the most frequent ones for the two respective sides. A summary of the encountered issues extracted from the studies is presented in Table 4. In terms of the 3D design

experience, evaluated through the question 2 score presented in Table 2, all participants had either no experience at all or professional experience with 3D experience design. We observed an increase of 30% in the amount of issues for scenarios made by designers without 3D experiences, both in the design phase and usage phase. Scenarios made by experienced designers (D1,2,5,7,8) uncovered a total of 32 design-system issues and 23 player-system issues, while non-experienced designers' (D3,4,6,9,10) scenarios encountered 42 design-system issues and 30 player-system issues. In terms of VR headset experience, we observed different types of profiles, evaluated through the question 1 score presented in Table 2. The Figure 4
<figure><img src="image_8.png" /><figcaption>Figure 3: The interior (top) and exterior (bottom) 3D environments were assigned to designers for the creation of their scenarios. Both environments offered a 5-meter by 5-meter design space. Objects can be moved and renamed to fit the designer's intention. Participants assigned to the interior environments had to include in their scenario the following objects: a golf club, a golf ball, and any fallen objects. Participants assigned to the exterior environment had to include in their scenario the following objects: a key, a trashcan, and any sweet object.</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>Figure 4: Visualization of the relation between participant's VR headset experience score (presented on Table 2) and the amount of communication issues encountered. The left chart shows the design issues encountered by the designers, and the right chart shows the usage issues encountered by the players. The size of the circles represents the number of participants who exhibited a similar combination of scores and issues. We observe a -32% correlation between experience and design issues, and a -20% correlation between experience and usage issues.</figcaption></figure>

highlight negative correlations between VR headset experience score and the total number of issues, with a correlation of -32% during the design phase and -20% during the usage phase. This means

that participants with more experience with VR headsets encountered fewer communication issues when creating their scenario, and when trying other participants scenarios.

These data allow us to perform two types of analysis to investigate: (1) the correlation between two different categories of issues, and (2) the impact of a category issue on the proper narrative interpretation by the players.

Here, we present a few illustrative examples. We then go over a quantitative and qualitative analysis of the issues observed, the correlations between issues, and the final comprehension, as well as some relevant feedback collected from the participants.

## 5.1 Example scenarios

In the ten scenarios created by participants, we observed numerous issues with a variable impact on each of the actors: the designer, player, and system. Here, we illustrate three study cases to highlight how we identify and classify some issues encountered.

**Case 1: Gulfs in design and evaluation resulting in gulfs in scenario comprehension and narrative comprehension (D4,D8).**
The designers of the experiences 4 and 8 both used the interior environment for their scenario. It was experienced by the players D4P1, D4P2, D8P1, and D8P2. A part of the presented scenario and communication issues encountered is shown on Figure 5.

**Design phase:** Both designers had the same idea of creating a scenario to simulate an office worker experiencing burnout. They found this idea when seeing in the 3D scene the whiteboard object, which depicted a red arrow like the plunging of stock prices, along with the unordered objects on the room floor: "I want a scenario about a person working here, alone in the office, in burnout due to the company results. This person wants to break, burn, or throw things in their office's desk to let their anger out" (D4) "The person is depressed, is angry against the world, want to do crazy things" (D8). Both designers encountered system limitations as they couldn't physically burn or break objects. Instead, noticing the mini-golf field, they decided to create tasks to position the "laptop" (D8) or the "golden trophy" (D4), which they considered as "precious objects", on the "golf field" and "office desk" respectively, and then to use the objects "golf club" and "golf ball", to play golf on them. However, no textual information or narrative elements were used to inform the players of the meaning of these interactions. During their self-testing, neither of the designers realized the lack of communication on the meaning of the interactions, leaving the scenario largely open to the player's interpretation. In addition, D4 encountered an evaluation perception issue, as they didn't expect the gravity to apply on objects in the scenes when the experience start, resulting in the trophy falling on the ground

**Usage phase:** All players successfully performed the tasks in the scenario. D4P1 interpreted the task of placing the "golden trophy" on the "golf field" to "showcase them". D8P1 and D8P2 expressed confusion on the purpose of the scenario, questioning the presence of a purpose to begin with, as the sequence of tasks didn't make much sense to them. D4P2 was the only player able to correctly interpret the scenario as a burnout when seeing the plunging stocks and described the actions as violent. All three other players, due to the nature of VR allowing free physical and visual exploration of the environment didn't even notice plunging stocks on the whiteboard.
<figure><img src="image_10.png" /><figcaption>Figure 5: The designer D4 created a scenario, to simulate an office worker in burnout, where the player has to play golf on a precious trophy. The player D4P2 did the tasks properly, however, the player didn't understand the tasks the way the designer expected, interpreting this set of tasks as a desk tidying scenario.</figcaption></figure>

D4P1, seeing the trophy on the ground, interpreted this task as tidying up the office.

**Gulf evaluation:** In the post-usage, both D4 and D8 said they were aware that their scenario was hard to understand. D4 described their scenario as "*a bit far-fetched*" and was surprised to learn that D4P1 successfully interpreted their intention. After revealing the designer's intention to the players, D8P1 responded: "*the scenario is not really well-communicated, I don't think I could have guessed it at all*". In this case study, we thus observed a design issue (lack of context and information to communicate the meaning of interactions), compounded by an evaluation comprehension issue (designers didn't realize the lack of communication on interaction meaning during the evaluation phase). These two communication issues resulted in situation comprehension issues for 3 players (D4P2, D8P1, D8P2), who did not understand the meaning of the scenario at all. These communication issues resulted in a gulf between the designer intentions and player interpretation and in an incorrect narrative interpretation by D4P2, D8P1, and D8P2.

**Case 2: Design and evaluation gulfs resulting in system interaction gulfs (D5).** D5 used the exterior environment. It was experienced by the players D5P1, D5P2. A part of the presented scenario and communication issues encountered are shown on Figure 6.

**Design phase:** D5 wanted to create a simple scenario where the player has to go outside from home to collect some sweets for an afternoon snack. They created tasks to "open the door of the house using the access card", "go outside", and then tasks to bring home multiple "sweets" objects. The designer intended to use a phone-looking object as the card needed to open the door by renaming it "access card". However, during the design phase, D5 forgot to change the name of the object, resulting in a design issue. During the evaluation phase, D5 didn't notice that the name of the object didn't matching their initial design intention, resulting in an evaluation comprehension issue. Because of this, the final scenario asks the player to take the phone and open the door with it.

**Usage phase:** Both the players D5P1 and D5P2 performed the scenario's tasks correctly. Both players, however, didn't understand the reason the task asked them to use the phone as a key to open

a door, nor why the phone did actually open the door. The rest of the scenario was performed without further issues.

**Gulf evaluation:** In the post-usage, D5 realized the confusion with an object's name not matching the tasks performed. As a result, both D5P1 and D5P2 encountered situation comprehension issues. D5P1 said that presenting an interaction that makes no sense was probably for humoristic purposes. On the other hand D5P2 said that either the interaction made no sense or they had missed the meaning. This created a communication gulf between the designer, who wanted to simulate a normal interaction with an access card, and the players, who both interpreted this interaction making no sense. During the post-usage, looking at the replay videos, D5 first assumed that players understood the interaction's meaning. However, when provided with the player interpretations, they understood that there was a communication gulf around the phone interaction. D5 then said, "*I can understand the gulf; it's true that opening a door with a phone isn't common; that's fun, but I didn't expect them to go as far as believing it's an absurd scenario for humoristic purposes.*"

**Case 3: Multi-interaction between all four gulfs (D10).** D10 used the exterior environment, experienced by the players D10P1, D10P2. A part of the presented scenario and communication issues encountered is shown in Figure 7.

**Design phase:** D10, motivated by the curiosity to experiment with how physics behaves in VR, decided to create a scenario based solely on 3D interactions, where players had to find, take, and place objects. Players were first asked to pile up multiple objects found in the 3D environment. The first group of tasks involve placing the "cube" on the "pile of paper", then place the "ball" on the "cube" previously placed "pile of paper". Then the second group of tasks involves placing the "plant" and the "book" and place them on the "ground". Finally, a task was created to ask players to place the "golf club" in balance on top of the "book" and the "plant" previously placed on the "ground". Multiple design issues occurred. For example; in the task to take the "book", there were over ten book objects in the scene. This design issue was linked to an evaluation issue, as D10 didn't notice the issue, since they knew exactly which book to interact with.
<figure><img src="image_11.png" /><figcaption>Figure 6: D5 created a scenario, where players perform a number of interactions, one of which is to open a door with an access card. However, the designer forgot to rename the target object (the phone) as an access card. D5P1 completed the tasks, but did not understand the logic of opening a door with a phone, interpreting this interacting as humorous.</figcaption></figure>

<figure><img src="image_12.png" /><figcaption>Figure 7: The designer D10 created a scenario where players have to take a book and place it on the ground. However, the designer did not specify which book exactly to take and didn't notice this issue in the evaluation phase. The player D10P1 did understand the meaning of the task, however, due to the lack of detail on the book to take in a scene full of books, the player took some time to find the right one.</figcaption></figure>

**Usage phase:** Both players D10P1 and D10P2 successfully piled up the objects of the first group of tasks in the scenario. On the second group of tasks, they both encountered similar task comprehension issues: the task did not specify which book was concerned. Both players encountered a performance issue, trying multiple books without being sure which one was the right one for the task. Afterward, both players had difficulty knowing exactly how to place the "golf club", given the rich 3D interaction possibilities VR offers, and the designer's instruction weren't explicit enough.

**Gulf evaluation:** During the post-usage phase, D10 immediately realized the issue with the books: "I should have specified which book, something like: the blue book next to the plant". Multiple comparable issues were encountered during this scenario, linked to a lack of information given by the designer on how or which objects to interact with, notably on how to place the "book" and the "golf club". However, while these issues impacted the proper 3D interaction and task execution, they didn't have a strong impact on the narrative interpretation. The narrative of the scenario

of experimenting with piling objects was partially understood by D10P2 and fully understood by D10P1.

These case studies were selected to illustrate the difficulty of achieving a full understanding of the initial designer intentions, but found that a partial understanding is mostly feasible. The overall narrative comprehension is shown in Table 4, where 0 means "Fully understood", 1 means "Partially understood" and 2 means "Not understood at all". We also observed issues that fit under multiple classes, each potentially cascading into other issues in later stages of the model and potentially widening the gulf between the designer and the player.

## 5.2 Correlation between issues

While the prior case studies present a number of complicated causal relations between issues and communication gulfs, we now investigate in more detail the correlations between multiple types of issues, as well as their impact on narrative interpretation. For this purpose, we build a correlation matrix as shown in Table 3 to highlight potential correlations between two issues, where -1 means a
negative correlation, 0 an absence of correlation, and 1 a correlation. This includes the score on the interpretation of the narrative given by players to estimate if an issue have an impact on the scenario comprehension. Here we discuss the results obtained for each issue and their potential impact :

**Design limitation issue**: two minor negative correlations (-0.3 and -0.4) exist between design issues and the narrative interpretation, indicating that setting design limitations could reduce design issues and better narrative interpretation. We believe this can be linked to the fact that designers who encountered limitation issues often tried to simplify their intentions, lowering the chance of design issues, making the scenario easier to communicate and interpret. For example, D7 wanted to make players run around and eat objects in the scene, but due to the limitations, they changed the scenario to simply find hidden objects that looked edible.

**Design issues**: there is a minor correlation of 0.3 with performance issues, and a notable correlation of 0.6 with the narrative interpretation. This can be because a design issue is unconscious to the designer and affects the communicated intention through the system. For example, when D5 forgot to rename the "phone" as an "access card", the system communicates an erroneous message to the player, changing the interpretation of the players in an unexpected way.

**Evaluation perception issue** we observed two minor correlations of 0.3 to 0.4 between environment comprehension issues and performance issues. This can be explained by the fact that an evaluation perception issue corresponds to a gulf between the designer intention and what the system actually displays. For example, D3 noticed that a target object was not visible enough when testing. This can result in an environment comprehension issue that makes the task harder for the players (performance issues) requiring more effort and time than originally expected. We also observe a negative correlation of -0.7 with evaluation comprehension issues, meaning that when the designer is conscious of a gulf between what they wanted and what the system displayed, the unconscious gulf between their intentions and the actual scenario communicated was reduced. For example, D1 noticed during testing that placing an object (a "lollipop") on the ground in VR can be dangerous as it asks people to bend down with a VR headset on. Due to this issue, the designer foresaw that the task might be done slowly or incorrectly.

**Evaluation comprehension issues & Task comprehension issue**: No notable correlations were observed with these two issues in relation to other issues later in the process.

**Situation comprehension issue** A minor correlation of 0.4 is present between situation comprehension issues and narrative interpretation. This can be because this issue corresponds to the proper understanding of the purpose of the tasks. For example, D4P1 and D4P2 encountered a situation comprehension issue when they didn't understand why they opened a door with a phone, and for this reason, they misinterpreted the overall narrative the scenario was communicating.

**Environment comprehension issues** we observe a notable correlation of 0.6 with performance issues, which is expected. When a player doesn't properly understand what the task is, they will often solve the problem through trial and error. For example D6P1 and D6P2 had difficulty finding a "penguin" (a spaceship toy renamed "penguin"), resulting in slow performance.

**Performance issue** there is a correlation of 0.5 between performance issues and narrative interpretation. Two explanations are possible. First, players failing to perform the tasks properly could need more concentration to succeed, spending less attention understanding the task purpose. Another possibility is that this issue is the last one in the design-usage process, thus being the accumulated outcome of multiple previous issues. We observe a strong correlation of 0.8 between the total number of issues and the performance issue type, meaning that more issues often means more performance issues.

Finally the last row of Table 3 shows a 0.5 correlation between the total amount of issues observed and the narrative interpretation, meaning that a higher number of issues also correlates with a worse interpretation by the player. Situation comprehension, environment comprehension, and performance issues all show a correlation above 0.5 with the total, which identify these issues as key impacts on good comprehension and interaction, and directly linked to narrative interpretation. On the design phase, only design issues displays a correlation of 0.5 or more, making it the main factor impacting narrative interpretation from the design phase.

These analyses allow us to have a view on the impact of the different issues, and which ones have the strongest influence on proper narrative interpretation. Based on our analysis, we believe the design issues, the situation comprehension issues, environment comprehension issues and the performance issues are the most meaningful issues to investigate if we want to reduce the gulf designer-player, as they are the issues that seem to be the ones that cause the biggest numbers of communication issues, as they show strong correlations with other issues and with a low narrative interpretation score.

## 5.3 Post-usage insight

This study and analysis allowed us to highlight the issues, conscious or not, that designers and players encountered. All ten designers participated in the post-usage to observe the issues in their design and suggested improvements to make an experience that better matched or communicated their intention. Out of the 20 played scenarios, three of the scenarios' issues, according to designers, were of the fault of the players. For three other scenarios, designers felt there were no issues at all. For the remaining fourteen scenarios played, the designers identified one or more ways to avoid the issues, inspired by the observed players experiences, to better communicate their initial intention. We detail here a few examples of issues that seven designers discovered during the post-usage in their created scenarios, as well as their ideas to prevent these issues in their experiences.

D2 and D6 were very satisfied with their scenarios after seeing player replays. D2 observed the players and saw them do exactly what was intended, and felt nothing needed to be modified. On the other hand, D6 was stressed when they realized that the design and evaluation issues encountered during the design phase created an experience potentially too hard or frustrating compared to what was intended due to a lack of instructions. However, seeing a very good comprehension and enjoyment of the scenario by the players, where the lack of instructions created an escape-game like experience,
they concluded the scenario is better the way it is, even if it wasn't
what they intended originally.

Multiple designers noticed issues created due a lack of clarity in
the vocabulary used. D3, D9 and D10 realized some object names
and models were confusing, and most of their players encountered
environment comprehension issues (i.e., confusing brown table
with black, newspaper holder with newspaper, and Rubik's cube
with cube). Both D3 and D9 believe they should have added more
instructions or renamed the objects, while D10 believes the observed
issue is a rare case which shouldn't require changes. D10 and D4
encountered comparable issues: D4 agreed that the phone should
have been renamed as access card to more clearly communicate the
meaning of the task. D10 noticed players had trouble identifying
the correct "book" amongst multiple similar objects and suggested
that they could be more precise, such as "small blue book".

D4, D8 and D9 noticed a bad narrative interpretation as well as
multiple situation comprehension issues by their players, due to
a lack of instructions in their scenarios. Multiple tasks required
players to perform interactions (e.g., place the golden trophy on the
table, go to the entrance with the golf club) without giving context
as to why these tasks are relevant. All three believe this is the main
reason why their scenarios were misunderstood by their players.

From the 10 participants and analysis of over 100 issues on the
design and the usage of scenarios, this study highlights multiple
important issues to the designers. The way designers proposed
potential solutions to reduce the designer-player gulf in their VR
experiences based on the structured interview responses was also
very insightful, such as by adding more instructions to better com-
municate the scenario intention, renaming objects in the environ-
ment to avoid confusion, or changing the task order to prevent
potential incomprehension by the player.

6 Discussion

We have presented a communication model revisiting Norman's
seven stages of action, and a structured interview to evaluate and
characterize the different issues occurring during the design-usage
process of an immersive narrative experience, leading to gulfs of
comprehension. The challenge of reducing the gulf between de-
signer's intention and player's interpretation is significant for re-
search in highly immersive media: an experience not lived the
intended way could alter the way the data are collected, result-
ing in unsuccessful investigation of the research questions. This is
especially true in immersive environments, where observing and
understanding the experience can be more complex due to the free
physical and visual exploration possibilities offered by the 3D en-
vironment, as well as the possibility to use the body to physically
move and interact with objects.

## 6.1 Implications

Our user study demonstrates that our proposed communication
model improves the understanding of designer-system-player com-
munication gulfs. Specifically, we identified five important aspects
of improvement:

1. *Intention Communication*. The presented model, through the definition of a taxonomy, exposes the underlying problems in communicating a design intention to a player. By creating a road map

of communication issues that highlight multiple communication
gulfs, we provide opportunities for a deeper understanding of in-
tentionality, by introducing new perspectives on designers' and
players' intentions. This can enrich and extend existing commu-
nicability guidelines [36, 49] to improve designer-player intention
communication mediated by a system.

2. *User Roles*. Norman's original model [32] did not differentiate between the user roles interacting with the system. In the presented model, the designer and player gulfs are separately modeled, opening pathways for a more fine-grained classification of issues occurring during the interaction and evaluation phases of the two user roles.

3. *Communication Issue Classification*. To collect information on the underlying communication issues, we proposed a structured interview that covers all phases. Using this structured interview, we have illustrated the possibility to find and classify qualitative data about the designer-system-player communication. We successfully categorized all 127 identified communication issues into one of the eight taxonomy categories, with between 6 to 30 issues identified per category. Based on the successful classification of all encountered issues, we believe that the taxonomy suggested by the model is sufficient to understand the designer-system-player communication gulf in the context of immersive experiences.

4. *Communication Issue Correlations*. By analyzing the data obtained from the study, this model has allowed us to observe the correlation between communication issues, and observed how compounding issues result in a wider designer-player gulf. We mapped 9 co-occurrences of issues across the designer and player gulf (such as *environment comprehension issues - performance issues* and *evaluation perception issues - environment comprehension issues*) that could not previously be observed with existing models and studies. In particular, environment comprehension issues show a strong correlation of 0.5 with performance issues. In turn, performance issues are the most frequently encountered usage issues (30 times), showing a correlation of 0.5 with incorrect narrative interpretation. This correlational analysis supports the identification of the categories of issues with a strong impact on the player's interpretation, and the categories of issues leading to the creation of more issues by co-occurrences.

5. *Communication Issues Prevention*. We identified critical areas and potential solutions to improve designers' understanding and prevention of designer-system-player communication issues in immersive environments. For example, during the design phase, the most impactful identified problems are design issues, encountered 30 times, showing a strong correlation of 0.6 with incorrect narrative interpretation, and a correlation of 0.5 with an increase in the total number of issues. The observed design issues underscore the importance of providing designers with additional guidance on the design possibilities offered by the system. By supporting designers, we will greatly enhance the designer-system-player communication to reduce the gulf between design intention, and player interpretation.

Nonetheless, the proposed structured interview is an instrument
to collect and analyze data, that can be replaced or coupled with
varied evaluation methods. This communication model and methodology possess the potential to be complemented with existing user study methodologies to further refine designer-system-player communication. Looking ahead, we are eager to explore several promising directions:

* **Designer's experience design phase:** usability heuristics [45] are an efficient way to avoid design issues, by providing general principles to designers. User sampling [19] could support an automatic collection of the designer's thoughts, directly implemented in the system, to identify issues in the design, notably linked to design limitation issues.

* **Designer's experience evaluation phase:** cognitive walks through [10, 24] are efficient, notably for the prevention of evaluation perception issues by anticipating potential issues encountered during a scenario. Think-aloud protocols [54] are an efficient solution to encourage designers to explicitly spell out any intention when evaluating, to detect a potential evaluation gulf.

* **Player's environment understanding phase:** cognitive mapping [18] can be particularly interesting to explore the player thought and to define if there is a mismatch between the designer intention and the player comprehension. Here again, think-aloud protocols [54] are an efficient solution to encourage players to explicitly explain their interpretations to pinpoint potential comprehension issues.

* **Player's environment interaction phase:** surveys such as the System Usability Scale (SUS), NASA-TLX, or more recently VR System Usability Questionnaire (VRSUQ) [17] could be used to gain insight on players' interactions. The collection and analysis of logs such as task time, or complex data like eye tracking and body movement [9, 37] can support the observation of environment comprehension and performance issues. These logs and behavioral data will allow designer to finely observe the player experience and understand how to improve the experience. This same collected data can enable the usage of system dynamics modeling coupled with guidance methodologies [28, 38], to reorient the player experience when they encounter difficulties or deviate from the expected experience. Possibilities to create personalized guidance are particularly important in immersive narrative experiences, where immersion and embodiment offer large possibilities of interaction exploration, increasing the chances for the user experience to deviate from the designer's intention.

By pursuing these directions, we aim to enhance the synergy between designers, systems, and players, ultimately advancing both research and practical applications in the domain of immersive narrative experiences. While the presented communication model and methodology were built to address challenges encountered in the context of highly immersive media, we believe this work could be generalized to other media in the future.

## 6.2 Limitations

The structured interview proposed allowed us to understand a wide range of comprehension and communication issues through correlational analysis. The results of using this questionnaire are

promising, and we plan to perform further studies in the future to determine the generalization of this work with more applications.

The correlation analysis presented depends on the population and the context: the observed correlations can be impacted by the environments used, by the constraints of the protocol, or by the participant's experience in 3D design. It can also be affected by factors that we did not take into account, such as the 3D gaming experience. We observed a clear impact of the design experience on communication issues; however, most of the participants with 3D design experience also had the most experience with VR systems. More research is needed to determine the precise impact of these factors on the amount of encountered issues, and their impact on the narrative interpretation for a given context or population.

A limitation linked to our model is the lack of representation of choices: when designing or experiencing a scenario, the designer can consciously decide to change the initial intention, or the player can decide to not follow the presented scenario. While this is a case we did not encounter in our study, it could complexify the analysis. This would result, for example, in a gulf between what the designer initially intended to create and what they actually created, but without any issues collected and classified to explain this gulf. A way to improve this could be to explore how to include the reiterating nature of the communication model in the taxonomy: in the design of a system, the design intention is usually not fixed and can evolve over time.

Another limitation lies in the focus of our analysis on the proper comprehension of the designer's intention by the player, presented as the narrative interpretation. The interaction performed could be further investigated by examining if the player did interact with the system in the way the designer expected them to. This is a case encountered with D10, who wanted players to place a plant on the ground, but both D10S1 and D10S2 dropped the plant on the ground instead, not satisfying the designer. Our communication model allows the observation and classification of these elements, however deeper investigations are needed to precisely characterize the articulatory distance between the designer's intended usage and the real usage, in the same way that we investigated the semantic distance by defining a score of narrative interpretation.

## 7 Conclusion

In this paper, we present our communication model, which aims at describing the design-usage process of a VR experience and the issues in communication and comprehension that can affect proper communication of the narrative. We presented our taxonomy as well as our structured interview, which was revealed to be suitable for our user study, allowing the identification and categorization of 127 issues among the eight defined categories. Through the analysis, we observed the issues and their impacts on other issues as well as on the narrative interpretation. The most meaningful categories of issues were design issues, situation comprehension issues, environment comprehension issues, and performance issues. We received a majority of positive feedback from the designers who got to identify the issues in their design, which helped them determine means to improve their experiences. While the presented communication model is built to specify the design-usage process in a virtual reality
environment, we believe this model could be extended and used for
other interactive systems used to communicate a narrative.

Acknowledgments

This work has been partially supported by the French National
Research Agency through the ANR CREATTIVE3D project ANR-21-
CE33-0001 and UCAJEDI Investissements d'Avenir ANR-15-IDEX-
0001 (IDEX reference center for extended reality XR²C²).

References

[1] Ruth Aylett and Sandy Louchart. 2003. Towards a narrative theory of virtual reality. *Virtual Reality* 7 (2003), 2–9.

[2] J Benthall. 1973. BODY AS A MEDIUM OF EXPRESSION. , 183–185 pages.

[3] Frank Biocca. 1997. The cyborg's dilemma: Progressive embodiment in virtual environments. *Journal of computer-mediated communication* 3, 2 (1997), JCMC324.

[4] Matthew L. Bolton. 2017. A task-based taxonomy of erroneous human behavior. *International Journal of Human-Computer Studies* 108 (Dec. 2017), 105–121. https://doi.org/10.1016/j.ijhcs.2017.06.006

[5] Janaina Cavalcanti, Victor Valls, Manuel Contero, Lien vers un site externe, celui-ci s'ouvrira dans une nouvelle fenêtre, David Fonseca, Lien vers un site externe, et celui-ci s'ouvrira dans une nouvelle fenêtre. 2021. Gamification and Hazard Communication in Virtual Reality: A Qualitative Study. *Sensors* 21, 14 (2021), 4663. https://doi.org/10.3390/s21144663 Num Pages: 4663 Place: Basel, Switzerland Publisher: MDPI AG.

[6] Rory MS Clifford, Sungchul Jung, Simon Hoermann, Mark Billinghamurst, and Robert W Lindeman. 2019. Creating a stressful decision making environment for aerial firefighter training in virtual reality. In *2019 IEEE Conference on virtual reality and 3d user interfaces (VR)*. IEEE, Osaka, Japan, 181–189.

[7] Nathan Crilly, David Good, Derek Matravers, and P. John Clarkson. 2008. Design as communication: exploring the validity and utility of relating intention to interpretation. *Design Studies* 29, 5 (Sept. 2008), 425–457. https://doi.org/10.1016/j.deustud.2008.05.002

[8] Brendan David-John, Candace Peacock, Ting Zhang, T. Scott Murdison, Hrvoje Benko, and Tanya R. Jonker. 2021. Towards gaze-based prediction of the intent to interact in virtual reality. In *ACM Symposium on Eye Tracking Research and Applications*. ACM, Virtual Event Germany, 1–7. https://doi.org/10.1145/3448018.3458008

[9] Brendan David-John, Candace Peacock, Ting Zhang, T Scott Murdison, Hrvoje Benko, and Tanya R Jonker. 2021. Towards gaze-based prediction of the intent to interact in virtual reality. In *ACM symposium on eye tracking research and applications*. ACM, Virtual Event Germany, 1–7.

[10] Ioannis Doumanis and Daphne Economou. 2019. Using cognitive walkthrough and hybrid prototyping to gather user requirements in early design virtual reality prototypes. In *Immersive Learning Research Network: 5th International Conference, iLRN 2019, London, UK, June 23–27, 2019, Proceedings* 5. Springer, Springer, London, 234–246.

[11] Paul MG Emmelkamp and Katharina Meyerbröker. 2021. Virtual reality therapy in mental health. *Annual review of clinical psychology* 17, 1 (2021), 495–519.

[12] Juan M Ferreira, Silvia T Acuña, Oscar Dieste, Sira Vegas, Adrián Santos, Francy Rodriguez, and Natalia Juristo. 2020. Impact of usability mechanisms: An experiment on efficiency, effectiveness and user satisfaction. *Information and Software Technology* 117 (2020), 106195.

[13] W. H. Gibson, E. D. Megaw, M. S. Young, and E. Lowe. 2006. A taxonomy of human communication errors and application to railway track maintenance. *Cognition, Technology & Work* 8, 1 (Jan. 2006), 57–66. https://doi.org/10.1007/s10111-005-0020-x

[14] Teresa Hirzle. 2023. *Digital Eye Strain in Virtual Reality Head-Mounted Displays: Properties, Causes, Solutions, and Perspective*. Ph. D. Dissertation. Universität Ulm.

[15] Robin Hunicke, Marc LeBlanc, Robert Zubek, et al. 2004. MDA: A formal approach to game design and game research. *AAAI Workshop on Challenges in Game AI* 4 (2004), 1722.

[16] Rogério Junior and Frutuoso Silva. 2021. Redefining the mda framework—the pursuit of a game design ontology. *Information* 12, 10 (2021), 395.

[17] Yong Min Kim and Ilsun Rhiu. 2024. Development of a virtual reality system usability questionnaire (VRSUQ). *Applied Ergonomics* 119 (2024), 104319.

[18] Rob Kitchin and Scott Freundschuh. 2018. Cognitive mapping. In *Cognitive mapping*. Routledge, London, 1–8.

[19] Reed Larson, Mihaly Csikszentmihalyi, et al. 1983. The experience sampling method. *New directions for methodology of social and behavioral science* 15, 15 (1983), 41–56.

[20] Marta Larusdottir, David Thue, and Hannes Högni Vilhjálsson. 2019. User Evaluations of Virtually Experiencing Mount Everest. In *Human-Centered Software Engineering: 7th IFIP WG 13.2 International Working Conference, HCSE 2018*,

Sophia Antipolis, France, September 3–5, 2018, Revised Selected Papers 7. Springer, HCSE 2018, Sophia Antipolis, France, 294–307.

[21] Page Laubheimer. 2015. Preventing user errors: avoiding unconscious slips. *Nielsen Norman Group* 1, 1 (2015), 1.

[22] Songpo Li and Xiaoli Zhang. 2017. Implicit Intention Communication in Human-Robot Interaction Through Visual Behavior Studies. *IEEE Transactions on Human-Machine Systems* 47, 4 (Aug. 2017), 437–448. https://doi.org/10.1109/THMS.2017.2647882

[23] Wanwan Li, Javier Talavera, Amilcar Gomez Samayoa, Jyh-Ming Lien, and Lap-Fai Yu. 2020. Automatic Synthesis of Virtual Wheelchair Training Scenarios. In *2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)*. IEEE, Virtual Event, 539–547. https://doi.org/10.1109/VR46266.2020.00075 ISSN: 2642-5254.

[24] Qiaoxi Liu and Anthony Steed. 2021. Social virtual reality platform comparison and evaluation using a guided group walkthrough method. *Frontiers in Virtual Reality* 2 (2021), 668181.

[25] Johan Lundin, Anders Lundström, Jan Gulliksen, Joakim Blendulf, Kersti Ejeby, Hedda Nyman, Daniel Björkander, and Erik Hedman-Lagerlöf. 2022. Using 360-degree videos for virtual reality exposure in CBT for panic disorder with agoraphobia: a feasibility study. *Behaviourial and Cognitive Psychotherapy* 50, 2 (2022), 158–170.

[26] Michaelia Matthews, Girish Chowdhary, and Emily Kieson. 2017. Intent Communication between Autonomous Vehicles and Pedestrians. http://arxiv.org/abs/1708.07123 arXiv:1708.07123 [cs].

[27] Aline Menin, Rafael Torchelsen, and Luciana Nedel. 2022. The effects of VR in training simulators: Exploring perception and knowledge gain. *Computers & Graphics* 102 (2022), 402–412.

[28] Lasse T Nielsen, Matias B Møller, Sune D Hartmeyer, Troels CM Ljung, Niels C Nilsson, Rolf Nordahl, and Stefania Serafin. 2016. Missing the point: an exploration of how to guide users' attention during cinematic virtual reality. In *Proceedings of the 22nd ACM conference on virtual reality software and technology*. VRST, New YorkNYUnited States, 229–232.

[29] Britta Nordin Forsberg, Anders Lundström, and Jan Gulliksen. 2023. VR for HR-A Case Study of Human Resource Development Professionals Using Virtual Reality for Social Skills Training in the Workplace. In *IFIP Conference on Human-Computer Interaction*. Springer, IFIP, Barcelona, Spain, 231–251.

[30] Donald Norman. 2008. Design as Communication.

[31] Donald Norman. 2010. Systems Thinking: A Product Is More Than the Product.

[32] A Norman Donald. 2013. *The design of everyday things*. MIT Press, United States.

[33] Jiwon Oh and Gerard J. Kim. 2019. The Effect of Onomatopoeia to Enhancing User Experience in Virtual Reality. In *Virtual, Augmented and Mixed Reality. Multimodal Interaction (Lecture Notes in Computer Science)*, Jessie Y.C. Chen and Gino Fragomeni (Eds.). Springer International Publishing, Cham, 143–152. https://doi.org/10.1007/978-3-030-21607-8_11

[34] Tim Paek. 2003. Toward a taxonomy of communication errors. In *ISCA Tutorial and Research Workshop on Error Handling in Spoken Dialogue Systems*, Vol. 1. ISCA Archive, Vaud, Switzerland, 1.

[35] Randy Pausch, Dennis Proffitt, and George Williams. 1997. Quantifying immersion in virtual reality. In *Proceedings of the 24th annual conference on Computer graphics and interactive techniques*. SIGGRAPH, New York, NY, United States, 13–18.

[36] Raquel O Prates, Clarisse S De Souza, and Simone DJ Barbosa. 2000. Methods and tools: a method for evaluating the communicability of user interfaces. *interactions* 7, 1 (2000), 31–38.

[37] Florent Robert, Hui-Yin Wu, Lucile Sassatelli, and Marco Winckler. 2024. Task-based methodology to characterise immersive user experience with multivariate data. In *2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)*. IEEE, Orlando, United States, 722–731.

[38] Lucile Sassatelli, Anne-Marie Pinna-Déry, Marco Winckler, Savino Dambra, Giuseppe Samela, Romaric Pighetti, and Ramon Aparicio-Pardo. 2018. Snap-changes: a dynamic editing strategy for directing viewer's attention in streaming virtual reality videos. In *Proceedings of the 2018 International Conference on Advanced Visual Interfaces*. AVI, Grosseto, Italy, 1–5.

[39] Valarie M Schroeder and LT Adam T Biggs. 2018. Fatigue and Eye Strain Associated with Virtual Reality and Augmented Reality Systems. *Adapting Virtual Reality and Augmented Reality Systems for Naval Aviation Training* 4 (2018), 104.

[40] Martijn J Schuemie, Peter Van Der Straaten, Merel Krijn, and Charles APG Van Der Mast. 2001. Research on presence in virtual reality: A survey. *Cyberpsychology & behavior* 4, 2 (2001), 183–201.

[41] Mel Slater. 2018. Immersion and the illusion of presence in virtual reality. *British journal of psychology* 109, 3 (2018), 431–433.

[42] Clarisse Sieckenius De Souza and Carla Faria Leitão. 2009. *Semiotic Engineering Methods for Scientific Research in HCI*. Morgan & Claypool Publishers, San Rafael, California.

[43] Vinicius Souza, Anderson Maciel, Luciana Nedel, and Regis Kopper. 2021. Measuring presence in virtual environments: A survey. *ACM Computing Surveys (CSUR)* 54, 8 (2021), 1–37.
[44] Maximilian Speicher, Katy Lewis, and Michael Nebeling. 2021. Designers, the stage is yours! medium-fidelity prototyping of augmented & virtual reality interfaces with 360theater. *Proceedings of the ACM on human-computer interaction 5*, EICS (2021), 1-25.

[45] Alistair Sutcliffe and Brian Gault. 2004. Heuristic evaluation of virtual reality applications. *Interacting with computers* 16, 4 (2004), 831-849.

[46] Heidi Sveistrup. 2004. Motor rehabilitation using virtual reality. *Journal of neuroengineering and rehabilitation* 1 (2004), 1-8.

[47] Katie Salen Tekinbas and Eric Zimmerman. 2003. *Rules of play: Game design fundamentals*. MIT press, Massachusetts, United States.

[48] Leimin Tian and Sharon Oviatt. 2021. A Taxonomy of Social Errors in Human-Robot Interaction. *ACM Transactions on Human-Robot Interaction* 10, 2 (June 2021), 1-32. https://doi.org/10.1145/3439720

[49] Stefano Valtolina, Ricardo Anibal Matamoros, and Francesco Epifania. 2023. Methods for Evaluating Conversational Agents' Communicability, Acceptability and Accessibility Degree. In *IFIP Conference on Human-Computer Interaction*. Springer, IFIP, Bologna, Italy, 372-382.

[50] Nicolas Vignais, Benoit Bideau, Cathy Craig, Sébastien Brault, Franck Multon, and Richard Kulpa. 2009. Virtual Environments for Sport Analysis: Perception-Action Coupling in Handball Goalkeeping. *International Journal of Virtual Reality* 8, 4 (Jan. 2009), 43-48. https://doi.org/10.20870/IJVR.2009.8.4.2748

[51] Loup Vuarnesson, Dionysios Zamplaras, Julien Laroche, Joseph Dumit, Clint Lutes, Asaf Bachrach, and Francois Garnier. 2021. Shared Diminished Reality: A New VR Framework for the Study of Embodied Intersubjectivity. *Frontiers in Virtual Reality 2* (2021), 646930. https://www.frontiersin.org/articles/10.3389/frmv.2021.646930

[52] Josef Wolfartsberger and Daniel Niedermayr. 2020. Authoring-by-Doing: An- imating Work Instructions for Industrial Virtual Reality Learning Environ- ments. In *2020 IEEE Conference on Virtual Reality and 3D User Interfaces Ab- stracts and Workshops (VRW)*. IEEE VR 2020, Virtual Event, 173-176. https://doi.org/10.1109/VRW50115.2020.00038

[53] Erwin Wu, Mitski Piekenbrock, Takuto Nakumura, and Hideki Koike. 2021. SPin- Pong - Virtual Reality Table Tennis Skill Acquisition using Visual, Haptic and Temporal Cues. *IEEE Transactions on Visualization and Computer Graphics* 27, 5 (May 2021), 2566-2576. https://doi.org/10.1109/TVCG.2021.3067761 Conference Name: IEEE Transactions on Visualization and Computer Graphics.

[54] Xuesong Zhang and Adalberto L Simeone. 2022. Using the think aloud protocol in an immersive virtual reality evaluation of a virtual twin. In *Proceedings of the 2022 ACM Symposium on Spatial User Interaction*. SUI, New York, NY, United States, 1-8.

[55] Jiayan Zhao and Alexander Klippel. 2019. Scale-unexplored opportunities for immersive technologies in place-based learning. In *2019 IEEE Conference on virtual reality and 3d user interfaces (VR)*. IEEE, Osaka, Japan, 155-162.