# Understanding the Challenges of Immersive Authoring for End-Users

Edwige Chauvergne

► To cite this version:

Edwige Chauvergne. Understanding the Challenges of Immersive Authoring for End-Users. IHM 2023 - 34ème Conférence Internationale Francophone sur l'Interaction Humain-Machine, AFIHM; Université de Technologie de Troyes, Apr 2023, Troyes, France. [hal-04042308](https://hal.science/hal-04042308)

HAL Id: hal-04042308

https://hal.science/hal-04042308

Submitted on 23 Mar 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Understanding the Challenges of Immersive Authoring for End-Users

Edwige Chauvergne

Inria de l'Université de Bordeaux
Bordeaux, France
edwige.chauvergne@inria.fr

## ABSTRACT

Over the years, virtual reality (VR) gains in popularity. Although VR becomes more affordable, VR content creation is restricted to expert developers which limits both its expansion and adoption. Enabling the authoring of VR experiences to non-developers would empower end-users and allow them to design their own VR applications. To do so, it is however necessary to support the onboarding process in VR. In this paper, we present our work and research direction planned for this PhD. We implemented a prototype for immersive authoring of interactive experiences, investigated VR onboarding, and provided a framework. Finally, we plan on exploring the authoring of data-driven storytelling.

## CCS CONCEPTS

* Human-centered computing → Virtual reality.

## KEYWORDS

Virtual reality, end-users, onboarding

IHM'23, April 03-07, 2023, Troyes, France

© 2023 Copyright held by the owner/author(s).

This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in IHM'23 : Extended Proceedings of 34ème conférence Francophone sur l'Interaction Humain-Machine, April 03-07, 2023, Troyes, France.
## INTRODUCTION

The use of virtual reality (VR) is rising. On one side VR headsets have become more affordable, and on the other side, VR applications have multiplied. As VR is no longer reserved for experts in the domain, there is a need to support the authoring of interactive VR experiences by non-developers. The immersive authoring of such interaction by demonstration appears as a promising method for end-users. Yet, before being able to produce content, end-users need to learn how to use VR in general and the VR applications they use. This onboarding process to new VR applications requires assistance tools. Finally, Immersive Analytics benefits from an immersive three-dimensional environment to explore and analyze data. Yet, visualizations still mainly remain objects that users examine. Authoring tools for data-driven storytelling would support the creation of new types of visualization and examination processes. In this paper, we summarize our work on end-user authoring of interactive virtual reality experiences, as well as VR onboarding. Then, we introduce our future research direction regarding the authoring of immersive data-driven storytelling.

## AUTHORING INTERACTIVE EXPERIENCES IN VIRTUAL REALITY THROUGH DEMONSTRATION

Enabling the prototyping and authoring of immersive VR experiences allows non-developers to appropriate the use of VR and create new content. Enabling development to non-experts has been broadly explored on desktop, using for instance visual programming like Scratch <fnref n="1" />. However, to author such interactive prototypes without code in VR is still an open research question. Various methods have been investigated such as simplified desktop interfaces [11], visual programming [12], and authoring by demonstration [3, 7, 10]. However, these approaches do not handle yet the creation of complex interactive experiences. Immersive authoring by demonstration appears as a promising creation process for end-users as its users directly manipulate objects in context and can immediately visualize and experience the created interactions. In order to better understand the challenges to the immersive authoring of interactive experiences by demonstration, we implemented an authoring tool, created two use cases, and organized a feedback session [5].

We implemented a prototype for an authoring tool on HTC Vive using Unity. Using this tool, users can record actions. As they start the recording, they can directly manipulate objects in the virtual environment to demonstrate actions. Users can create eleven types of actions using three different recording modes. After recording actions, users can link actions as triggers and effects (see Figure 1) and possibly inspect actions to modify trajectories. Finally, users can directly test the interactions they authored by activating a test mode.

In order to identify the main challenges in both our prototype and VR authoring by demonstration in general we created two use cases and organized a feedback session. The two use cases were an

<figure><img src="image_2.png" /><figcaption>Figure 1: Events are visualized with a floating bubble. In this simple example the Collision of the sphere with the user's hand, makes the cube become invisible (Hide)</figcaption></figure>

<footnote n="1">¹https://scratch.mit.edu/</footnote>
<table><caption>Figure 2: Main challenges in immersive authoring of interactive experiences by demonstration</caption><tr><td>C1</td><td>Handling impossible demonstrations</td></tr><tr><td>C2</td><td>Demonstration of abstract concepts</td></tr><tr><td>C3</td><td>Processing ambiguities</td></tr><tr><td>C4</td><td>Creating and visualizing logic</td></tr><tr><td>C5</td><td>Providing assistance to users</td></tr></table>

<table><caption>Figure 3: Framework for general VR onboarding.</caption><tr><td colspan="3">Onboarding Context</td></tr><tr><td>Out of context</td><td>Formation specific context</td><td>Context-aware</td></tr><tr><td colspan="3">Degree of flexibility</td></tr><tr><td>Linear and rigid</td><td>Skippable and rigid</td><td>Skippable and flexible</td><td>Focus on specific functionalities</td></tr><tr><td colspan="3">Degree of coverage</td></tr><tr><td>Basic functionalities</td><td>All the functionalities without level discrimination</td><td>Level specific training</td></tr><tr><td colspan="3">Inputs onboarding</td></tr><tr><td>Outside VR</td><td>With visual cues</td><td>No onboarding</td></tr><tr><td colspan="3">Degree of interactivity</td></tr><tr><td>Observer</td><td>Partially enabled functionalities</td><td>Freely interact with the virtual environment</td></tr><tr><td colspan="3">Type of assistance</td></tr><tr><td>Instructions</td><td colspan="2">Demonstration</td></tr><tr><td colspan="3">Degree of feedback</td></tr><tr><td>Implicit feedback</td><td>Explicit feedback</td><td>Corrective feedback</td></tr><tr><td colspan="3">Instruction modalities</td></tr><tr><td>Visual</td><td>Sound</td><td>Haptic</td></tr></table>

escape game scenario and a game inspired by Beat Saber<fnref n="2" />. Finally, the feedback session involved six participants (4 female, 2 male) and consisted of a demonstration of the Escape game scenario, a training phase in which participants were asked to author two short interactive experiences, and a free authoring phase, followed by a NASA-TLX questionnaire and a semi-structured interview. Based on the use cases' observations and the feedback sessions' results we identified five main challenges for authoring interactive experiences by demonstration, depicted in figure 2.

## LEARNING VIRTUAL REALITY

Renz et al [8] define the onboarding process as *the sum of methods and elements helping a new user to become familiar with a digital product*. During the user study, we noticed that assisting participants through the onboarding process happened to be difficult.VR is still new to many users and still lacks conventions which creates many discrepancies between applications. As VR application multiply, they are expected to become more powerful and thus, to some extent, more complex. Using these complex applications involves going through a learning phase even for regular VR users. Thus, onboarding is necessary for VR, yet Ashtari et al [1] mention the fact that users need to go through a long and cumbersome onboarding process during user testing, which not only is time-consuming but also adds a significant cognitive charge to users, making it important to increase research in this domain.

This onboarding process can take several forms. Learners can follow a tutorial which usually is either a 2D video on desktop or a tutorial integrated into the VR application. The onboarding can also be supervised by an expert instructor. This type of onboarding is less accessible than tutorials, however, it benefits from high flexibility.

Based on Bach and Scapin's ergonomic criteria [2], we reviewed 21 immersive VR tutorials from a broad range of applications. This review allowed us to identify three main challenges for tutorials: 1) Intelligibility, 2) Feedback and feedforward, and 3) Adaptability and accessibility.

In order to get an overview of the challenges and practices in instructor-based VR onboarding we interviewed 15 experts. 11 of these experts performed VR onboarding in a research context and 4 of them for application training purposes. Based on the transcriptions of the interviews, we performed a thematic analysis [4] and identified three main themes: 1) the novelty of virtual reality, 2) the lack of awareness, and 3) the communication barrier in virtual reality.

Based on our tutorial review and our expert interviews we created a framework for general onboarding and provided guidelines for general and instructor-based onboarding [6]. Our framework comprises eight dimensions and is depicted in figure 3. We expect this framework to become a support to researchers and onboarding designers as (1) it has descriptive power, (2) it can be used to compare designs, and (3) it supports generating new designs using the Zwicky box method [9]. However, this framework is not meant to be prescriptive, as further research is required to understand which design better fits a situation. We also provided a set of guidelines for general onboarding (see figure 4)

<footnote n="2">²https://beatsaber.com/</footnote>
<table><caption>Figure 4: Guidelines for general onboarding</caption><tr><td>G1</td><td>Instructions need to remain available</td></tr><tr><td>G2</td><td>Provide corrective feedback</td></tr><tr><td>G3</td><td>Support the use of visual cues</td></tr><tr><td>G4</td><td>Use multiple instruction modalities</td></tr><tr><td>G5</td><td>Favor context-based onboarding</td></tr></table>

<table><caption>Figure 5: Guidelines for instructor-based onboarding</caption><tr><td>G6</td><td>Do not rush the hardware and inputs onboarding</td></tr><tr><td>G7</td><td>Experts need at least a mirrored view</td></tr><tr><td>G8</td><td>Instruction is not collaboration</td></tr></table>

meant for both tutorials and instructor-based onboarding, and a second set of guidelines specific to instructor-based onboarding (see figure 5).

## AUTHORING NEW TYPES OF VISUALIZATION FOR DATA-DRIVEN STORYTELLING

We believe that authoring tools for end-users would support the appearance of new types of visualizations. Current visualizations in immersive analytics mostly rely on objects representing the data enabling users to visually quantify and interpret the data. Yet, VR is a promising tool to enable new ways of representing and interpreting the data and promoting empathic reactions.

Data-driven storytelling is the practice of using narratives based on data in order to convey information to a non-expert public. To continue this Ph.D., we first plan on exploring the use of data point embodiment in data-driven storytelling as an example of a new visualization method that could be authored. Embodiment has been used in previous studies to raise empathy as it helps its users to take a different perspective. However, such embodied experiences do not share any quantitative data such as the scale of a problem. On the opposite, data-driven storytelling based on traditional graphical visualization supports the understanding of quantitative data, yet raising empathy is more complicated as people become a point among others. In traditional graphical visualization, the user only sees the visualization from outside and does not embody anything. With data point embodiment, users become a data point and thus embody a person from the data.

We aim at implementing an authoring tool for this kind of visualization. However, we first need to study its impact on the users' experience. We plan on implementing a data point embodiment experience with two possible scenarios, the Titanic's sinking, and the Mediterranean refugee crisis. Then, we wish to study the impact of this type of visualization through a user study.

We expect this experience to support both empathy raising and data understanding. We hypothesized that embodied experience raises more empathy than the data point embodiment but fewer data understanding. We also expect traditional visualization to better support data understanding but to raise fewer empathy. We expect this type of visualization to become a bridge between data understanding and raising empathy.

## CONCLUSION

Making VR headsets affordable and broadly available is only the first step to supporting the use of VR by end-users in various domains. Research and improvements are required to support the onboarding and authoring of interactive experiences. We hope this PhD will provide some guidelines to reach this goal. Although this work targets virtual reality, a broad range of our work is suitable for augmented reality. Indeed, VR and AR share several particularities and challenges such as the difficulty for outsiders to understand what the user is experiencing.
# ACKNOWLEDGMENTS

I thank my two supervisors, Arnaud Prouzeau and Martin Hachet, who provide great advise and support during this Ph.D, as well as the Potioc Team.

# REFERENCES

[1] Narges Ashtari, Andrea Bunt, Joanna McGrenere, Michael Nebeling, and Parmit K. Chilana. 2020. Creating Augmented and Virtual Reality Applications: Current Practices, Challenges, and Opportunities. In *Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20)*. Association for Computing Machinery, New York, NY, USA, 1-13.
https://doi.org/10.1145/3313831.3376722

[2] Cédric Bach and Dominique Scapin. 2005. Critères Ergonomiques pour les Interactions Homme-Environnements Virtuels : définitions, justifications et exemples. (01 2005).

[3] Lee Beever, Serban Pop, and Nigel W. John. 2020. LevelEd VR: A virtual reality level editor and workflow for virtual reality level design. In *2020 IEEE Conference on Games (CoG)*. 136-143. https://doi.org/10.1109/CoG47356.2020.9231769

[4] Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. *Qualitative Research in Psychology* 3, 2 (2006), 77-101. https://doi.org/10.1191/1478088706qp063oa arXiv:https://www.tandfonline.com/doi/pdf/10.1191/1478088706qp063oa1

[5] Edwige Chauvergne, Martin Hachet, and Arnaud Prouzeau. 2023. Editer des expériences interactives et immersives en utilisant la programmation par démonstration. In *IHM '23: Proceedings of the 34th Conference on l'Interaction Humain-Machine (IHM '23)*. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3583961.3583981

[6] Edwige Chauvergne, Martin Hachet, and Arnaud Prouzeau. 2023. User Onboarding in Virtual Reality: An investigation of current practices. In *Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI '23)*. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3544548.3581211

[7] Gary Ng, Joon Gi Shin, Alexander Plopski, Christian Sandor, and Daniel Saakes. 2018. Situated Game Level Editing in Augmented Reality. In *Proceedings of the Twelfth International Conference on Tangible, Embedded, and Embodied Interaction (TEI '18)*. Association for Computing Machinery, New York, NY, USA, 409-418. https://doi.org/10.1145/3173225.3173230

[8] Jan Renz, Thomas Staubitz, Jaqueline Pollack, and Christoph Meinel. 2014. Improving the Onboarding User Experience in MOOCs.

[9] Tom Ritchey. 1998. Fritz Zwicky, Morphologie and Policy Analysis. In *16th EURO Conference on Operational Analysis. EURO, NA, 12.*

[10] Tianyi Wang, Xun Qian, Fengming He, Xiyun Hu, Yuanzhi Cao, and Karthik Ramani. 2021. GesturAR: An Authoring System for Creating Freehand Interactive Augmented Reality Applications. In *The 34th Annual ACM Symposium on User Interface Software and Technology (UIST '21)*. Association for Computing Machinery, New York, NY, USA, 552-567.
https://doi.org/10.1145/3472749.3474769

[11] E. Yigitbas, J. Klauke, S. Gottschalk, and G. Engels. 2021. VREUD - An End-User Development Tool to Simplify the Creation of Interactive VR Scenes. In *2021 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)*. IEEE Computer Society, Los Alamitos, CA, USA, 1-10. https://doi.org/10.1109/VL/HCC51201.2021.9576372

[12] Lei Zhang and Steve Oney. 2020. FlowMatic: An Immersive Authoring Tool for Creating Interactive Scenes in Virtual Reality. Association for Computing Machinery, New York, NY, USA, 342-353. https://doi.org/10.1145/3379337.3415824