# Handing Pedagogical Scenarios Back Over To Domain Experts: A Scenario Authoring Model for VR with Pedagogical Objectives

Mathieu Risy, Valérie Gouranton, Bruno Arnaldi

► To cite this version:

Mathieu Risy, Valérie Gouranton, Bruno Arnaldi. Handing Pedagogical Scenarios Back Over To Domain Experts: A Scenario Authoring Model for VR with Pedagogical Objectives. VISIGRAPP 2024 - 19th International Conference on Computer Graphics Theory and Applications, GRAPP, Feb 2024, Rome, Italy. pp.1-12. [hal-04489791v2](https://hal.science/hal-04489791v2)

HAL Id: hal-04489791

https://hal.science/hal-04489791v2

Submitted on 21 Mar 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

Public Domain
# Handing Pedagogical Scenarios Back over to Domain Experts: A Scenario Authoring Model for VR with Pedagogical Objectives

Mathieu Risy<sup>a</sup>, Valérie Gouranton<sup>b</sup> and Bruno Arnaldi<sup>c</sup>

Univ. Rennes, INSA Rennes, Inria, CNRS, IRISA, France
mathieu.risy@irisa.fr, valerie.gouranton@irisa.fr, bruno.arnaldi@irisa.fr

**Keywords:** Virtual Reality, Scenario, Pedagogy, Authoring, Welding Use Case.

**Abstract:** Teachers and trainers make pedagogical decisions for their training courses, so why not do the same for Virtual Reality (VR) training courses? Virtual Environments for Training (VETs) are becoming prominent educational tools. However, VET models have yet to propose scenario authoring aligned with pedagogical objectives that can account for the diversity of approaches available to teachers. This paper proposes a scenario authoring model for VET that directly involves domain experts and validates their pedagogical objectives. In addition, it proposes the coexistence of multiple pedagogical scenarios within the same VET, using three types of scenarios. The validity of the model is then discussed using a VR welding application as a use case.

## 1 INTRODUCTION

Virtual Reality (VR) has shown promising educational benefits (Dalgarno and Lee, 2010) manifested by the growing number of Virtual Environments for Training (VET). However, current VR authoring tools remain difficult for trainers (Ashtari et al., 2020). While hardly standardized, VET design generally requires a Domain Expert at its initiative, and an expert in VR development. In this paper, for the sake of simplicity, the term Domain Expert refers to a person knowledgeable in the learning content, and includes the roles of both teachers and pedagogical experts.

A scenario, in the context of a Virtual Environment (VE), organizes the sequences of events that unfold at runtime. It characterizes the user's actions and interactions, and the VE entities' behaviors. The execution of a VR application always results in a scenario unfolding, whether expressed explicitly or implied. The explicit expression of the scenario serves a monitoring purpose and allows it to manage the VE. Scenario authoring is the writing of a machine-interpretable scenario for the VE.

Current development practices leave scenario authoring to the VR expert, with no standards taking into account the Domain Expert's Pedagogical Specifications. We argue that it would be more efficient for

the Domain Expert to write the VET scenario as the person responsible for the learning content and pedagogical decisions. The term VET here includes all types of higher education learning in VEs and does not only apply to professional training. In this paper, we propose a scenario authoring model (Section 3) that meets the following criteria:

* Domain Experts are able to author Pedagogical Scenarios according to their Pedagogical Objectives.

* The model uses pedagogical principles that are familiar to Domain Experts and understandable by the VET.

* The model allows for the definition of multiple teaching methods under the same VET.

We then illustrate the model implementation using a use case (Section 4): a welding training application (Figure 1). We conclude by discussing the capabilities of the model with Domain Experts (Section 5).

## 2 RELATED WORK

This section highlights related work on pedagogy integration in VETs and VET models. It outlines the main pedagogical approaches used in VETs. It then presents the integration of their specifications in machine-interpretable models. Finally, observation of related VET models provides insight into the current state of pedagogical scenario authoring.

<footnote>a https://orcid.org/0009-0004-6248-9988</footnote>

<footnote>b https://orcid.org/0000-0002-9351-2747</footnote>

<footnote>c https://orcid.org/0000-0002-2868-8826</footnote>
<figure><img src="image_1.png" /><figcaption>(a) Error-spotting environment.</figcaption></figure>

<figure><img src="image_2.png" /><figcaption>(b) Welding station environment.</figcaption></figure>

<caption>Figure 1: Environments of the VR welding training application use case.</caption>

## 2.1 Pedagogical Approaches in VET

The educational use of VR has extended beyond the field of research and is mature enough to allow for a closer examination of its practices. Some authors (Garzón et al., 2020; Mikropoulos and Natsis, 2011; Radianti et al., 2020) state that a minority of VETs declare a pedagogical approach. Among these approaches, almost all take reference from the learning theory of constructivism (Piaget, 1950) or more specific theories derived from its principles. According to constructivism learners actively construct their knowledge based on their previous experiences. This emphasizes the need for an active learning process with contextualized learning material. Constructivist principles lend well to VR learning affordances (Dalgarno and Lee, 2010), notably allowing for engagement in contextualized experiential learning.

While constructivism seems to be the most commonly used learning theory, some VETs rely on other pedagogical approaches. The “VR nugget” based approach (Horst et al., 2022) combines generic standalone micro-learning patterns in VR to construct a course that provides on-demand interactions. Constructivism is also less compatible with lecture-based learning. Bowman et al. decided not to use a constructivist approach to illustrate abstract design principles to students (Bowman et al., 1999). Finally, Radianti et al. reported on a VET using the learning theory of behaviorism. This theory states that knowledge is external and acquired through reinforcement of rewarding or punishing consequences (Schunk, 2012).

## 2.2 Pedagogical Specifications in VET

Machine-interpretable Pedagogical Specifications introduce pedagogical logic in virtual environments. Using generic models, they can represent multiple types of pedagogical decisions in the VET.

The structure of a course is a classic specification for planning learning sessions. However, VET models often represent only low-level activities, such as

tasks and actions (Johnson and Rickel, 1997; Gerbaud et al., 2008; Buche et al., 2010). While this may be sufficient for short procedures and intervention, it fails to meet the organizational needs for longer and more modular learning interventions. Higher levels of structure can be achieved by interconnecting multiple learning activities (Udeozor et al., 2023) or using existing learning modeling specifications. IMS Learning Design (IMS LD) (Koper et al., 2003) is a well-known specification that has been integrated in VET models (Marion et al., 2009). It uses a generic “Play-Act-Learning Activity” structure to model learning interventions of arbitrary length, while being generic enough not to enforce a procedural type of learning.

Role specifications can be effective for modeling multiple behavior types for pedagogical agents. They allow for the assignment of different objectives and interactions to multiple actors (Claude et al., 2014). Roles can represent asymmetric interactions such as a trainer-trainee collaboration or hierarchical positions. This idea can be expanded to teams and organizations (Buche et al., 2004; Claude et al., 2015). Such teams and organizations facilitate modifications to the behavior of a group of actors with different roles but similar objectives.

Recurring pedagogical patterns are also a form of Pedagogical Specifications. “VR nuggets” (Horst et al., 2022) use generic VR patterns such as manipulating, decomposing, or tagging objects.

The expression of learning theory specifications in VET models raises the question of their integration in automated environments. The Game-Based Assessment Framework (GBAF) (Udeozor et al., 2023) uses Constructive Alignment (Biggs, 1999) as a design approach for VET. This constructivist theory proposes to design learning activities from reliable and observable descriptions of the Pedagogical Objectives. The learner's progress toward the objectives is measured using Assessments derived from these criteria. Thus, it provides easily implementable automatic assessment that is relevant to the Domain Expert.
## 2.3 Pedagogical Scenario Authoring

Pedagogical scenario authoring is the writing of pedagogical decisions in the VET scenario. The simplest approach is to manually enable or disable pedagogical support such as instructions, guidance, and feedback. However, this approach lacks adaptability and can result in the pedagogy blending with the code. As a result, pedagogical scenarios typically use explicit representations to facilitate their editing, analysis, and reuse (Gerbaud et al., 2008).

Procedural scenarios provide a clear baseline of the task sequences and goals to be achieved in order to complete the procedure (Claude et al., 2014). For example, pedagogical components can hint at future steps (Gerbaud et al., 2008; Richard et al., 2021). Conversely, deviations from the baseline favor the detection of errors. Procedural scenarios also provide precise control over authorized actions. In GVT (Gerbaud et al., 2008), the pedagogy engine supports strategies to allow or block the actions available to users in real-time. The Domain Expert can write adaptive strategies, but still requires a VR expert to implement them. Another approach is the transmission of pedagogy through agents. In MAS-CARET (Buche et al., 2004), agents form pedagogical strategies based on their role, knowledge, and the learner's progress. A domain model, in the form of an ontology written by the Domain Expert, contains semantic information about the environment and the learning domain. While this provides a more ecological environment, pedagogical authoring is scarcely constrained by the Domain Expert. The types of reactions of the agents are pre-planned, leaving only the selection of the agents and their roles to the authoring.

The authoring of error responses is a powerful pedagogical tool. HERA's (Amokrane et al., 2008) risk model and errors notify a rule-based pedagogical module that allows or blocks scripted responses and the unfolding of risk consequences. The PEGASE model (Buche et al., 2010) introduces the authoring of multiple pedagogical approaches in the form of rules for agents' attitudes and reactions to error descriptors. However, defining these rules still requires coding knowledge. Procedural approaches have potent capabilities, but action sequence descriptions lack generality. They quickly become unsuitable for scenarios with low interest in representing intermediate steps.

“Emergent approaches, by a clever modelling of small behaviours of the world, allow new situations to arise” (Lanquepin et al., 2013). These approaches focus on a rich description of interactions and constraints rather than modeling users' progression (Claude et al., 2014).

The HUMANS suite (Lanquepin et al., 2013) relies on reasoning based on semantic information, causality, and domain knowledge without using scripted events. It provides a reactive ecological environment, but pedagogical authoring is limited to defining the initial situation and to non-trivial authoring of agents' reasoning.

Goal-based authoring approach uses environmental constraints and VET states as key scenario points to be achieved (Porteous et al., 2010). It provides a more familiar authoring format while retaining pedagogical control by using a higher-level description of the VE. It relies on modeling shared properties (Bouville et al., 2015) and causal information (Buche et al., 2010). Goal-based approaches remain rare, although Steve (Johnson and Rickel, 1997) is one of the best-known VET examples. Steve is an agent that provides guidance on a maintenance task using a set of goals connected by causal links and preconditions.

Assessment-based approaches (Udeozor et al., 2023) are more flexible than goal-based approaches. Progress within the scenario is based on complete or partial validation of assessment. Such assessments can concern, for example, following each step of a procedure, setting the environment in a specific state, attaining an objective, or providing a correct answer. Assessment-based approaches propose a new scenario layer that can observe both the progression of goals and procedures. To the best of our knowledge, there have been very few assessment-based proposals. Udeozor et al. proposed the GBAF as a framework for Domain Experts to author VETs aligned with Pedagogical Objectives. Assessment-based approaches present interesting properties drawing on procedural and goal-based approaches while being closer to the pedagogical practices of Domain Experts.

## 2.4 Toward a New VET Framework

While great efforts have been made toward producing credible agents and ecological environments, we have found no satisfying solution to provide a complete VET scenario authoring process for Domain Experts, aligned with their objectives and supporting multiple learning approaches. Only one model has precisely described support for multiple pedagogical approaches (Buche et al., 2010). While it offers precise authoring control, it is inaccessible without a strong background in coding and does not guarantee pedagogical coherence.

Procedural approaches lend themselves well to pedagogical authoring, but lack generality. While ontologies seem to provide powerful insight into the learning domain, they do not ensure pedagogical con-
<figure><img src="image_3.png" /><figcaption>Figure 2: Overview of the components that make up the scenario authoring model.</figcaption></figure>

In addition, they are difficult for Domain Experts to find and write. Emergent scenarios offer rich environments but give little control to the Domain Expert. Finally, goal- and assessment-based approaches have rarely been implemented, but show promising capabilities for a generic pedagogical authoring model. Additionally, these approaches highlight interesting authoring possibilities that may conform with the Domain Expert's objectives and assessments.

## 3 SCENARIO AUTHORING MODEL WITH PEDAGOGICAL OBJECTIVES

We argue that the scenario of a VET should be written by the Domain Expert. Much like Domain Experts, scenarios drive the learning session, monitor progress, and can adapt both content and Pedagogical Guidance. We propose a scenario authoring model that aligns the VET with the Pedagogical Objectives of the Domain Expert. Furthermore, we introduce the authoring of multiple pedagogical approaches within the same VET in the form of a new pedagogical scenario layer.

This section provides a general overview of the model. It then details each type of scenario that composes the model and their use of the Pedagogical Specifications. Finally, it explains the monitoring aspect of the model.

### 3.1 Model Overview

Our scenario authoring model uses an assessment-based approach (Udeozor et al., 2023). It allows for an intuitive and flexible authoring process based on

the Pedagogical Specifications of the Domain Expert. In essence, Assessments represent monitoring specifications that validate the Pedagogical Objectives and influence progression within the scenario.

The scenario authoring model presents the learner with a global learning scenario that combines three types of scenarios: a Reference Scenario, Error Scenarios, and the active Pedagogical Scenario (Figure 2). The Reference Scenario (Section 3.2) defines what the learner can do as long as the action is not classified as an error. Most importantly, the Assessments and objectives it contains define what the learner is expected to do. Error Scenarios (Section 3.3) represent typical deviations from the Reference Scenario that should not occur. Finally, the Pedagogical Scenario (Section 3.4) implements the pedagogical decisions. It provides Pedagogical Guidance and adapt the learning experience. This distinction facilitates the writing and coexistence of multiple pedagogical approaches by separating the measurement of the learner's progress from the pedagogical decisions. These decisions are added on top of the Reference Scenario by Pedagogical Scenarios.

The scenarios belongs to one of the four key interacting components that compose the model. Figure 2 illustrates the communication between the Pedagogical Specifications, Scenario Component, Virtual Environment (VE), and Monitoring Component.

* **Pedagogical Specifications** contain the definition of the Pedagogical Objectives, the Assessments required to achieve them, and the Learning Activities that support their achievement (Section 3.2.1).

* **The Scenario Component** is informed by the Pedagogical Specifications and manages their implementation in the VE. Its task is to orchestrate events and interactions to help achieve the Pedagogical Objectives (Section 3.4).
* The learner interacts with a **Virtual Environment** informed by the Pedagogical Specifications and controlled by the Scenarios' progression.

* The **Monitoring Component** observes the state of the VET and provides the means to interact with it outside of scenario execution (Section 3.5).

## 3.2 Reference Scenario

The Reference Scenario sets the expected standard of the learner's proficiency. It represents the achievement of the Pedagogical Objectives.

First, the Reference Scenario provides the means of monitoring the learner's progress toward the objectives. It considers a theoretical user who validates the highest level of each Assessment without the need for support or instructions. Consequently, it does not force the learner to pass the Assessments. These decisions are left up to the Pedagogical Scenarios. Using a welding training application as an example (see Section 4), one Assessment of its Reference Scenario evaluates weld quality. The reference does not change whether a learner succeeds or fails the Assessment. However, Pedagogical Scenarios can react to the Assessment in order to change the outcomes.

Second, the Reference Scenario defines the scope of the actions that can be performed without being classified as an error. In the case of a procedural activity, the definition of possible actions might take the form of branching paths. However, this representation is not suitable for every activity. In such cases, a goal-based approach might be more appropriate. It implicitly contains every action possible, unless they trigger a state of error. We propose a hybrid approach that uses the most appropriate scenario representation for each specification on a case-by-case basis.

The Reference Scenario uses automated Assessments to monitor the learner's actions and measure progression toward the Pedagogical Objectives. Therefore, the Pedagogical Specifications need to be machine-interpretable. Consequently, we decided to introduce the Constructive Alignment principle (Biggs, 1999) in our model. It allows for the alignment of the VET's behavior with the Domain Expert's needs (Udeozor et al., 2023). It has the additional benefit of facilitating discussions between Domain Experts and VR experts.

### 3.2.1 Constructive Alignment

Constructive Alignment (Biggs, 1999) is an educational principle for the design of learning interventions and programs. Its core proposal is that the learner's progression toward Pedagogical Objectives

<figure><img src="image_4.png" /><figcaption>Figure 3: Constructive Alignment, adapted from the works of John Biggs (Biggs, 1999) and Elie Milgrom.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>Figure 4: Pedagogical Specifications of the scenario authoring model.</figcaption></figure>

should be assessed using reliable and observable criteria, known as Intended Learning Outcomes (ILOs). ILOs are defined as “statements, written from the students' perspective, indicating the level of understanding and performance they are expected to achieve as a result of engaging in the teaching and learning experience” (Biggs and Tang, 2011). They derive directly from the objectives and are used as core elements to create Learning Activities. Constructive Alignment requires three steps (see Figure 3):

1. Definition of the ILOs.

2. Definition of the Assessments based on the descriptions of the ILOs.

3. Designing of the Learning Activities, using Assessments to validate the ILOs.

Our model directly integrates these three key elements into the Pedagogical Specifications Component (Figure 4). They provide information relevant to both Domain Experts and VET scenarios. The Constructive Alignment process is illustrated in Section 4.3 using a welding training application as a use case.

### 3.2.2 ILOs

An ILO describes one or multiple levels of proficiency. It is written using Observable Action Verbs (OAVs) to define precisely how its statements are to be evaluated. Each ILO level is related to a learning
taxonomy level to express the type of cognitive task involved. Bloom's revised taxonomy (Anderson and Krathwohl, 2001) and SOLO taxonomy (Biggs and Collis, 1982) are two well-known examples. Using the same example of a welding training application, Table 1 illustrates an ILO associated with the Pedagogical Objective of learning welding practice. The Reference Scenario ensures the implementation of every ILO under the Pedagogical Specifications.

<table><caption>Table 1: Example of an ILO description. Basic welding practice for a welding training application.</caption><thead><tr><th colspan="2">ILO - Basic welding practice</th></tr></thead><tbody><tr><td><strong>Time:</strong></td><td>After preparation of the welding station and application of safety measures.</td></tr><tr><td><strong>Level 1</strong></td><td></td></tr><tr><td><strong>Taxonomy (Bloom):</strong></td><td>Remember</td></tr><tr><td><strong>Taxonomy (SOLO):</strong></td><td>Unistructural</td></tr><tr><td><strong>Description:</strong></td><td>The learner is able to:<ul><li><em>Move</em> the welding torch in a straight line.</li><li><em>Maintain</em> constant and appropriate <em>speed, height, and angle</em>.</li></ul></td></tr><tr><td><strong>Level 2</strong></td><td></td></tr><tr><td><strong>Taxonomy (Bloom):</strong></td><td>Analyze</td></tr><tr><td><strong>Taxonomy (SOLO):</strong></td><td>Multistructural</td></tr><tr><td><strong>Description:</strong></td><td>The learner is able to:<ul><li><em>Correct</em> the welding parameters during practice if they deviate from the standard.</li></ul></td></tr><tr><td><strong>Level 3</strong></td><td></td></tr><tr><td><strong>Taxonomy (Bloom):</strong></td><td>Evaluate</td></tr><tr><td><strong>Taxonomy (SOLO):</strong></td><td>Relational</td></tr><tr><td><strong>Description:</strong></td><td>The learner is able to:<ul><li><em>Evaluate</em> the weld quality visually after practice.</li><li><em>Correct</em> their next practice using the identified errors.</li></ul></td></tr></tbody></table>

### 3.2.3 Assessments

Assessments naturally derive from the observable statements of the ILOs. While they can be used for grading purposes, their primary objective is to monitor the learner's progression. They are not required to be visible to the learner and are often implicitly integrated within the VE. An Assessment derives from a single ILO and contributes to its validation. For example, two Assessments can be derived from the ILO in Table 1. The first compares weld and gesture parameter values to the standard. The second evaluates the evolution of weld quality over time. The validation of an Assessment is set by the Domain Expert as a percentage of the progression of the ILO. Some Assessments may even allow for partial validation.

### 3.2.4 Learning Activities

Learning Activities are defined in the final step of the Constructive Alignment. They describe the learning context for the achievement of the ILOs and how the Assessments are applied. Let us take the example of a Learning Activity relating to the ILO described in Table 1 and other practice-oriented ILOs. This activity describes the means to achieve ILOs and their Assessments in the form of a contextualized welding environment that contains several metal plates. It describes the interactions and behaviors for the welding of metal plates according to the configurations described by the ILOs.

Learning Activities represent the basic units of a scenario that a learner can complete. Pedagogical Specifications include a Structure specification (Figure 4) to help group and filter these activities. It uses three hierarchical levels “Training Course-Modules-Learning Activities”, where the Learning Activity is the lowest. The Structure does not provide the order of execution of its sub-parts. This pedagogical decision is left up to each Pedagogical Scenario.

## 3.3 Error Scenarios

Error Scenarios represent typical deviations from the Reference Scenario. For the purpose of complementing the Reference Scenario, the definition of an error includes any behavior that demonstrates the non-mastery of an ILO. This may include hesitations, or taking too long without committing any mistakes. The Domain Expert is responsible for defining whether part of a scenario part belongs to the Reference Scenario, or is represented as an Error Scenario.

Error Scenarios have two roles. First, they notify the VET when a state of error is triggered. This allows the active Pedagogical Scenario and the Monitoring Component to act where appropriate. Second, they can represent either or both error consequences and corrective measures that allow the learner to return to the Reference Scenario. For example, in the case of a welding training application, welding without opening the gas bottle will trigger an Error Scenario. This notifies the VET of the error and represents the consequences, namely more sparks and poor weld quality. As the error is not fatal, it allows the learner to stop welding, open the gas bottle, and return to the Reference Scenario.

Triggering an Error Scenario does not necessarily result in its execution. Pedagogical Scenarios handle the VET's reaction to the triggering of an Error Scenario, based on the pedagogical decisions they represent.
<figure><img src="image_6.png" /><figcaption>Figure 5: Detailed view of the scenario authoring model.</figcaption></figure>

## 3.4 Pedagogical Scenarios

Pedagogical Scenarios represent the pedagogical decisions made to allow the learner to achieve all or part of the Pedagogical Objectives. Because of this status, they complement and have control over the Reference Scenario and Error Scenarios.

Each Pedagogical Scenario contains a subset of the Reference Scenario's ILOs, Assessments, and Structure (Figure 5). This subset represents the focus of the Pedagogical Scenario. The Structure can automatically filter out its sub-parts that are not associated with the subset of ILOs. It narrows down the Learning Activities and Assessments that the learner will interact with, either by blocking or not monitoring parts of the Reference Scenario. For example, in the case of a welding training application (Section 4), the Domain Expert may wish to focus solely on safety as part of an introductory course. The associated Pedagogical Scenario thus contains only activities relevant to safety and does not evaluate welding practice. In addition, it provides relevant guidance to emphasize the role of each welding safety practice and piece of equipment, and to prevent dangerous behavior.

### 3.4.1 Structure Ordering

Each Pedagogical Scenario provides the ordering of the Learning Activities contained in its Structure. By default, the Reference Scenario does not provide an order in which to execute the Learning Activities.

The ordering of activities is not necessarily linear. For example, one may establish a pedagogical decision to provide the learner with a choice every

time, or only in some cases. In non-linear scenarios, the definition of pedagogical prerequisites is an important specification that allows the learner to go from one Learning Activity to the next in a logical order. Such prerequisites may include the validation of other Learning Activities, or the validation of an Assessment above certain a threshold.

Functional prerequisites are indicative prerequisites regarding the state of the VE. For example, the “welding practice” Learning Activity requires the “preparation of the welding station” Learning Activity to be performed beforehand. However, the Domain Expert can always decide that the latter Learning Activity is not to be performed by the learner, but is assumed to be performed correctly. In such case, the application must handle the prerequisites directly instead of the learner, in order to begin the “welding practice” Learning Activity in the correct conditions.

### 3.4.2 Error Handling

Pedagogical Scenarios handle the triggering of Error Scenarios according to pedagogical decisions. Through this process, a Pedagogical Scenario can modify or block the execution of an Error Scenario. Taking the example of welding training detailed in Section 4, “welding without a welding helmet” is an error that deviates from an ILO on compliance with safety measures during practice. Indeed, this error represents a serious risk for the learner’s eyesight in real-life situations. This error can be handled in multiple ways. The scenario can prevent the error from happening by blocking the interactions susceptible to causing it. It can also display an informative message
when the Error Scenario is triggered, or, it can illustrate the consequences that would unfold. In the latter case, the Pedagogical Scenario manages the error by deciding not to intervene in its resolution.

### 3.4.3 Pedagogical Guidance

We define Pedagogical Guidance as elements that orient the learning process and are superfluous to an expert in the learning domain. They encompass elements such as instructions, feedback, or interaction management. Consequently, they rely on the Pedagogical Specifications to inform their content. Pedagogical Guidance is used as an interface between Pedagogical Scenarios and the VE.

The activation of Pedagogical Guidance depends on the scenario. For example, it may be active at all times, triggered by an Assessment, or react to the triggering of an Error Scenario. Conversely, interacting with Pedagogical Guidance may trigger progression of the scenario. It should be noted that the actions of Pedagogical Guidance are not only additive. Preventing actions and events is also an important role of Pedagogical Guidance. Most notably, this might take the form of blocking actions in order to prevent errors.

Pedagogical Guidance plays an intermediate pedagogical role, as illustrated in the detailed view of the model (Figure 5). It is a key part of Pedagogical Scenarios, but can also be triggered by the Domain Expert through the Monitoring Component. In addition, fostering learner autonomy is a recurring Pedagogical Objective; therefore Pedagogical Guidance can also be requested directly by the learner to provide control over the learning process.

## 3.5 Monitoring

The Monitoring Component allows an external user to monitor the VET and act upon it. The characteristics of VETs make it difficult for the Domain Expert to monitor the progress of a learner using a VR device. Providing a video feed might not be sufficient for correct monitoring and does not scale well for multiple learners. Thus, the monitoring process needs to provide other information. The Monitoring Component observes the state of the VE, progression within the scenarios, and the validation of ILOs, as illustrated in Figure 5. It provides the Domain Expert with information that directly derives from the measurement of the learner's progress. In addition, the Monitoring Component can also provide help and guidance to the learner at runtime, without leaving the VE. The Monitoring Component can interact directly with the VE, but also provides the means of triggering Assessments and Pedagogical Guidance if needed. It bridges

the gap between the immersive virtual environment and the "outside world". It introduces possibilities for asymmetric modes of collaboration due to its ability to monitor learners' performance and act upon the VE. Such collaborations could take place between the learner and the Domain Expert or between multiple learners.

# 4 USE CASE

To assess the viability of our model, we applied it to a real use case of a VR training course. The use case was designed in collaboration with Domain Experts who teach introductory courses on welding theory and practice to undergraduate students. These courses constitute an introduction to industrial processes and risk management as part of an engineering curriculum.

The Domain Experts are involved in a development project to create a VR training application that helps teach safety and welding basics before actual welding practice. In parallel with its development, we had the opportunity to implement our model's approach within this application and gather feedback from the Domain Experts.

## 4.1 Welding Training Application in VR

The welding training application teaches safety practices and introductory notions for Metal Inert Gas (MIG) welding – a type of Gas Metal Arc Welding (GMAW) that uses an electric arc to provoke the fusion of metal. The application contains an error-spotting environment (Figure 1a) and a welding practice environment (Figure 1b). The error-spotting environment is designed to help identify clothing that is unsuited for safe welding practice, and risks posed by loose hair. The second environment features a functional MIG welding station, welding material, and safety equipment. The learner can interact with Personal Protective Equipment (PPE), set up the welding station, and perform a weld on a simple metal plate.

## 4.2 Implementation

The welding training application is being developed with Unity 2021. This preexisting code basis features ecological interactions with welding material and two environments (Figure 1). Additional Pedagogical Guidance was developed and integrated into the environment to answer the needs of the Pedagogical Scenarios.
<figure><img src="image_7.png" /><figcaption>Figure 6: Pedagogical Specifications writing protocol.</figcaption></figure>

The model was implemented using Xareus<fnref n="1" />, a Petri net-based scenario engine (Bouville et al., 2015; Claude et al., 2014), and C# classes. The Pedagogical Specifications (Figure 4) are represented in C# and instantiated from the application's Pedagogical Specifications written in natural language. Their instantiation allows the construction of objects representing the focus of a Pedagogical Scenario (Figure 5). The scenarios, including the Learning Activities content, were created with Xareus' visual authoring interface for no-code approaches. In addition, Xareus' sensors/effectors logic was extended with the automatic activation of the relevant scenario sections based on the content of a given scenario focus parameter.

## 4.3 Model Application

Our scenario authoring model was applied to the welding training application in accordance with the Domain Expert's authoring process (Figure 6). First, this requires the definition of the Pedagogical Specifications. Second, these specifications are used to author the Reference Scenario and Error Scenarios. Finally, two Pedagogical Scenarios were proposed to show the possibilities of our model.

### 4.3.1 Reference Scenario

The Reference Scenario uses Pedagogical Specifications to align the VET with the Pedagogical Objectives. This allows for the automation of the relations between the specifications and facilitates authoring. For example, selecting an ILO subset disables the Learning Activities that do not serve to achieve them. The specifications were written using a six-step protocol (Figure 6) based on Constructive Alignment (Figure 4). Following the validation of the specifications

by Domain Experts, minor corrections were made to the welding practice ILOs.

Five ILOs were identified for this application.

* *ILO*₁: Identification of safe welding clothing among a panel of virtual humans.

* *ILO*₂: Choosing and wearing suitable PPE.

* *ILO*₃: Preparation of the welding station.

* *ILO*₄: Application of safety instructions during welding practice.

* *ILO*₅: Basic welding practice.

A detailed description of *ILO*₅ is presented in Table 1. Each ILO contains a validation value made accessible to the Assessments and the Monitoring Component.

The Reference Scenario then exposes each VET's aspect monitored by the Assessments of the ILOs. For example, the *ILO*₂ Assessment records the interactions of the learner with PPE. It expects the *apron*, *welding helmet*, and *gloves* to be in an “equipped” state and to have been equipped in this order. Thus, the Reference Scenario provides a procedural description of the actions and access to the states of the PPE.

Finally, three Learning Activities (LA) were devised using the ILOs and Assessments.

* *LA*₁: Identification of safe welding clothing.

* *LA*₂: Preparation of the welding station in accordance with the safety measures.

* *LA*₃: Basic welding practice.

Each Learning Activity knows the Assessments it implements and thus transitively knows the ILOs it serves to achieve. The Reference Scenario segments the Assessments' validations into sections representing their Learning Activity. Each section can be easily enabled or disabled without impacting the others.

### 4.3.2 Error Scenarios

In this use case, most Error Scenarios are used to notify the VET. For example, in *LA*₃ touching a burning metal plate is a fatal error. Representing its consequences was deemed unnecessary, but it does not prevent Pedagogical Scenarios from handling the Error Scenario using Pedagogical Guidance.

Some cases are better suited to represent the consequences of errors. If the welding action is triggered while the gas bottle is in the “closed” state, the Error Scenario increases the production of sparks and degrades the quality of the weld. In addition, if the state of the gas bottle changes to “open”, the scenario stops and allows the learner to return to the Reference Scenario. This case supports the enabling or disabling of consequences based on the needs of the Pedagogical Scenarios.

<footnote n="1">¹https://xareus.insa-rennes.fr/</footnote>
### 4.3.3 Pedagogical Scenarios

This use case contains two Pedagogical Scenarios to illustrate the capabilities of the model – a safety-focused scenario and a practice-focused scenario.

The safety-focused scenario introduces safety notions to the learner using a significant amount of Pedagogical Guidance. It makes use of the three Learning Activities. At the end of each activity, the Pedagogical Scenario informs the Reference Scenario of the next activity to be performed. This scenario provides instruction and corrective feedback to validate safety ILOs. For example, in LA₂, it triggers instructional guidance in the form of a checklist, highlights PPE entities, and provides dynamic textual feedback if components of PPE are missing or equipped in the wrong order. This Pedagogical Scenario does not evaluate ILO₃ “Preparation of the welding station”. Consequently, the scenario handles the preparation of the welding station by changing the state of the relevant elements. In LA₃, the Pedagogical Scenario features the handling of Error Scenarios. The learner can test the welding torch while dangerous interactions are blocked with explanatory feedback. This prevents the activation of the torch while the welding helmet is not lowered, for example.

The practice-focused scenario is oriented toward teaching welding practice and the preparation of a MIG welding station. It uses only LA₂ and LA₃ and disables LA₁. In this scenario, LA₂ explicitly evaluates the preparation of the welding station and implicitly expects compliance with safety measures. It uses Pedagogical Guidance to prevent welding until all preparation steps have been completed. Additionally, it does not indicate nor enforce safety-related Assessments, although failure to apply safety measures triggers an alert via the Monitoring Component. Finally, LA₃ uses dynamic Pedagogical Guidance in the form of a “ghost” of the welding torch (Figure 7) to demonstrate the expected speed, height, and angle parameters. It helps with the achievement ILO₅ (Table 1). This activity also provides the learner with the possibility to display informative graphics as optional guidance.

## 4.4 Discussion with Domain Experts

This section reports feedback on the acceptance of the authoring model from three Domain Experts. Two of them are welding teachers with no VR experience, and one is a pedagogical expert familiar with VR development using no-code approaches. The discussion followed a qualitative guided interview. Before and after the model presentation, Domain Experts were

asked to express the pedagogical objectives, specifications, and scenarios they considered necessary to create the use case. Then, they were asked about their reported confidence and motivation to use the components of the model. Finally, they were given the opportunity to comment on the relevancy of the model.

Domain Experts reported feeling confident in writing each type of Pedagogical Specifications (Figure 6). One reported a strong interest in Constructive Alignment, as it allows for introspection on the learning material and evaluations. It should be noted that two experts already had intermediate knowledge in terms of describing learning outcomes with OAVs.

When asked about directly authoring a scenario, Domain Experts reported irresolute confidence in writing and high confidence in editing. Welding teachers did not feel confident writing Pedagogical Scenarios without the support of a pedagogical expert. They reported having a "one scenario per student" approach, and not being in a scenario authoring mindset. Conversely, the pedagogical expert felt confident in writing scenarios using our model and expressed interest in using it for professional projects.

Higher levels of confidence were reported for editing pre-existing Pedagogical Scenarios. Domain Experts were interested in the possibility of selecting and potentially editing a scenario just before a training session, for example, by choosing only the relevant Learning Activities and Assessments.

The capabilities of a model with multiple Pedagogical Scenarios were of great interest to the Domain Experts, particularly to allow for adaptation to different categories of learners. When presented with this possibility, they suggested using Pedagogical Scenarios to provide more guidance to foreign students or to gradually increase the level of difficulty.

<figure><img src="image_8.png" /><figcaption>Figure 7: Pedagogical Guidance in the form of a welding torch ghost and weld quality feedback.</figcaption></figure>
# 5 DISCUSSION

This section discusses the capabilities of the model based on the feedback from Domain Experts (Section 4.4). They reported high motivation and feeling confident in using the model in its entirety, showing a promising acceptance of its principles.

Domain Experts' confidence in authoring scenarios highlighted potential difficulties in understanding VR scenarios without pedagogical expertise or examples. On the other hand, Pedagogical Specifications authoring received a high level of confidence. This shows the importance of the model's specifications to provide a common language understandable by both Domain Experts and VR experts. While complete authoring might require additional guidance, Domain Experts exhibited a keen interest and confidence in editing pre-existing scenarios. This interest extends after the development phase, suggesting that the VET can become a more time-resilient educational tool.

This work assumes the existence of a simplified authoring interface for Domain Experts. Such an interface is not the focus of this paper as its implementation would require its own pedagogical decisions. Nonetheless, we consider it beneficial for scenario editing and generating code-based specifications.

Domain Experts identified the adaptation to different learner categories as an advantage of using multiple Pedagogical Scenarios. Extending the model with the authoring of automatic adaptation would provide Domain Experts with adaptive authoring tools that can inspire new ways of thinking about teaching.

# 6 CONCLUSION

This paper proposed a scenario authoring model for VET to respond to a lack of solutions providing Domain Experts with hands-on control over the pedagogical scenario authoring of VETs. It described how the integration of Constructive Alignment principles can shape VET design to guarantee the validation of Pedagogical Objectives. It then proceeded to explain the three types of scenarios used by the model. This separation of scenario roles allows for the authoring of multiple Pedagogical Scenarios by separating the learner's Assessment from pedagogical decisions.

Following the description of the model, its capabilities were illustrated through a use case pertaining to a welding training application. While an overall assessment of the model is a difficult task, discussions with Domain Experts revealed high levels of interest in its use. In particular, for its learner progress monitoring, authoring of multiple pedagogical scenarios,

and editing capabilities.

Future work should aim to expand scenario adaptation to learners. Notably, the addition of dynamic pedagogical scenario changes and scenario blending should be investigated. Further integration of Pedagogical Specifications could lead to the automated generation of a VET skeleton for the benefit of VR development. Finally, the high-level nature of the model opens up its extension to Collaborative Virtual Environments and to the eXtended Reality (XR) continuum.

# ACKNOWLEDGEMENTS

This work has benefited from state aid managed by the "Agence Nationale de la Recherche" under the "Investissements d'Avenir" program under the reference ANR-21-DMES-0001.

We want to thank the AIR Project pedagogical engineer and INSA Rennes' teachers for their participation in the model discussion.

# REFERENCES

Amokrane, K., Lourdeaux, D., and Burkhardt, J.-M. (2008). Hera: Learner tracking in a virtual environment. *Int. J. Virtual Real.*, 7(3):23–30.

Anderson, L. W. and Krathwohl, D. R. (2001). *A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives: complete edition*. Addison Wesley Longman, Inc., New York, complete edition.

Ashtari, N., Bunt, A., McGrenere, J., Nebeling, M., and Chilana, P. K. (2020). Creating augmented and virtual reality applications: Current practices, challenges, and opportunities. In *Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems*, page 1–13, Honolulu HI USA. ACM.

Biggs, J. (1999). *Teaching for Quality Learning at University: What the Student Does*. SRHE and Open University Press imprint. Society for Research into Higher Education.

Biggs, J. B. and Collis, K. F. (1982). *Evaluating the quality of learning: the SOLO taxonomy (structure of the observed learning outcome)*. Educational psychology. Academic Press, New York.

Biggs, J. B. and Tang, C. S.-k. (2011). *Teaching for quality learning at university: what the student does*. SRHE and Open University Press imprint. McGraw-Hill, Society for Research into Higher Education & Open University Press, Maidenhead, England New York, NY, 4th edition.

Bouville, R., Gouranton, V., Boggini, T., Nouviale, F., and Arnaldi, B. (2015). #five: High-level components for
developing collaborative and interactive virtual environments. In *2015 IEEE 8th Workshop on Software Engineering and Architectures for Realtime Interactive Systems (SEARIS)*, page 33–40, Arles, France. IEEE.

Bowman, D. A., Hodges, L. F., Allison, D., and Wineman, J. (1999). The educational value of an information-rich virtual environment. *Presence: Teleoperators and Virtual Environments*, 8(3):317–331.

Buche, C., Bossard, C., Querrec, R., and Chevaillier, P. (2010). Pegase: A generic and adaptable intelligent system for virtual reality learning environments. *International Journal of Virtual Reality*, 9(22):73–85.

Buche, C., Querrec, R., de Loor, P., and Chevaillier, P. (2004). Mascaret: A pedagogical multi-agent system for virtual environment for training. *International Journal of Distance Education Technologies*, 2:41–61.

Claude, G., Gouranton, V., and Arnaldi, B. (2015). Roles in collaborative virtual environments for training. In Imura, M., Figueroa, P., and Mohler, B., editors, *Proceedings of International Conference on Artificial Reality and Telexistence Eurographics Symposium on Virtual Environments*, pages 1–8, Kyoto, Japan.

Claude, G., Gouranton, V., Bouville Berthelot, R., and Arnaldi, B. (2014). Short paper: #seven, a sensor ef-fector based scenarios model for driving collaborative virtual environment. In Nojima, T., Reiners, D., and Staadt, O., editors, *ICAT-EGVE, International Conference on Artificial Reality and Telexistence, Eurographics Symposium on Virtual Environments*, page 1–4, Bremen, Germany.

Dalgarno, B. and Lee, M. J. W. (2010). What are the learning affordances of 3-d virtual environments? *British Journal of Educational Technology*, 41(1):10–32.

Garzón, J., Kinshuk, Baldiris, S., Gutiérrez, J., and Pavón, J. (2020). How do pedagogical approaches affect the impact of augmented reality on education? a meta-analysis and research synthesis. *Educational Research Review*, 31:100334.

Gerbaud, S., Mollet, N., Ganier, F., Arnaldi, B., and Tisseau, J. (2008). Gvt: a platform to create virtual environments for procedural training. In *2008 IEEE Virtual Reality Conference*, page 225–232, Reno, NV, USA. IEEE.

Horst, R., Naraghi-Taghi-Off, R., Rau, L., and Dolerer, R. (2022). Authoring with virtual reality nuggets—lessons learned. *Frontiers in Virtual Reality*, 3:840729.

Johnson, W. L. and Rickel, J. (1997). Steve: an animated pedagogical agent for procedural training in virtual environments. *ACM SIGART Bulletin*, 8(1–4):16–21.

Koper, R., Olivier, B., and Anderson, T. A. (2003). Ims learning design specification (version 1.0).

Lanquepin, V., Carpentier, K., Lourdeaux, D., Lhommet, M., Barot, C., and Amokrane, K. (2013). Humans: a human models based artificial environments software platform. In *Proceedings of the Virtual Reality International Conference: Laval Virtual*, page 1–8, Laval France. ACM.

Marion, N., Querrec, R., and Chevaillier, P. (2009). Integrating knowledge from virtual reality environments to learning scenario models - a meta-modeling approach. In *Proceedings of the First International Conference on Computer Supported Education*, volume 1, page 253–258, Lisboa, Portugal. SciTePress - Science and Technology Publications.

Mikropoulos, T. A. and Natsis, A. (2011). Educational virtual environments: A ten-year review of empirical research (1999–2009). *Computers & Education*, 56(3):769–780.

Piaget, J. (1950). *The psychology of intelligence*. Routledge, New York.

Porteous, J., Cavazza, M., and Charles, F. (2010). Applying planning to interactive storytelling: Narrative control using state constraints. *ACM Transactions on Intelligent Systems and Technology*, 1(2):1–21.

Radianti, J., Majchrzak, T. A., Fromm, J., and Wohlgenannt, I. (2020). A systematic review of immersive virtual reality applications for higher education: Design elements, lessons learned, and research agenda. *Computers & Education*, 147:103778.

Richard, K., Havard, V., His, J., and Baudry, D. (2021). Intervales: Interactive virtual and augmented framework for industrial environment and scenarios. *Advanced Engineering Informatics*, 50:101425.

Schunk, D. H. (2012). *Learning theories: an educational perspective*. Pearson, Boston, 6th ed edition.

Udeozor, C., Chan, P., Russo Abegão, F., and Glassey, J. (2023). Game-based assessment framework for virtual reality, augmented reality and digital game-based learning. *International Journal of Educational Technology in Higher Education*, 20(1):36.