# Immersive Crisis Training: Features, Observations and the Shift From Traditional Crisis Exercises

Alexis Evain, Jess Kropczynski, Shane Halse, Audrey Fertier, David Panzoli,
Frederick Benaben

► To cite this version:

Alexis Evain, Jess Kropczynski, Shane Halse, Audrey Fertier, David Panzoli, et al. Immersive Crisis Training: Features, Observations and the Shift From Traditional Crisis Exercises. Journal of Contingencies and Crisis Management, 2025, 33 (2), pp.e70049. 10.1111/1468-5973.70049. hal-05046995

HAL Id: hal-05046995

https://imt-mines-albi.hal.science/hal-05046995v1

Submitted on 25 Apr 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_2.png" /></figure>

ORIGINAL ARTICLE

OPEN ACCESS

# Immersive Crisis Training: Features, Observations and the Shift From Traditional Crisis Exercises

Alexis Evain¹ | Jess Kropczynski² | Shane Halse² | Audrey Fertier¹ | David Panzoli³ | Frederick Benaben¹

¹IMT Mines Albi, Albi, France | ²University of Cincinnati, Cincinnati, Ohio, USA | ³INU Champollion, Albi, France

**Correspondence:** Alexis Evain (alexis evain@mines-albi.fr)

**Received:** 19 December 2023 | **Revised:** 7 April 2025 | **Accepted:** 10 April 2025

**Funding:** This research work was carried out within the framework of the EGCERSIS project funding. The work was carried out with the support of public and European institutions and industrial partners.

**Keywords:** crisis management | extended reality | immersive simulation | scenario | serious game | situation awareness

## ABSTRACT

This article provides insights from the 'Lessons from the Field' perspective into crisis training using virtual reality (VR). Conventional crisis simulations often involve costly in-person events, limiting accessibility and effectiveness. VR technology has emerged as a promising alternative to overcome these challenges. This article delves into the application of VR in crisis management training, examining real-world exercise examples, and presenting innovative experimentation. A comprehensive experiment, grounded in real-life exercise scenarios, was conducted to gather feedback from participants. The findings demonstrate the efficacy of VR in replicating crisis training environments, providing an innovative solution to simulate real-world use cases. These results suggest that investing in VR research and development can significantly enhance training opportunities for public safety agencies.

## 1 | Introduction

Crisis management plays a critical role for anyone who works in a position of responsibility, whether in the public or private sector. This multifaceted discipline draws upon various fields, including communication, safety, regulation, risk management, ethics, and decision-making (Reason 1997; Murria 2005). A crisis can be defined as a situation that poses a significant threat or disruption to political, social, or economic systems (Vašíčková 2020; Benaben et al. 2016).

Wallace and Balogh (1985) explains that crisis management involves several key steps: Prevent (i): This involves identifying risks, putting in place response plans and educating employees.

Preparation (ii): This involves detecting the early warning signs of a crisis and triggering the relative response dedication plan to mitigate its impact. Response (iii): This involves implementing the necessary action plans to handle the disaster and limit the negative consequences. Recovery (iv): This phase concerns getting the organisation back to a working state after the incident and restoring stakeholder confidence. In this article, the main step is on the training in Prevent (i).

It is important to understand the different aspects that can affect an organisation's ability to manage a crisis effectively. The main factors to consider are the frequency, severity, and variety of cases. Frequency is a factor to consider when planning. The more frequently an organisation is exposed to crises, the more prepared it must be to deal with these situations. Severity concerns the impact of an event. The more severe an event, the

<footnote>For the purpose of Open Access, a CC-BY public copyright licence has been applied by the authors to the present document and will be applied to all subsequent versions up to the Author Accepted Manuscript arising from this submission.</footnote>

<footnote>This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.</footnote>

<footnote>© 2025 The Author(s). *Journal of Contingencies and Crisis Management* published by John Wiley & Sons Ltd.</footnote>
greater its impact on a government or business's activities. In addition, the higher severity the more rapid and effective response is needed. Crises can manifest themselves in a variety of forms such as technical incidents, financial crises, etc. Crises are not always isolated and often combined forming disasters of a larger scale (Cartapanis and 2004 2004; Gill and Malamud 2014).

Crisis simulations are frequently employed to replicate real-world crisis scenarios, often incorporating actual locations and personnel. The development and design phase of each in-person exercise can take several weeks or even months. To maintain the element of surprise, participants are often kept unaware of the simulation's nature. The choice between virtual reality (VR) training and in-person exercises depends on the specific crisis scenario being simulated, the intended learning objectives, available time constraints, and budgetary considerations.

VR has the potential to offer a more cost-effective, time-efficient, and accessible alternative to traditional in-person training. VR has the potential to revolutionise crisis training by providing a more cost-effective, time-efficient, and accessible alternative to traditional in-person training activities. VR simulations can also create immersive environments that appeal to the physical, emotional, and environmental aspects of training, which can help participants to learn and retain information more effectively (Mystakidis and Lympouridis 2023).

Crisis preparedness is therefore crucial to control impacts of crises on societies. Frequent, pre-configurable, repeatable exercises with simplified feedback systems can help managers to better prepare for and respond to crisis situations (Van Laere and Lindblom 2019). The VR crisis exercise should be designed to fully immerse participants in a realistic context. This means that the experience must be surprising and uncertain, so that participants are able to react appropriately when confronted with a real situation. An immersive crisis exercise must be based on realistic and believable scenarios, which reflect the types of crises the participants might face (Ulsamer et al. 2021). This in turn will prepare participants to think quickly and make important decisions under pressure (Campanale 1963).

This study is driven by an interest in understanding the extent to which virtual training can complement or replace traditional in-person training. If virtual training proves effective, the research aims to identify which targeted features, tailored to specific crisis scenarios, could be integrated into this approach. This exploration is guided by the following research questions:

**RQ1.** *How does the perception of physical, emotional, environmental factors contribute to the level of immersion in crisis training?*

**RQ2.** *What features of immersive simulation could help to adopt this virtual training?*

To address these questions, an experiment was conducted, replicating real-life crisis scenarios, to gather feedback from 11 participants. The findings demonstrate the effectiveness of VR in simulating real-world crises and providing valuable training opportunities. This innovative approach of recreating real-world

scenarios in a simulated environment underscores the potential of VR to enhance training for public safety agencies.

This article starts with an overview of immersive solutions in section 2 related to the RQ, and explains the methodology in section 3, from the observation of real exercises to their virtual training counterparts. Finally, a discussion based on the results and feedback is presented in section 3.5.

## 2 | Literature Review

This study builds upon prior studies that have investigated the effectiveness of immersive systems for crisis training. Research aim to design a system that would allow for the design, monitoring, modification and evaluation of crisis response exercises in an efficient and affordable manner (Van Laere et al. 2021; Khanal et al. 2022). If VR were used as an immersive platform to simulate situations realistically, managers could familiarise themselves with different scenarios and train their response (Van Laere and Lindblom 2019; Kwok et al. 2019). Furthermore, by using a VR simulation system, the exercise could be tailored to the needs of each community, making it more effective. If crisis managers were to use VR simulations, they could possibly be better prepared for crisis situations and thus minimise the negative impacts of crises on societies.

Traditional crisis management exercise and planning methods have been widely used and proven over the years (Hough and Spillan 2005). They have the advantage of being well-established and providing reliable results. However, they can be limited in terms of cost, flexibility and customisation, and may not be as immersive or realistic as VR-based methods (Wheeler et al. 2021). It is by making all these observations that it has been proven that VR exercise can help improve exercise in any field, and in this article the research focuses on the field of crisis management (Li et al. 2013; Druzhinina and Hvannberg 2013).

The articles suggest that perceptions of physical, emotional and environmental factors can contribute to the level of immersion in crisis training. Larsen et al. (2018) found that the emergency situation can be perceived as stressful, complex, chaotic or unpredictable, and can create worry, anxiety or panic.

Atkins and Burnett (2016) found that being trained in key disaster behavioural health interventions can increase resilience and reduce burnout among disaster behavioural health providers. While Haus et al. (2012) suggests that unexpected events in training simulations should expose learners to complex, realistic situations to improve their ability to manage crises. Bacon et al. (2011) and Cortellessa et al. (2011) describe the use of emotional and physiological data to personalise crisis training environments and adjust training exercises to maximise effectiveness. Nevertheless in the literature, articles does not explore which functionality or feature of crisis training can enhance the level of immersion and effectiveness.

The absence of an analysis on the particular features of immersive training in crisis situations is particularly noteworthy. As an increasing number of organisations utilise immersive technologies to enhance crisis management training, it is vital to comprehend
the influence of these specific features on learning. Within this
field, it is important to distinguish between task-oriented and user-
oriented learning. A comprehensive understanding of the impact
of immersive crisis management training may be hindered if these
aspects are not analysed in depth.

Articles in literature explain that adoption of VR is becoming
concrete. These technologies are becoming more and more
widely accepted (Ustun et al. 2023; Sagnier et al. 2019). And in
some cases, their use is even becoming commercialised. Their
acceptance is subject to improvements in hardware and the
integration of tools and software.

Exercise writers and animation directors are generally referred to as exercise animator according to official documents on crisis management training and its scientific literature. The authoring aspect (Smith et al. 1999) is mostly quoted has a great solution to add but not enough work on it are in the literature.

In this review, studies show limited exploration around im-
mersive simulation features. And all the more so since there is
no research concerning these functionalities during crisis
management training. While the application must be produced
around the need. For this article, the aim will be to examine the
transcription of a real exercise into an immersive serious game
to see which features is more important for users and their
needs in crisis management training.

3 | Methods

Producing digital and scalable virtual exercises in the crisis field
is becoming increasingly easy to simulate and accessible for
persons to use. The goal of this article is to understand better
the features that allow participants of these exercises to feel
immersed in a way that is similar to in-person field exercises
and lead to their adoption by groups implementing them. The
authors of the article were able to observe two exercises, adapt
scenarios from those exercises, and then pilot these scenarios in
VR exercises.

## 3.1 | Observations

Members of the research team participated in two in-person
field exercises that were carried out by the Prefecture of Tarn

(France) in coordination with all first responder groups in the
area (i.e. police, firefighter, and medical responders, decision
makers) and the employees working within each setting. One fo-
cused on an event taking place in a school while another took
place in a historic cathedral that serves as a major landmark for
the city. The research team observed all aspects of the exercise
but spent a great deal of time with the exercise animation team.
The exercise animation team serves as the closest approxima-
tion to the role of an exercise writer in a VR exercise. Spending
time with the design team also allowed for high-level observa-
tion and feedback about how various parts of the exercise were
being carried out. Notes, pictures, and videos were used to
gather feedback about the observations.

3.1.1 | Observation 1: High School

The first exercise, illustrated in Figure 1, involved exercise in a French high school with the objective of confronting a mass casualty attack. At the various observation sites, observations were made of the entire exercise process. For this exercise, it took several months to prepare. On the day of the exercise, a scenario sheet is followed between the scriptwriters and observers. More than 100 persons were involved in the exercise. The attack started at 7.30 pm and resolved at 10.30 pm.

3.1.2 | Observation 2: Cathedral

For the second exercise, illustrated in Figure 2, it involved fires
in a French Cathedral at different location and firefighters had
to extinguish fires and also in the same time evacuate paints
and statues. The objectives were to work on the evacuation of
the painting and statues and also to try the new equipment
inside the Cathedral. More than 90 persons were involved in the
exercise. The fire was detected at 8 pm and the exercise finished
at 10 pm.

## 3.2 | Translating Scenarios Into Virtual Reality Exercises

Based on scenarios and the observations, VR exercises were
designed to be a close approximation of the immersive experi-
ence of the in-person field exercises.

<figure><img src="image_3.png" /><figcaption>**FIGURE 1** | Two official photos published by the Préfecture du Tarn, France, on social and local medias in December 2022 of the exercise of the attack's shooter at the high school. At left, several people exchange ideas to define an action plan. At right, a team of police continue their progress down a corridor.</figcaption></figure>
<figure><img src="image_4.png" /><figcaption>**FIGURE 2** | Two official photos published by the Préfecture du Tarn, France, on social and local medias in May 2023 of the exercise on the painting's evacuation and extinguishing fire at the cathedral. At left, at the top of the cathedral of a fire chief using his radio. At right, a view of the cathedral from above, showing the fire trucks and crews setting up.</figcaption></figure>

The aim was not to transcribe the entirety of the two scenarios, but to extract isolated sequences. As the real-life exercises were too long (around 2–3 h each), the experiments focused on virtual sequences lasting less than 10 min. To design similar scenarios, the authors had access to various documents such as the complete exercise scenarios. And depending on the degree of confidentiality, the scenarios were modified so that everyone could try them out. The scenarios were therefore simplified to make them more accessible, while retaining their pedagogical objectives. So as not to make the sequences too difficult to experience, the school shooter exercise has only been reproduced through the observation aspect. So there is no action to perform. Two similar scenarios have been defined for the cathedral exercise. Both would involve firefighters acting on the crisis. One to reduce civilian and material losses and another scenario to reduce the source of danger (the fire). The last scenario is based on the feedback of the animation team, designing trainings is complicated and the wish is to have a tool to create and modify the scenario during the training the crisis.

No characters were included in the scenarios specifically to help and guide. If the crisis situation was not sufficiently explicit, the participant would receive oral help from the person in charge of the experiment. For example, the person can indicate where an object is if the participant has passed by it without seeing it. Or in other cases, to show how to handle a fire extinguisher.

The equipment used was the same for all participants. A Vive Focus 3 headset and its controllers. The stream was limited to 90 Hz to allow everyone to have the same stable performance. The application has not been launched internally to make the headset self-sufficient. Here, the application is launched from the computer to use its performance.

### 3.2.1 | Scenario 1: Extinguish Fire

The first scenario was to extinguish a fire in the ceiling of the cathedral. The participant needs to understand how to move in the environment. And then to follow firefighters avatars to go to the upstairs to be teleport to the ceiling. In Figure 3, the participant can see non-player character (NPC) going to the upstairs, the user has to follow them.

Arriving at the ceiling, the participant needs to grab the extinguisher and pull the trigger of the controller to activate the

water. Participants complete this scenario after directing the fire extinguisher at the fire for 10 s. The accompanying instructor then removes the headset and explains that the fire is spreading too quickly, requiring additional support.

### 3.2.2 | Scenario 2: Evacuating a Painting

The second simulation focus on the evacuation of works of art. As in the Figure 4, participants were put in the role of firefighters and had to evacuate works of art efficiently using existing safety procedures and emergency plans. People can take their time to observe the environment, and witness a parallel evacuation of tourists by non-player firefighter characters. Participants had to choose which route to take to evacuate the objects, and to manage navigation and interaction with the objects.

There are two possible routes to the drop-off point, one much shorter than the other. These paths are not indicated visually, but only verbally by the person in charge of the observation of the experimentation. As in a real-life situation, the firefighters do not necessarily have a guide and must follow sometimes limited radio indications.

### 3.2.3 | Scenario 3: Observing a Shooter in a Corridor

The third simulation was a scenario of a shooter moving through a school corridor. The participants had to manage their reactions to this violent situation. As in the Figure 5 in the picture at left, the user is in the corridor and watching NPC. And as comparison at the right of the Figure 5, there is the real location.

The participants also had to observe the simulation without having any interaction. This is the least technically challenging scenario, as participants do not have to move or use objects. Here, they are not represented in the environment, they are just invisible observers.

### 3.2.4 | Scenario 4: Exercise Animator

The fourth simulation was the creation of a crisis by moving and triggering events. The participant has to activate a fire in a
<figure><img src="image_5.png" /><figcaption>**FIGURE 3** | Comparison of similar activity with screenshot at left (a) from the point of view of a participant evacuating in VR a paint and at right and with (b) official photo published by the Préfecture du Tarn, France, on Social and local Medias in May 2023 showing a firefighter in the Cathedrale during the real exercise.</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>**FIGURE 4** | Comparison of similar activity with screenshot at left (a) from the point of view of the participant evacuating in VR a paint and at right (b) photos published by the Préfecture du Tarn, France, on Social and local Medias in May 2023 showing a firefighter protecting a painting during the real exercise.</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>**FIGURE 5** | Comparison of the same place, at left (a) the screenshot in the virtual environment from the point of view of the participant observing the shooter and at right (b) the real corridor with the same point of view.</figcaption></figure>

cathedral and also to go to the school to activate the shooter in the corridor. This role allows flying in the environment. The participants had to manage the navigation and interact with the proposed interfaces.

From the authors' point of view as seen in the Figure 6, this is the most difficult simulation to hold on. The controls are not the same as for the other simulations. The character can move completely freely through the environment and select objects from a distance. This flying navigation can cause motion sickness more quickly. And that is why it was only suggested to those who can handle it.

All these scenarios were practiced with the aim of understanding which features could have an impact on people's immersion, their feelings about these technological approaches and their sense of learning.

### 3.3 | Pilot of Virtual Reality Exercise

To try to understand how people can react to these adapted scenarios and to answer to the research questions, several experiments have been carried out on targeted population groups. The aim is to analyse whether these VR exercise can be effective
<figure><img src="image_8.png" /><figcaption>**FIGURE 6** | At left (a) a photo showing the setup during experimentation and one of the participant. At right (b), the screenshot of the point of view of the exercise animator.</figcaption></figure>

and successful enough, including crisis management professional or non-professional.

VR experiences was carried out with 11 individuals from the United States. The recruited participants may have already experienced crises. Some as professionals and others as victims or first responders. Of these 11 people, 7 were considered civilians without experience in the field of crisis management and 4 as crisis professionals. These people were from 21 to 59 years old. During and after each simulation, a series of questions were asked using an online survey to distinguish the strengths and weaknesses of each use case. The survey included questions regarding the physical, emotional, and environmental immersion that each participant experienced (see Appendix A).

The responses to questions and notes based on observation of the individual utilising the tools were used to analyse themes of participant behaviour and user perception of the exercise.

## 3.4 | Results

### 3.4.1 | Feedback From Observation of Field Exercises

From the literature, the observations and expert reports obtained, the results show the exercises work and could work even better if done more regularly. Improvement could be possible if additional tools were available for the non digital exercise. With regard to training management, a tool for synchronising events could improve the way exercises are run. With this kind of tool, discrepancies and doubts in the scenario could be avoided. It would replace the printed schedule, which is fixed and sensitive to change. In the case of immersive digital simulations, paper schedules would be obsolete thanks to the synchronised environment for all.

For the training concerning the high school, the scenario does therefore evolve at different rates, sometimes causing misunderstandings between the different locations of the crisis. For example, a network rupture may be started with a delay of several tens of minutes, causing potentially impossible exchanges of information. In the end, there is no communication between scriptwriters, except in cases of doubt imposed by an unforeseen event. A technological base shared by all to lock each event would be a significant advantage. The exchange of

information remains a central point, which has its shortcomings, often on a human scale.

For the training concerning the Cathedral, the result made was good and to the assigned objectives. The results show good communication and reactivity from the different participant. It was difficult for some participants to fully engage in their roles during these field exercises. They were surrounded by photographers, journalists, and crisis management professionals who were observing them. These journalists were tasked with reporting on the event afterward, which could increase pressure and disrupt participants' focus. For instance, participants who were not directly involved in the action could speak freely without being reminded to stay in character. Similarly, some civilians were chatting with each other even though they had a role to play and, in a real crisis situation, they would not have been standing side by side. Despite some role-playing errors on the part of some participants, the exercise ran smoothly from start to finish. Non-professionals in crisis management were introduced to the various actions to be taken in such a situation. And the action plans will evolve according to the feedback received. A number of remarks were made about the location of protective equipment.

For both exercises, documents were issued for the exercise organisers to notify all the points that could be improved from the point of view of the researchers or consolidated by the digital approach that can be added. Each exercise is every time conduct on a briefing called experience feedback. During the post-training debriefings, not everyone was able to participate because the venues were not designed to accommodate all participants. This can create challenges in teaching effectively and ensuring that all perspectives are heard. During these debriefings, issues such as lack of communication and discrepancies in information reporting, particularly regarding the scenario, were frequently raised. Feedback reports provide good documentation for identifying compliant and non-compliant resources. This allows to establish areas for improving knowledge and skills Merian (2022).

It was suggested that several immersive exercises be carried out, so that comparisons could be made on the advantages and disadvantages of each approach. To compare or at least understand, the following section describes the method used to analyse the different functionalities. That could have an impact on VR immersion in crisis simulations.
3.4.2 | Overview

The article details the overall results, to give an overview. The aim is to find information and feedback common to all scenarios. These are then explored in greater detail in the following sections.

To answer to Research Question 1, the Table 1 shows partici-
pant's average, minimum and maximum values for each ques-
tion, depending of the scenarios.

The averages results are similar for each scenario, but had
different results on specific questions. Each result will be ex-
plored in greater depth through the testimonials of users who
have made associations with the functionalities. To address the
issues raised in the article, a minimum score of 3 is expected.
This will help us to understand our strengths and weaknesses.
Scores above 3 are considered positive. The various results
support the hypothesis that the general level of immersion is
sufficient to obtain interesting answers. Overall, the crisis
simulations worked well for the participants and they were able
to carry out their tasks without too many difficulties.

The aim of this article was not to analyse motion sickness. Motion sickness is described as a feeling of nausea that can sometimes disorientate users immersed in VR. Motion sickness needs to be integrated in the development on a commercial application. This is something that is strongly recommended for any usage of VR, but the article didn't focus on motion sickness, and it didn't impact the majority of participants. No features have been added to reduce this effect, such as the shaded tunnel effect on the field of view (Zhang et al. 2021).

For the question 19, the Figure 7 shows the difference of the
time measured by the participants them-self. Although the
scenarios were short, some participants entered longer times in
their questionnaires. This is the case with the painting scenario,
where 2 participants answered for 20 and 22 min. On average,
the other participants in this scenario answered for less than
5 min. The participants who answered these particular times
nevertheless answered that they had not had any difficulty with
the tasks and did not verbally express any problems.

The Scenario 4 Exercise Animator scenario shows the highest averages for Q2 and Q3. However, since it was assessed by fewer participants in comparison with the other scenarios, it is essential to analyse these results alongside each other. From this analysis, it became evident that the S2 Evacuating scenario is better rated for physical aspects, such as visual and sound quality, as well as general physical features. Scenario S1 Firefighting also rates highly in terms of physics, whereas scenario S3 Observing is marginally less positively reviewed for its audio and visual quality. However, it is important to acknowledge that there is variation in the responses of participants, and individual preferences may differ.

Scenario S3 Observing shows the highest average for emotion-
related queries (Q5-Q8). This finding implies that of the first
three scenarios, scenario S3 is the most emotionally appealing.
Considering the fourth scenario, scenario S4 Exercise Animator
presents the highest averages for Q5-Emotional and

Q6-Comfortable. This indicates that this scenario is regarded as
providing the most satisfactory emotional experience.

For the question 9, answers illustrated in Figure 8 shows that
most of the participants felt their simulations as a original
approach able to surprise them. And for the bystander scenario
it would also provoke empathy for student attacked with fear
and sadness for this situation of crisis.

The survey results reveal significant differences between the scenarios with regard to the environment. Indeed, if the focus is on the overall environment and its structures, scenario 4 Exercise Animator stands out with the highest average. This favourable distinction is related to the fact that this is the role that combines all environments (Cathedral and School). All this while being able to stand back and see the buildings from all sides. The fact that the participant can fly over and go into every place favours this preference for participants. However, if the focus is on the number of objects and their usefulness, it is scenario S2 Evacuating that is favoured according to the results. In fact, this role is distinguished by its proximity to other characters and its ability to handle paints to evacuate them.

Considering all three categories, the S3 Observing situation
(without S4 Exercise Animator) demonstrates the highest
overall average, indicating that it is the most immersive sce-
nario in terms of physical, emotional, and environmental as-
pects. If assessed, S4 Exercise Animator would be positioned as
more captivating than S3 Observing. S3 Observing and S4 Ex-
ercise Animator seems to have the most features necessary to
have a great level of immersion for crisis training.

To the question ‘Did you hear another person in the environ-
ment,’ most of the people say no, and explains that they only
hear effects such as a siren or water from a fire extinguisher.
One person thought he heard people speaking on S1 Extin-
guishing and S2 Evacuating, and had doubts after completing
S3 Observing. This person explained that the speed of events
and the fact of running into people without taking the time to
do so accentuated the panic effect. To answer to Research
Question 2, the sensory-centred question shows that audio and
visual are important for users. One participant testifies that the
contribution of sound effect immerses him as much as the
visual. The participant adds: ‘For example if there was bullet
noise it would be more scary for me.’

3.4.3 | Scenario 1 and 2: Extinguishing Fire and Evacuating a Painting

These two scenarios are similar in terms of functionality and
approach. Both require participants to move around with an
object to perform tasks indicated by a tablet on the arms. The
article therefore presents them grouped together for results.

S1 (Extinguishing) appears to excel primarily in question
Q2-Physics when compared to the other scenarios, exhibiting
a difference of 0.4 in comparison to S2 (Evacuating) and
S3(Observing). S2 (Evacuating) performed better than the
other scenarios in question Q8-Impacted by Other, with a
difference of 0.5 compared to S1 (Extinguishing). Nonetheless,
<table><caption>TABLE 1 | Overview results of the survey on the different scenario. 3 values is set by cell: average, minimum and maximum answered.</caption><thead><tr><th></th><th>S1 Extinguishing</th><th>S2 Evacuating</th><th>S3 Observing</th><th>S4 Exercise animator</th></tr></thead><tbody><tr><td>Number of participants</td><td>7</td><td>7</td><td>7</td><td>4</td></tr><tr><td>Phys. questions</td><td></td><td></td><td></td><td></td></tr><tr><td>Q1-Did the scenario engage your [...] visual/audio/touch experience?</td><td>3.6, min:1, max:5</td><td>4, min:1, max:5</td><td>3.2, min:1, max:5</td><td>3.4, min:1, max:5</td></tr><tr><td>Q2-Laws of physics were applied to movements?</td><td>4.4, min:3, max:5</td><td>4.3, min:3, max:5</td><td>4, min:1, max:5</td><td>4.5, min:3, max:5</td></tr><tr><td>Q3-Did you feel physical resistance when touching them?</td><td>1.9, min:1, max:5</td><td>2.3, min:1, max:4</td><td>1.9, min:1, max:5</td><td>4.5, min:4, max:5</td></tr><tr><td>Q4-Was your body to move coherently with the avatar?</td><td>4.7, min:4, max:5</td><td>4.6, min:4, max:5</td><td>4.7, min:3, max:5</td><td>3.1, min:1, max:4</td></tr><tr><td>Emotion questions</td><td></td><td></td><td></td><td></td></tr><tr><td>Q5-Did you feel emotional during your experience?</td><td>3.3, min:1, max:5</td><td>3.7, min:2, max:5</td><td>3.4, min:1, max:5</td><td>3.8, min:1, max:5</td></tr><tr><td>Q6-Did you feel comfortable during the experience?</td><td>4.7, min:4, max:5</td><td>4, min:1, max:5</td><td>4.9, min:4, max:5</td><td>4.8, min:4, max:5</td></tr><tr><td>Q7-To what degree did you feel emotion seeing other avatars?</td><td>3.7, min:1, max:5</td><td>3.6, min:1, max:5</td><td>3.6, min:1, max:5</td><td>3.5, min:1, max:5</td></tr><tr><td>Q8-Did you feel the environment being impacted by other persons?</td><td>2.4, min:1, max:5</td><td>2.9, min:1, max:5</td><td>3.3, min:1, max:5</td><td>2.8, min:1, max:5</td></tr><tr><td>Env. questions</td><td></td><td></td><td></td><td></td></tr><tr><td>Q10-Did you have opportunity to interact with multiple object?</td><td>3.3, min:1, max:5</td><td>4.7, min:4, max:5</td><td>2, min:1, max:5</td><td>3.8, min:2, max:5</td></tr><tr><td>Q11-Do you wish having more objects to interact with?</td><td>3.3, min:1, max:5</td><td>3.9, min:1, max:5</td><td>4, min:1, max:5</td><td>3, min:1, max:5</td></tr><tr><td>Q12-Did the buildings of the environment engage you?</td><td>4.6, min:3, max:5</td><td>4.6, min:4, max:5</td><td>4, min:2, max:5</td><td>4.5, min:4, max:5</td></tr><tr><td>Q13-Did the avatars of the environment engage you?</td><td>4.4, min:4, max:5</td><td>3.7, min:1, max:5</td><td>3.3, min:1, max:5</td><td>4.8, min:4, max:5</td></tr><tr><td>Q14-General feedback on sound/touch?</td><td>3.6, min:1, max:5</td><td>3.7, min:1, max:5</td><td>2.7, min:1, max:5</td><td>4.5, min:3, max:5</td></tr><tr><td>Q15-Did the coherence of the environment to you?</td><td>4.1, min:3, max:5</td><td>4.3, min:4, max:5</td><td>4.3, min:4, max:5</td><td>4.5, min:4, max:5</td></tr><tr><td>Q16 - Did the environment react to your actions?</td><td>4.7, min:4, max:5</td><td>4.4, min:4, max:5</td><td>3, min:1, max:5</td><td>5, min:5, max:5</td></tr><tr><td>Q17-Did you hear another person in the environment?</td><td>yes, no</td><td>1 yes, 6 no</td><td>0 yes, 7 no</td><td>1 yes, 3 no</td></tr><tr><td>Q18-Did you have difficulties to do tasks?</td><td>2.1, min:1, max:4</td><td>2.4, min:1, max:4</td><td>1.3, min:1, max:3</td><td>2, min:1, max:4</td></tr><tr><td>Average</td><td>3.67</td><td>3.85</td><td>3.37</td><td>3.82</td></tr></tbody></table>
<figure><img src="image_9.png" /><figcaption>**FIGURE 7** | Comparison graph from results timing felt answered.</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>**FIGURE 8** | Comparison graph from results emotions answered.</figcaption></figure>

S2 (Evacuating) does not possess a significant advantage over S1 (Extinguishing), S3 (Observing), or S4 (Exercise Animator) in the remaining questions.

Doing longer scenario can also create good and pleasant experience, for example one participant went down from the cathedral ceiling and evacuate holding the extinguisher and saying 'it's just for fun' forgetting the crisis situation.

While the scenario of evacuating the works seems one of the most difficult. One person was able to take two paintings in the same time frame as the others, and was able to move around easily without getting lost. The participant suggests that perhaps it's his ease with video games that

makes him better able to orientate himself and understand the tasks.

### 3.4.4 | Scenario 3: Observing a Shooter in a Corridor

In the third scenario, participants were able to observe the appearance of the gunman, and all reacted to this scenario either physically or verbally. They all agreed that the speed of the staging was striking.

This scenario has the lowest average for Q18-Difficulties Task, indicating less difficulty in this specific task than the other scenarios. The S3 (Observing) scenario is particularly notable
concerning questions Q6-Comfortable, Q5-Emotional and Q8-Impacted by Other, as is evident from the numerical disparities in scores in these specific questions compared to the other scenarios.

Four people said they had difficulty observing the sequence without moving to hide. After this discovery phase, if the participants didn't understand that the shooter couldn't see them, the evaluator informed them. Three persons had a physical reflex to move back or protect themselves when the shooter got too close to them.

One participant stated: 'No face, less empathy' talking about the faces of the civilian avatars attacked in the corridor. For professional participants, the effect is more neutral. Whereas, on the contrary, another participant said 'The fire was okay, but the shooter is scarier' and it was more difficult to manipulate the controllers after.

### 3.4.5 | Scenario 4: Exercise Animator

This scenario recorded the lowest participation during the tests. This limitation can be attributed in part to time constraints, as this scenario was generally presented last, and the other scenarios could sometimes take longer than expected. In addition, this scenario generated fewer explicit comments from participants. Overall, opinions were positive and encouraging, but there was also some nuance regarding its potential integration. One professional participating in the tests noted the practical challenges of using and teaching these new tools, underlining the need for a period of adaptation and appropriate training for professional users.

For question Q8-Impacted by Other, S4 scores slightly better (difference of 0.4). than S1 (Extinguishing) It is important to note that overall, the S4 (Exercise Animator) scenario seems to score less well in these specific questions compared to the other scenarios, particularly when it comes to movement consistency (Q4).

This specific configuration stands out as potentially the least ergonomic and accessible of the various alternatives considered. It presents significant challenges in terms of user experience. The interfaces implemented can be complex to use, potentially requiring extensive revision to ensure a fluid and intuitive experience. What is more, the interface is positioned relative to the left hand and has no stabilisation. As a result, the interface will move unintentionally, even when the participant is using it. And the buttons can be too small and difficult to select for multiple participants. This reduces comfort and makes the experience more laborious.

During testing, one participant pointed out a lack of clear visual clues in this particular configuration. The current interface seems insufficiently informative to help users understand the various possible actions and anticipate their consequences. This observation underlines the need to improve the visual representation of the virtual environment, so as to effectively guide users throughout their experience.

This setup poses ergonomic and accessibility challenges that deserve particular attention. Feedback from participants, while generally positive, highlights opportunities for improvement to ensure a smoother user experience and successful integration of these emerging technologies. To qualify these results, it is worth noting that not all users had used this kind of tool before.

To explore these results further, the discussion section attempts to interpret the data and conclude on what it implies.

## 3.5 | Discussion

In this experiment, the results were analysed to assess the effectiveness of the VR exercise. The results showed that VR is an effective tool for improving crisis preparedness and response. This is the case with comparable outcomes observed among the seven participants who used virtual reality and the four who did not. Additionally, similar results were observed between the four participants possessing crisis management skills and occupations and the seven without. Non-crisis experts are mostly enthusiastic about being able to project themselves into difficult situations to better understand and react to real incidents. Stressful situations are experienced with more vivid reactions. The professionals react well and do not seem to be afraid. They are able to project themselves into the future and into the potential help of this tool. And the pros react more in this case. They share their concerns and suggest modifications and features.

### 3.5.1 | Factors on Level of Immersion

For some simulations, users have suggested features on their own. For example, one person proposed to be able to activate or not the sounds, and also to add the bullet impacts in the environment or the blood. This person was not considered as crisis professional, but has first aided skills and knowledge. This person had thought that depending on the severity of the crisis experience, these configurable realism thresholds could help.

The seven non-professional users explained that they were more interested in the simulation in which they were in the role of the firefighter. The four professionals, on the other hand, are more in favour of the possibility of creating a scenario and being able to edit it as they wish, so that others can practice it. However, they explain that the tool is not yet mature and stable enough to be used.

It seems that *Evacuating the painting* was the most interesting for all participants, they had the feeling that they did a long and original scenario. It is the one which had the most positive feedback emotion results, even if it was difficult for them to do the task.

This might suggest that a longer scenario could make the mission more difficult if the task does not include pleasant features. This could lead to frustration if the features are not accessible and adapted to the person. This is not new, and it is in line with the idea of providing simulations and applications that adapt to the person, to accompany them in a playful progression.
### 3.5.2 | Features Feedback

To address the second research query, a range of characteristics was created based on both those used in the experiment and those suggested during the interviews.

The implementation of *suggestive interactions* and hovering visual guides reveals an approach to facilitating the learning and mastery of serious game mechanics. This could take the form of a contextual tutorial. The aim of this feature is to promote progressive, autonomous discovery of the serious game's functionalities, while providing unobtrusive guidance for users.

The integration of variable *difficulty levels* demonstrates a heightened sensitivity to the heterogeneous needs and skills of the stakeholders involved in crisis management. This variable should be measured in advance to guarantee that the experience is tailored from the outset. This feature makes it possible to adapt the experience according to the level of expertise of users, thus encouraging the participation of various categories of stakeholders.

Taking into account individual preferences regarding levels of *graphic violence* in the game reflects an ethical and responsible approach to the development of crisis management simulations. This feature aims to avoid inappropriate or potentially traumatising content, while offering users the opportunity to personalise their gaming experience according to their sensitivities. For example, the graphic integration of blood or the sound of screaming can be added or deactivated according to the user. But to know this limit, you need to know beforehand where the line is between effective and too violent for the specific user.

The integration of *expressive facial animations* on avatars represents a significant advance in the design of crisis management simulation games. By using cutting-edge technologies to capture and reproduce the emotions of virtual characters, this feature aims to enhance users' emotional immersion. It also facilitates non-verbal communication between stakeholders, promoting mutual understanding and cooperation in crisis situations where effective communication is crucial.

The implementation of a well-designed inventory system with accessories such as belts and bags demonstrates a practical approach to the creation of crisis management training simulations. This functionality aims to improve user efficiency by enabling them to easily transport and access essential resources during critical situations. By providing a realistic and functional inventory system, the designers increase the realism and relevance of the games, thereby reinforcing the preparedness and skills of the responders in crisis contexts.

The incorporation of realistic sounds for objects and avatars, in addition to environmental sounds, demonstrates an immersive approach to the design of crisis management training. This feature aims to enhance users' sensory experience, by creating an authentic and captivating sound environment. The precise sounds of actions, interactions, and the movements of objects and characters help to increase users' emotional engagement, enhancing the effectiveness of simulations.

### 3.5.3 | Adoption

From a civilian perspective, the adoption of VR in crisis management offers a number of encouraging benefits. It allows citizens to better understand the risks they may face and the measures they can take to protect themselves. By simulating situations in realistic virtual environments, citizens can be better prepared and more confident to deal with real-life emergencies. These exercises can also improve communication and coordination between authorities and citizens. Using virtual simulations, authorities can better understand the needs and concerns of citizens and adapt their planning and response accordingly. The use of virtual reality can also improve citizen participation in prevention efforts. Citizens can participate in virtual simulations and provide feedback on prevention and response measures, which can encourage them to be more active and involved in protecting their community. They encourage those opportunities and would like to add functionalities as guided signs in the virtual environment to help.

Technological advances in the field of VR can improve exercise and planning methods. Professionals may be reluctant to integrate this technology due to the lack of government support and innovation in VR. It is crucial that government authorities invest in the research and development of this technology to ensure its effectiveness and relevance to community needs. Without adequate government support, it can be difficult to establish effective VR-based exercise and planning programs. There is also a need for policies and standards to ensure the quality and safety of VR-based exercise and planning programs.

This is evident in meetings with a diverse audience of professionals and non-professionals. These testimonies confirm their agreement with the integration of new tools to complement the old ones. Exercises, which are rarely carried out, should be carried out more regularly, according to the professionals. According to the latter, there should be 2 to 3 times more exercises. And the contribution of these new tools could be a complementary solution to this problem.

## 4 | Limitations and Future Work

This study provides information about the involvement created in the exercise, but it also has some limitations. First, the study involved a limited number of exercise professionals. Additional perspectives from professionals would be beneficial for the further development of this tool. Additionally, the scenarios were presented to participants as part of a research experiment rather than a formal training setting. Future work should expand the study to include these considerations and settings. Another limitation is that the questionnaire used in the study was only administered to individuals who engaged in VR crisis training. There is no comparable data set for individuals who participated in VR games or in-person training activities. Future work should compare the results of the questionnaire across all three settings: virtual crisis training, in-person crisis training, and VR games. Furthermore, the questionnaire could be improved for instance, the questionnaire could be categorised from the environment view, as suggested by (Congès 2022), or from a more human-centred perspective, as in the work of (Tao et al. 2022).
The methodology employed in this study demonstrates the potential for shifting from in-person training to their integration into simulated training encompassing various VR scenarios. The evaluation of these scenarios revealed both positive and negative outcomes. It is conceivable to envisage more comprehensive exercises that incorporate a wider range of scenarios and challenges. These exercises can benefit crisis management professionals, civilians, and learners alike. Future work should focus on expanding the study to include a larger and more diverse group of participants. Additionally, future research should investigate the long-term effects of virtual reality crisis training on participants' performance and confidence in real-world crisis situations.

5 | Conclusion

Lessons learned from real-world training exercises highlight both the challenges and opportunities of replicating in-person scenarios in a virtual environment. While adapting these exercises to a digital scale presents obstacles, virtual training—grounded in real-life crisis simulations—can still achieve a high level of immersion. The integration of physical, emotional, and environmental factors creates an experience immersive enough to meet the pedagogical objectives of crisis training. Therefore, this conclusion directly addresses the first research question posed in this article.

And to answers to the second research question, the article's approach enables to observe various functionalities that have been implemented and those that are projected. The functionalities and their level of involvement in the application will not be uniform and depend on the training requirements.

Serious games are already used in various fields for training purposes. Their digital format provides added benefits, including cost-efficiency, repeatability, realism, adaptability, and usability. With the aid of VR simulations, these games yield a perception akin to in-person training. Additionally, the games can be customised to fit the user's requirements by adjusting different functionalities. In this way, one can achieve varying levels of awareness concerning violence or crisis, depending on the degree to which the individual is immersed within a challenging context.

This technique would enable professionals to better equip themselves for genuine crisis scenarios and maximise the potential of immersive technologies in their crisis-management training procedures. Nevertheless, creating a crisis training exercise that is adjustable for all users is a difficult task.

These innovative strategies might be incorporated into professionals' comprehensive training. The tool's accessibility enables crisis professionals to recreate any situation entirely safely. Additionally, they may be utilised in workshops aimed at raising awareness among non-professionals.

Adaptive training (Kelley 1969) in this form exploits all the components to offer dedicated, personalised training. Adaptive training adjusts according to the performance of the trainee, tailoring both content and feedback to match their specific

needs and abilities. Within VR contexts, this process relies on three crucial elements: measuring the performance of the learner, adaptive logic, and adaptive variables. These components work together automatically to adjust the simulation and feedback, improving the overall effectiveness of the training. These novel tools enable scenarios and educational content to be incorporated into exercises while offering a personalised visual and audio format.

This type of exercise, however short, could in future become a new way of informing and teaching safety measures for public transport, high-risk buildings, etc. Just like demonstrations of emergency procedures at the start of an airplane take-off. A form of active training in which everyone participates from their own point of view and at their own level, together or separately.

Acknowledgements

This study work was carried out within the framework of the EG-CERSIS project funding. The work was carried out with the support of public and European institutions and industrial partners. Thanks to the Prefecture du Tarn for permission to use their photos in this article.

Conflicts of Interest

The authors declare no conflicts of interest.

Data Availability Statement

The data that supports the findings of this study are available in the main content of this article.

References

Atkins, C. D., and H. J. Burnett. 2016. "Specialized Disaster Behavioral Health Training: Its Connection With Response, Practice, Trauma Health, and Resilience." *Disaster Health* 3, no. 2: 57-65. https://doi.org/10.1080/21665044.2016.1199151.

Benaben, F., M. Lauras, S. Truptil, et al. 2016. "A Metamodel for Knowledge Management in Crisis Management." *In* 2016 *49th Hawaii International Conference on System Sciences (HICSS)*: 126-135. IEEE. https://doi.org/10.1109/HICSS.2016.24.

Bacon, L., G. Windall, and L. MacKinnon. 2011. "The Development of a Rich Multimedia Training Environment for Crisis Management: Using Emotional Affect to Enhance Learning." *Research in Learning Technology* 19, no. suppl 1: 7780. https://doi.org/10.3402/rlt.v19s1/7780.

Campanale, R. P. 1963. "Realism in Disaster Exercises—A True Challenge, Realism in Disaster Exercises—A True Challenge." *Military Medicine* 128, no. 5: 418-427. https://doi.org/10.1093/milmed/128.5.418.

Cartapanis, A. 2004. "Le Déclenchement Des Crises De Change: Qu'avons-Nous Appris Depuis Dix Ans?" *Economie internationale* 97, no. 1: 5-48. https://doi.org/10.3917/ecoi.097.0005.

Congès, A. 2022. "Conception and Evaluation of Virtual Environments Dedicated to Crisis Management." Thèse de doctorat, Ecole nationale supérieure des Mines d'Albi-Carmaux.

Cortellessa, G., R. D'Amico, M. Pagani, et al. 2011. "Modeling Users of Crisis Training Environments by Integrating Psychological and Physiological Data." *In* Modern Approaches in Applied Intelligence, edited by K. G. Mehrotra, C. K. Mohan, J. C. Oh, et al, 6704, 79-88. Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-642-21827-9_9.

Druzhinina, O., and E. T. Hvannberg. 2013. "Feedback in a Training Simulator for Crisis Management Compared to Feedback in a Real-Life Exercise." *In* *Human Work Interaction Design. Work Analysis and HCI*,
edited by P. Campos, T. Clemmensen, J. A. Nocera, et al, 407, 124-138. Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-642-41145-8.11.

Gill, J. C., and B. D. Malamud. 2014. "Reviewing and Visualizing the Interactions of Natural Hazards: Interactions of Natural Hazards." *Reviews of Geophysics* 52, no. 4: 680-722. https://doi.org/10.1002/2013RG000445.

Haus, M., C. Rooney, J. Barnett, D. Westley, and W. Wong. 2012. "Evaluating the Effect of Startling and Surprising Events in Immersive Training Systems for Emergency Response." *Proceedings of the Human Factors and Ergonomics Society Annual Meeting* 56, no. 1: 2467-2471. https://doi.org/10.1177/1071181312561502.

Hough, M., and J. Spillan. 2005. "Crisis Planning: Increasing Effectiveness, Decreasing Discomfort." *Journal of Business & Economics Research (JBER)* 3, no. 4: 19-24. https://doi.org/10.19030/jber.v3i4.2762.

Kelley, C. R. 1969. "What Is Adaptive Training?" *Human Factors: The Journal of the Human Factors and Ergonomics Society* 11, no. 6: 547-556. https://doi.org/10.1177/001872086901100602.

Khanal, S., U. S. Medasetti, M. Mashal, B. Savage, and R. Khadka. 2022. "Virtual and Augmented Reality in the Disaster Management Technology: A Literature Review of the Past 11 Years." *Frontiers in Virtual Reality* 3: 843195. https://doi.org/10.3389/frvir.2022.843195.

Kwok, P. K., M. Yan, B. K. P. Chan, et al. 2019. "Crisis Management Training Using Discrete-Event Simulation and Virtual Reality Techniques." *Computers and Industrial Engineering* 135: 711-722. https://doi.org/10.1016/j.cie.2019.06.035.

Van Laere, J., J. Lindblom, and M. de Wijse-van Heeswijk. 2021. "Complexifying Facilitation by Immersing in Lived Experiences of on-the-Fly Facilitation." *Simulation & Gaming* 52, no. 3: 346-363. https://doi.org/10.1177/10468781211006751.

Van Laere, J., and J. Lindblom. 2019. "Cultivating a Longitudinal Learning Process Through Recurring Crisis Management Training Exercises in Twelve Swedish Municipalities." *Journal of Contingencies and Crisis Management* 27, no. 1: 38-49. https://doi.org/10.1111/1468-5973.12230.

Larsen, T., R. Beier-Holgersen, D. Østergaard, and P. Dieckmann. 2018. "Training Residents to Lead Emergency Teams: A Qualitative Review of Barriers, Challenges and Learning Goals." *Heliyon* 4, no. 12: e01037. https://doi.org/10.1016/j.heliyon.2018.e01037.

Li, X. L., H. Li, M. G. Cheng, and Z. Q. Fan. 2013. "Framework for Emergency Decision Exercise System of Urban Crisis Based on War-gaming." *Applied Mechanics and Materials* 373-375: 1139-1143. https://doi.org/10.4028/www.scientific.net/AMM.373-375.1139.

Merian, Y. (2022) Apports et Difficultés Du Retour D'expérience (RETEX/REX) Pour La Gestion De Crise Et La Résilience. https://hal.science/hal-03966596.

Murria, J. 2005. "Review of La Fin Du Risque Zero by Xavier Guilhou and Patrick Lagadecla Fin Du Risque Zeroeyrolles Editeur 22-7081-2707-1: 2002, Eyrolles Editeur." *Natural Hazards Review* 6, no. 1: 54. https://doi.org/10.1061/(ASCE)1527-6988(2005)6:1(54).

Mystakidis, S., and V. Lympouridis. 2023. "Immersive Learning." *Encyclopedia* 3, no. 2: 396-405. https://doi.org/10.3390/encyclopedia3020026.

Reason, J. 1997. "Managing the Risks of Organizational Accidents. Brookfield, Aldershot Sagnier C, Loup-Escande Valléry G (2019) Technology Acceptance of Virtual Reality: A Review." *Le Travail Humain* 82, no. 3: 183-212. https://www.cairn-int.info/article.php.

Sagnier, C., É. Loup-Escande, and G. Vallèry. 2019. "Technology Acceptance of Virtual Reality: A Review." *Le travail humain* 82, no. 3: 183-212. https://shs.cairn.info/journal-le-travail-humain-2019-3-page-183?lang=en.

Smith, W., J. Dowell, and M. A. Ortega-Lafuente. 1999. "Designing Paper Disasters: An Authoring Environment for Developing Training

Exercises in Integrated Emergency Management." *Cognition, Technology and Work* 1, no. 2: 119-132. https://doi.org/10.1007/s101110050038.

Tao, K., Y. Huang, Y. Shen, and L. Sun. 2022. "Automated Stress Recognition Using Supervised Learning Classifiers by Interactive Virtual Reality Scenes." *IEEE Transactions on Neural Systems and Rehabilitation Engineering* 30: 2060-2066. https://doi.org/10.1109/TNSRE.2022.3192571.

Ulsamer, P., A. Schutz, and T. Fertig, et al. 2021 "Immersive Storytelling for Information Security Awareness Training in Virtual Reality." In *Proceedings of the Annual Hawaii International Conference on System Sciences. Hawaii International Conference on System Sciences*. https://doi.org/10.24251/hicss.2021.861.

Ustun, A. B., F. G. Karaoglan-Yilmaz, and R. Yilmaz. 2023. "Educa-tional UTAUT Based Virtual Reality Acceptance Scale: A Validity and Reliability Study." *Virtual Reality* 27, no. 2: 1063-1076. https://doi.org/10.1007/s10055-022-00717-4.

Vašíčková, V. 2020. "Crisis Management Process – A Literature Review and a Conceptual Integration." *Acta Oeconomica Pragensia* 27, no. 3-4: 61-77. https://doi.org/10.18267/j.aop.628.

Wallace, W. A., and F. D. Balogh. 1985. "Decision Support Systems for Disaster Management." *Public Administration Review* 45: 134. https://doi.org/10.2307/3135008.

Wheeler, S. G., H. Engelbrecht, and S. Hoermann. 2021. "Human Factors Research in Immersive Virtual Reality Firefighter Training: A Systematic Review." *Frontiers in Virtual Reality* 2: 671664. https://doi.org/10.3389/frvir.2021.671664.

Zhang, Q., H. Yamamura, and H. Baldauf, et al. 2021. "Tunnel Vision – Dynamic Peripheral Vision Blocking Glasses for Reducing Motion Sickness Symptoms." In *Proceedings of the 2021 ACM International Symposium on Wearable Computers* 21, 48-52. ISWC: Association for Computing Machinery. https://doi.org/10.1145/3460421.3478824.

# Appendix A

## Questions Overview

Questions appear in different groups as physical, emotional, environ-ment, and other questions related to VR experience. The final section of the appendix includes a list of other data collected from the observation phase. When the participant is doing the experiment, the person can share their feedback and can answer to small questions.

A.1 Survey before starting the experiment, participants were asked to share the following demographic information

* Age

* Gender

* Occupation

* Experience with virtual reality

A.2 Questions' survey after each scenario

A.2.1 Physical experience

1. To what degree did the scenario engage your senses with regard to visual/audio/touch experience?

2. Laws of physics were applied to movements?

3. Did you feel physical resistance when touching them?

4. Was your body to move coherently with the avatar?

A.2.2 Emotional experience
1. Did you feel emotional during your experience?

2. Did you feel comfortable during the experience?

3. To what degree did you feel emotion seeing other avatars?

4. To what degree did you feel the environment being impacted by other persons?

5. What kind of emotion you felt during the simulation (Fear, Curiosity, Surprise, Disgust, Sadness, Anger)?

### A.2.3 Engagement with the environment

1. Did you have opportunity to interact with multiple objects?

2. Do you wish having more objects to interact with?

3. To what degree did the buildings of the environment engage you?

4. To what degree did the avatars of the environment engage you?

5. General feedback on sound/touch?

6. To what degree did the coherence of the environment to you?

7. To what degree did the environment react to your actions?

8. Did you hear another person in the environment?

9. Did you have difficulties to do tasks?

10. Time to complete the task demanded according to the participant?

### A.3 Other data collected during and/or after the observation phase

* What could you suggest for improvement?

* Perception of motion sickness?

* How do you feel?

* What is the most interesting scenario?