# Towards VR Training Adapted to Affective and Cognitive States: General Method and Evaluation of Mental Workload

Emilie Hummel, Mélanie Cogné, Anatole Lécuyer, Marie Lange, Valérie Gouranton

<figure><img src="image_1.png" /></figure>

To cite this version:

Emilie Hummel, Mélanie Cogné, Anatole Lécuyer, Marie Lange, Valérie Gouranton. Towards VR Training Adapted to Affective and Cognitive States: General Method and Evaluation of Mental Workload. IEEE Transactions on Visualization and Computer Graphics, 2026. hal-05503705

HAL Id: hal-05503705

https://hal.science/hal-05503705v1

Submitted on 10 Feb 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# Towards VR Training Adapted to Affective and Cognitive States: General Method and Evaluation of Mental Workload

Emilie Hummel ID, Mélanie Cogné ID, Anatole Lécuyer ID, Marie Lange ID, and Valérie Gouranton ID

<figure><img src="image_3.png" /><figcaption>Fig. 1: We propose a method to adapt VR training applications to the affective and cognitive states of the user. Our method was implemented and tested within a VR training application involving a sorting task (see Fig. 1-A). During the VR training, the level of difficulty of the task was automatically adapted with regards to the user's mental workload measured using simple, and repeated questionnaires on a virtual discrete scale (see Fig. 1-B).</figcaption></figure>

**Abstract**— Adaptive virtual reality (VR) applications are used in training and rehabilitation to provide personalized experience, through the adaptive logic that adjust the virtual environment based on the user's behavior. While users' behavior is influenced by affective and cognitive states (ACS), adaptive logic typically relies only on users' performance. First, this paper introduces a general method for the adaptation of VR applications that integrates ACS. We then provide an implementation of this model adapting difficulty with regards to both performance and mental workload. Finally, we present a user study (N=30) to compare our mental workload-based method (experimental) to a state-of-the-art adaptation relying on performance only (control). Results show that our adaptation method led to a decrease of 10.7% in mental workload, an increase of 22.8% in performance, and an overall better experience for most participants. These results were achieved without the participants' awareness of the adaptive logic of each condition. Taken together, our results promote the integration of ACS in adaptive VR to enhance users' experience and efficiency, and better fit the function of VR training applications.

**Index Terms**—Virtual reality, adaptation, cognition

## 1 INTRODUCTION

Virtual Reality (VR) applications allow users to explore, interact, and learn in a controlled environment. VR is particularly useful to safely reenact dangerous situations [9], or perform rehabilitation exercises in an immersive environment [32]. The adaptation of a VR task according to the user interaction is made possible by the monitoring of their behavior, including their actions and evolving states [9]. This allows the VR application parameters to be tailored to one or more users [8]. The benefits of adaptive VR are especially highlighted in training or rehabilitation [19, 45] contexts, as it provides a personalized experience adapted for trainees or patients, enhancing their performance by improving var-

ious behavioral aspects [18]. In these contexts, performance is a central feature of the task, as it reflects the user's progress in training or rehabilitation. However, the computation of adaptive elements is constrained by the need for real-time interactions in VR. The “adaptive logic” is behind the automatic adjustment of virtual environment variables based on the user's behavior [20], and performance is the most common input [45]. Yet, a user's progress and success in a given task is determined by various factors, such as their affective and cognitive states. The integration of affective and cognitive states in adaptive training aligns with established links between learning, cognition [23] and affect [13].

Affective and cognitive states (ACS) are mental states characterized by their impacts on behavior and physiological responses. Affective states are defined by Scherer et al. as the states of emotions, influenced by preferences, attitude, mood, and personality traits [36]. Cognitive states, characterized by mental workload, and cognitive load, can be defined as states describing the pressure, effort, and frustration related to the perception and processing of information [3]. Affective and cognitive states are an important part of people's conduct and contribute to their behavior in both real life and VR applications. Although ACS exploitation to provide feedback has been investigated in the literature [30], only one study assesses the use of ACS for the adaptation of the virtual environment [1]. This study [1] focuses on emotional states, demonstrating improved game experience through their inclusion. Conversely, the user study presented in our paper focuses on cognitive states.

In this paper, we present an innovative general method to integrate user's ACS for adaptive VR training (see Fig. 1), with a focus on cognitive states for an implementation of the method and its evaluation. Our

* Emilie Hummel and Valérie Gouranton are with Univ Rennes, INSA Rennes, Inria, CNRS, IRISA, France. E-mail: emilie.hummel@univ-rennes.fr | valerie.gouranton@irisa.fr.

* Mélanie Cogné is with Rehabilitation Unit, Rennes University Hospital, France. E-mail: melanie.cogne@chu-rennes.fr.

* Mélanie Cogné and Anatole Lécuyer are with Inria, Univ. Rennes, IRISA, CNRS, France. E-mail: anatole.lecuyer@inria.fr.

* Marie Lange is with ANTICIPE U1086 INSERM-UCN, Centre François Baclesse, Normandie Université UNICAEN, France, and Clinical Research Department, UNICANCER, Centre François Baclesse, France. E-mail: m.lange@baclesse.unicancer.fr.
method relies on measuring the user's performance and ACS, so to
potentially adapt in real-time several aspects of the virtual environment.
This adaptation is based on the interpretation of user metrics, the desired
level of intervention, and the objectives of the VR application. Then, we
introduce an implementation of this method, which adapts task difficulty
according to performance and mental workload. Finally, we compare
our approach to adaptation based on performance only and assess its
impact on participants' performance and experience.

## 2 RELATED WORK

Flow theory aims to quantify the experience of immersion, focus, and engagement under the term “flow” [7]. A higher flow induces greater motivation, productivity, and enjoyment, which are key elements of training and rehabilitation. Additionally, Cognitive Load Theory [39] brings a better understanding of cognitive mechanisms during training and learning tasks. Users’ flow tends to be higher when the difficulty matches their skills, i.e., the task is neither too difficult nor too easy [25]. Adaptive VR allows to adjust the difficulty to fit users’ skills, but reducing cognitive load in such task requires to take into account the users’ specific features, e.g., age, gender and cognitive skills [22]. The adaptation of difficulty in a VR task has an important impact on users’ experience and outcomes.

### 2.1 Adaptive logic in VR applications

Adaptive VR applications in the literature feature a wide range of
adaptive content, from explicit feedback to complex models integrating
every aspect of the virtual environment [45]. The nature of the adaptation
is determined by the adaptive logic and its parameters. In virtual reality
adaptive training, various variables can be modified using adaptive
logic: the virtual environment, the pressure applied to the trainee,
the parameters of elements controlled by the trainee, the parameters
of elements controlled by the operator, the display, the task problem
generation and the secondary tasks besides the main task [20]. All these
variables have been identified as factors affecting the task's difficulty.
In Kelley's model of virtual reality adaptive training, the difficulty is
defined as the output of the adaptive logic, and the input is the data used
to compute the adaptation [20]. To identify various adaptive VR training
application, Zahabi et al. conducted a systematic literature review and
an analysis of the type of study, adaptive logic, feedback, along with
other aspects [45]. The following sections are partially based on their
work, with a focus on adaptive logic and adaptive logic input.

In the literature, the most frequently used adaptive logic methods
are supervised machine learning algorithms, optimization methods,
and conditional statements, although a part of the reviewed studies
did not describe their adaptive logic [45]. Overall, researchers have
described the use of adaptive VR for training, e.g., with vocational
training [31], driving training [24], piano training [44], or rehabilitation,
e.g., with motor [15, 17] or cognitive rehabilitation [6]. Regardless of
the context of application, the adaptive logic affects the difficulty of the
task, although the definition of the difficulty is context-dependent.

The analysis of Zahabi et al. highlights the variety of VR applications that benefit from the adaptation. The adaptive logic affects these adaptation variables at a different level, to provide a difficulty change fitting one context of use but that cannot be generalized. For example, a user study assessing adaptive driving training in VR for people with autism spectrum disorder defined the variables of the difficulty of the task as the brake pedal, accelerator pedal, and steering wheel responsiveness, the ambient luminosity, and the speed of other vehicles on the road [4]. These variables are relevant for driving simulations, which were essential in their study. However, their results are unlikely to be useful in other user studies on adaptive VR training due to their specific context of driving training.

### 2.2 Variables of adaptive logic

The input of the adaptive logic can be composed of online measures,
i.e., data collected during the VR experience, or offline measures, i.e.,
data collected prior to the VR experience. Here, we mainly focus on
online measures, and we describe the input of adaptive logic used in
the literature and their assessment.

Performance, as a numeric representation of success, is the most
predominantly used input for the adaptive logic as it is fundamental

in the context of training, rehabilitation or games [45]. Performance
measures reflect the user's progress in a training or rehabilitation task
in various forms, such as motion data, accuracy, task completion speed,
task completion score. Cognitive load, affect regulation and expertise
assessment are critical elements to provide an adapted experience [34].
However, little research has been conducted to evaluate ACS as the
adaptive logic input. Examples of studies assessing ACS as adaptive
logic input include emotional state in an experimental study [1] and
mental workload in a feasibility study [11].

Emotions are defined as “an episode of interrelated, synchronized
changes in the states of all or most of the five organismal subsystems
in response to the evaluation of an external or internal stimulus event
as relevant to major concerns of the organism” [35].

Mental workload is defined by Hart and Staveland as the load that “emerges from the interaction between the requirements of a task, the circumstances under which it is performed, and the skills, behaviors, and perceptions of the operator” [16]. With a more concise definition, De Waard defines that mental workload as “the ratio of demand to allocated resources” [10]. ACS are similar in terms of the behavioral and physiological response they trigger. In this regard, ACS such as emotions and mental workload are measured through various models, with objective, e.g., electroencephalography, electrocardiography [12, 41], or subjective measures, e.g., questionnaires [5, 16]. These forms of adaptive logic input bring specific insights into how the user experiences the task.

In the feasibility study using mental workload in the adaptive logic,
mental workload was assessed using electroencephalography [11]. Com-
pared to subjective measures, electroencephalography allows the moni-
toring of ACS in real time, without interrupting the participants. However,
the apparatus is cumbersome and subject to interferences. Subjective
measures cannot retrace small fluctuations in ACS, but these details are
not always necessary: objective measures, from physiological sensors or
electroencephalography, are averaged over a period of time. Yet, they can
still be sensitive to noise, and suffer from a high variability related to the
users or context of measurement, making them difficult to generalize [43].
Subjective measures provide high face validity [29], through the percep-
tion of participants without invasive or cumbersome devices. In general,
subjective measures allow to capture the holistic experience as felt by
participants [42], and are not affected by noise. Short questionnaires or
scales can be asked continuously during the VR task [28] to avoid recall
bias and break of immersion [38]. Additionally, no studies have been
conducted using subjective measures to assess ACS for adaptive VR.

In summary, while the benefits of adaptive VR training and the role
of cognitive states in user behavior have been identified in the literature,
studies investigating novel adaptive methods are sparse. We present
a general method that integrates cognitive states in the adaptive logic.
This involves the characterization of the user profile and performance
to provide a more nuanced representation of the user's actions. We then
provide an implementation of this model with the adaptation of difficulty
according to performance and mental workload, and we investigate the
impact of our approach on performance and experience.

## 3 ADAPTATION METHOD

Although the use of cognitive states in adaptive VR training has been
considered in the literature [11], this approach has only been mentioned
as the focus of a future user study and therefore only assessed regarding
its feasibility. Our general method modulates the adaptation of difficulty,
with consideration for the function of the VR application, and for the
cognitive states of the user that impact their performance and experience.
The general architecture and its main components are presented in Fig. 2
and are described in the following. While this paper focuses on the user's
cognitive states, ACS are commonly considered together. Affective
states also affect behavioral and physiological response and could be
similarly included in this method. For this reason, the following section
about the adaptation method features the more general term ACS.

### 3.1 General method of adaptation to integrate users' affective and cognitive states

Virtual reality applications are designed with a specific function, e.g.
training, rehabilitation, entertainment, etc. Their function shapes the
<figure><img src="image_4.png" /><figcaption>Fig. 2: Architecture of our general method to integrate ACS for the adaptation of the difficulty. Block 1 corresponds to the virtual environment, block 2 to the user profile model including performance and ACS of the user, and block 3 to the adaptive logic selecting the modifications of the virtual environment.</figcaption></figure>

design of VR applications and the content of the task performed by the users. Among the adaptation variables identified by Kelley [20], this method of difficulty adaptation focuses on the dynamic elements of the virtual environment that evolve in real time. The user interacts with or witnesses these elements during their immersion in the virtual environment. The adaptation of these variables is done regarding the measures gathered from the user's interactions and behavior. All these measures are combined to characterize the user's internal state, according to a user profile model (see element 2 of Fig. 2).

### 3.1.1 User profile model

Performance is the most common metric used in adaptive training [45]. In addition to this standard metric, the user profile is completed with the ACS. Affective states may have a positive or negative impact on people's decision-making process [26], while cognitive states are directly related to how people perceive or process information in general [3]. Our user profile model was built to integrate such aspects of the user's inner states.

Affective and cognitive states can be deduced from objective and subjective measures of related metrics. Stress and fatigue, for example, are ACS that are correlated with heart rate variability [33] or eye movement [2]. These metrics can be measured in real time, providing an estimation of the state of the user. However, some ACS cannot be measured with physiological data, and the same physiological data can be used for measuring different ACS. Subjective measures, in the form of questionnaires, offer a more direct approach to the estimation of ACS. Questionnaires and subjective scales allow one to interrogate the user on a specific aspect but provide punctual data. Obtaining these data over a period of time requires asking, and therefore interrupting, the user performing the task. These interruptions can affect the ACS of the user, and disrupt their progression within the task.

It is important to note that both objective and subjective measures are relative. Each individual has a different physiology and perception of their ACS, therefore, our user model includes the initial state of the user. The initial state features the calibration of sensors for objective measures but also a starting point, at rest, for the ACS measures. This baseline is essential for post hoc analysis of the data and is an important aspect to take into account during the task. A user would act differently if they feel stressed, overworked, or exhausted at the beginning of the task.

With the user profile model, we aimed to provide a more complete representation of the user. It complements the information on performance brought by conventional measures. In this approach, the user is represented between the virtual environment, with which they interact, and the user profile that they feed with initial and real-time data (see transition 1 to 2 of Fig. 2). From the user profile, the adaptive logic interprets this data (see transition 2 to 3 of Fig. 2), along with other factors, to modify, i.e., adapt, the virtual environment accordingly.

### 3.1.2 Adaptive logic

The measures collected in the user profile are used as the input to the adaptive logic. The data is interpreted using thresholds adjusted to the nature of the data, to identify the aspects that are the most distant from standard values. This interpretation targets where the adaptive logic needs to intervene, compared to the baseline set by the initial state of the user profile.

Additionally, the objectives of the adaptive logic are motivated and determined by the function of the VR application which itself is defined by the VR application's design. For a training application, the function is usually to improve performance, while for a rehabilitation application, the function could be to target the actions most difficult for users [46] but also to relax or reassure the user regarding their abilities [27].

The function, along with the initial state of the user profile, defines the parameters and level of intervention of the adaptive logic. These parameters constrain and reorder the list of priorities from the data interpretation, to align the highest priorities with the objectives of the application. For example, assuming the desired function is to relax the user, a very low performance on a given task would not have a higher priority over stress level in terms of adaptation. The list of priorities, weighted by the parameters of the adaptive logic, is used to select an appropriate modification for the virtual environment, given multiple input variables.

### 3.1.3 Guide and disturbance

The modification chosen by the adaptive logic is selected from a pool of potential adaptations, e.g., changes in the complexity of the task, visual or auditory noises, or feedback. This set of adaptations is organized depending on the effect (positive or negative) on the various aspect of the user's states. We called the adaptations with a negative effect "disturbances" and the adaptations with a positive effect "guides". The nature of these guides and disturbances affects the ACS of the user.

Disturbances and guides are grouped depending on the effect they have on the user's state and performance, and therefore on difficulty. They can be characterized by their impact on the three types of loads (intrinsic, extraneous, germane) from the Cognitive Load Theory [40]. They are then ordered according to the magnitude of their impact and triggered accordingly to the appropriate intensity and target. With a history of the modifications, the adaptive logic can recall which adaptations have been applied, to avoid repetition of similar elements.

With the fine characterization of each modification, e.g., based on their impact, intensity or scope, the adaptive logic adapts the virtual environment in real time, with a consideration for the user's performance and affective and cognitive state. This method nuances the adaptation to target a specific function of the VR application and modify the virtual environment accordingly.
## 3.2 Implementation: adaptation of the difficulty to performance and mental workload

This method has been implemented with a VR task which difficulty is adapted based on performance, with the success or failure in the task, and cognitive state, with mental workload. The monitoring of mental workload gives insight of the cognitive resources available for the user to perform the task. This implementation focuses on cognitive states to limit the number of variables assessed in the user study presented in Sec. 4. The task presented in this section is a cognitive task.

### 3.2.1 Cognitive task

The cognitive task used in this implementation is a sorting task stimulating the processing speed of the user, designed by an expert neuro-psychologist.

The task takes place in an office environment, with various files and receipts to be sorted into different boxes, according to a set of criteria. The complexity of the criteria increases with the difficulty. Documents must be sorted according to:

* An explicit information to read (e.g., an integer, a modality of payment);

* An information to translate or interpret (e.g., the month of a date to translate from number to the name of the month, the category of a purchase from the name of the product);

* A specific information among similar data (e.g., the price of an item among price adding taxes or reductions);

* An information to deduce from other data (e.g., an adherent's diet based on preference and allergies);

* Simple to complex arithmetic calculus (e.g., addition, subtraction, multiplication or/and division of numbers).

Additionally, the users are given a limited time to complete the task. The office is filled with ambient noise, e.g., co-workers chatting and roadworks, in order to provide an ecological environment for the task. The user profile records an objective performance score, based on the number of documents correctly sorted and the mental workload assessed through frequent subjective measures with a virtual slider discretized into a 10-point scale, as seen in Fig. 1-B. Maintaining good results with high processing speed task requires a high level of concentration: mental workload, being predictive of performance [43], would then have a large impact on the user's outcome and experience, justifying the choice of measured ACS. The initial state is the initial mental workload level of the user at the beginning of the task. In this implementation, the function of the VR application is to train the users for the cognitive task while maintaining a low mental workload. It sets the following objectives:

* To challenge users with low mental demands, and avoid spending time on difficulty levels that are too easy, and therefore irrelevant.

* To prevent the user from being overwhelmed: pressuring the users to do too demanding cognitive tasks would drain their mental resources.

* To check that the users mastered one variation of instructions before moving on to the next: users cannot reach difficult levels without succeeding in the most complex levels of less difficult variations.

The variations of difficulty for the sub-task, evolving in regards to guides or disturbances, are based on the following elements:

* The number of boxes to sort the documents;

* The quantity of text on each document, with a variable amount of unnecessary information;

* The color logic for the objects, from congruent with the sorting criterion to random;

* The volume of ambient noise.

A single trial is a sequence of five documents to sort one by one in a limited time, according to a given sorting criterion. After each trial, the difficulty of the task is adapted and another trial starts. The sorting criterion is explained in the task instructions and users are

<figure><img src="image_5.png" /><figcaption>Fig. 3: Virtual environment used in our implementation and user study, from a third-person point of view. The cognitive task takes place in a virtual office, with document to sort in a range of boxes within a limited time. The task instructions are displayed on a monitor screen on the right.</figcaption></figure>

given a certain amount of time to read the instructions and sort each document. This time is shown on an analog timer. The duration of the instructions reading timer for each trial is based on a high reading speed for processing and understanding a text, i.e., five words per second, and the number of words in the instructions. The documents have to be sorted in boxes labeled accordingly with the sorting criterion. The labels appear at the same time as the first document to sort. If the document is sorted correctly, it disappears and the next document appears. If the document is misplaced, or the timer runs out of time, a tone warned the user of the mistake, then the document disappears and the next one appears. Figure 1-A shows an example of the sorting task.

### 3.2.2 Adaptive logic

The inputs of the adaptive logic are performance and mental workload. The performance score for one trial is rated between -5, i.e., -1 for each document incorrectly sorted, and +5, i.e., +1 for each document correctly sorted. The timer running out of time results in a -0.25 score. This scoring highlights the importance of sorting as many documents correctly as possible in each trial. The adaptive logic penalizes mistakes more than lack of speed and does not encourage random sorting. The mental workload score is the user's answer to the question "What level of mental workload do you feel?" (written in users' native language) on a 10-point scale from "Very low" to "Very high".

The adaptive logic presented in this paper changes the task difficulty from -1 (i.e., a decrease in the difficulty of one step) to +3 (i.e., an increase in the difficulty of three steps). This asymmetrical evolution comes from the observation that, with a fine level of detail between each difficulty level, repetition of a task with the same difficulty (i.e., an increase of +0) is not truly neutral, because of the learning effect taking place during the first occurrence. Additionally, because each sub-task is composed of four variations with the same instruction, a modifier of +3 ensures that the users could not jump from the first level, i.e., the easiest level, of one sub-task to the first level of the next one.

The adaptive logic takes into account the input measures (i.e., performance and mental workload) as well as the current difficulty level. The function set for the adaptive logic translates in computational rules that set the impact of mental workload for the following trial, given the performance.

Even if the performance is sufficient to increase the difficulty, a high mental workload score would slow down progression, up to repeating the same variation before reaching a new sub-task. It gives the user additional time to master the current sub-task before discovering a new one. With a low mental workload, a moderate to good performance would result in quicker progression and spares the user the effort of doing exercises that are not challenging. Conversely, if the performance is low, a low mental workload would tend to avoid reducing the difficulty. The user would repeat the same variation of a sub-task, in order to train. With a high mental workload, a low performance would lead to reduce the difficulty as the exercise is too challenging. The computational rules for the adaptive logic are represented in Fig. 4-A. According to these
<figure><img src="image_6.png" /><figcaption>Fig. 4: Rules integrating mental workload and fitting the function. The cells represent a difficulty level, numbered from 1 to 4 for each sub-task. The arrows between cells indicate a potential step between the current and next difficulties. The groups of cells are organized depending on mental workload and performance difficulty score. The rules not displayed are inferred from the adjacent rules. Sub-figures show a) the rules for the adaptation based on performance and mental workload, and b) the rules for the adaptation based on performance only.</figcaption></figure>

rules, the performance and mental workload scores are then translated into adaptation modifiers. For each adaptation step, the performance and mental workload are measured and weighted by a set of coefficients reflecting these rules, to compute an adaptation modifier between -1 and +3. These asymmetrical rules can lead users with moderate success to repeat the same task multiple times, as they are expected to master the current task before progressing to higher levels.

In comparison, simple adaptation (see Fig. 4-B) based on performance only would directly map the performance score onto a difficulty modifier, and change the difficulty accordingly. This results in the difficulty increasing when the user performs well, and staying even or decreasing when the user does not.

# 4 USER STUDY

The implementation described in Sec. 3 presents one approach to integrate ACS in an adaptive VR training application. We assess this approach in a user experiment to evaluate participants’ performance and experience regarding their mental workload. In this study, participants have to perform the task introduced in Sec. 3.2.1, as accurately as possible. The task difficulty is adapted by either: 1) mental workload and performance; or 2) performance only. This procedure was evaluated and accepted by the institutional ethical committee of Inria, France.

## 4.1 Fine-tuning of the task parameters

The design of the protocol of this user study required a process of fine-tuning to adjust the difficulty between various levels of the cognitive task. We performed a pilot study with the objective of tuning the task and adaptation parameters. We divided this pilot study into two steps. First, we asked the participants to rate the difficulty of all levels on a scale from 1 to 100, and then to identify what elements of the levels contributed the most to the difficulty. Finally, we converged to the final version of the protocol by performing moderate adjustments from participants’ feedback to balance the adaptive logic.

For the first step, five participants were included and tried a shorter version of all the difficulty levels. Their feedback allowed us to reorder the difficulty levels to have a smoother difficulty curve and to identify the elements in the instructions that lacked clarity. On a scale of 1 to 100 (1 being very easy and 100 being very hard), the hardest levels were graded at 33. Twelve more difficulty levels were added, with three new sorting criteria, each instantiated in four different variations. These levels tended to feature mental arithmetic, as it was identified from the pilot participant’s

answers as the most difficult type of cognitive task. We made the calculations more complex, by gradually adding subtraction, multiplication, and a combination of both. Time pressure was also added: each object had to be sorted in a limited time, based on the average time taken by the first five pilot participants. Finally, the 29th out of 68 difficulty level was identified as the first level that started to be challenging for a majority of participants. It was then selected as the starting level of each condition.

After revising the difficulty levels, 10 additional participants took part in the pilot during the second step. These participants performed 20 trials, with 5 documents to sort in each task, either with or without our adaptive logic. We collected their feedback on the modifications done after the first step, and we cross-checked their perception of the difficulty with their performance collected during the second step. This step led to the addition of 12 more difficulty levels, with 3 new sorting criteria adding mental arithmetic with division. During this step, the parameters of performance-based and mental workload-based adaptation were also tuned to meet the objectives.

## 4.2 Participants

A total of 30 participants (13 women, 13 men, and 4 others) with ages between 21 and 53 years (30.3±8.0, mean±SD) volunteered for this study. These participants did not take part in the fine-tuning process, and were therefore new to the experiment. The majority were either occasional or novice VR users (25 out of 30) and played video games either occasionally or frequently (18 out of 30). Due to the mental arithmetic in part of the task, we report the participants educational background and rating of their own arithmetic skills. When rating their arithmetic skills between bad and good, 8 participants rated it as rather bad, 6 as neutral, 14 as rather good, and 2 as good. Most of the participants, 18 out of 30, had a Master’s degree or equivalent, 6 had a PhD, 5 had a Bachelor’s degree or equivalent, and 1 had a technical degree or equivalent. Although most participants had a college education, participants were recruited to ensure variety in their educational backgrounds and expertise.

## 4.3 Virtual task and apparatus

For this user study, the participants were equipped with a Valve Index head-mounted display and controllers. The virtual task used for this study is described in Sec. 3.2. The interactive elements and the scenario were implemented with Unity 2022.3, using Xareus<fnref n="1" /> [14].

<footnote n="1">¹https://xareus.insa-rennes.fr/</footnote>
## 4.4 Procedure

At the beginning of the experiment, procedure was explained to the participants who then signed a consent form if they agreed to proceed. For the duration of the experiment, participants were sat on a chair and equipped with a head-mounted display. Before the task started, they could select a virtual avatar among a list of 10 models, with variation of gender, skin color and clothing.

The experiment was composed of two conditions, both completed by the participants (within-subjects study), with the order of the conditions randomly assigned resulting in two evenly-sized groups. The control condition had the difficulty varying based on a performance score, i.e., the number of documents sorted correctly, incorrectly or not sorted in time. The experimental condition had difficulty varying based on performance and mental workload, as a subjective score measured between each sorting task with a 10-point scale. To avoid inducing a bias, mental workload was assessed during both conditions, but only used in the experimental condition. Additionally, to keep the experimental variables secret from the participants, we asked them to also complete a 10-point scale regarding their mental fatigue, which was not used in the adaptation. For the remainder of this paper, the control condition will be referred to as condition “Perf” for performance-based adaptation, while the experimental condition will be referred to as condition “Perf+MW” for performance and mental workload-based adaptation.

The participants were shown a tutorial on the sorting task, composed of typical documents they could put in boxes without any time limit. When they felt ready, they were asked to answer about their current mental workload and fatigue, on scales from 1 to 10.

The remainder of the experiment consisted of a sequence of trials each trial starting with a manual validation of the participant, then the instructions for the tasks appearing on a monitor and the participants given time to read them using a timer. Once the first timer ran out, a document appeared, starting a new timer of seven seconds to let the participant sort it. After the participants put the document in a box, or after the timer ran out, a new document appeared, repeating this step for a total of five repetitions. After the fifth repetition, participants were asked again about their mental workload and fatigue. The participants had to complete 20 sorting tasks as described above for each condition, with an adaptation of the difficulty depending on the condition.

After each conditions, participants were asked to answer a set of questionnaires, described in detail in Sec. 4.6. Additionally, they could share their comments at the end of the experiment. The experiment lasted around one hour, including 15 minutes in VR per condition, with a break between the two conditions.

## 4.5 Hypotheses

The adaptive logic for this study challenges the participants while preventing them from feeling overworked. Acknowledging mental workload pressuring a person helps to understand their behavior, especially when performing a cognitive task. We believe that taking into account mental workload to nuance the difficulty change, otherwise based on performance, would reduce the pressure and improve the experience of users, leading to the following hypotheses:

**H1:** The mental workload measures, during the condition and after, would be lower in the experimental condition ("Perf+MW").

**H2:** The performance measures would show no significant differences between the two conditions.

**H3:** The experimental condition ("Perf+MW") would be rated more successful, less stressful and preferred.

**H1** was motivated by the logic of decreasing or increasing with a slower pace the difficulty for participants with high mental workload. With this adaptive logic, participants' resources would be less solicited, delaying the moment when the task requirement would become too high. **H2** was motivated by Dey et al.'s feasibility study highlighting that participants' performance was not affected by the inclusion of mental workload in their adaptive logic [11]. Finally, **H3** followed the previous hypotheses. Assuming a lower mental workload, we expected condition "Perf+MW" to be perceived as more successful, less stressful and preferred.

## 4.6 Measures and statistical analysis

After each condition, the participants answered a questionnaire containing self-assessed mental workload, the NASA Task Load Index (NASA-TLX) evaluation [16], the Simulation Sickness Questionnaire (SSQ) [21], and questions about their perception of what was the input of the adaptive logic, with independent questions about performance, mental workload, and mental fatigue. Additionally, online measures collected during the experiment (performance, mental workload and mental fatigue) were used in the post hoc analysis.

After both conditions, the participants answered a questionnaire in which they selected the condition they felt most successful in, the condition they felt most stressed in, and their preferred condition. For each comparison, they provided an interval of confidence for their answers. Finally, participants answered the iGroup Presence questionnaire (IPQ) [37].

Additionally, the difficulty, performance (i.e., the number of failures and successes), and the online measures regarding mental workload and fatigue during each level were recorded.

The data was compared between each condition using a paired *t*-test, after normality verification using a Shapiro test. The effect size was computed using Cohen's *d* measure. When the data was not normally distributed, we performed a Wilcoxon signed-rank test. For the answers comparing the two conditions, data was evaluated using a binomial test to assess if the participants' answers were randomly distributed.

# 5 RESULTS

The following results are reported with *p* the p-value, *d* the effect size computed from Cohen's *d*, and *m*±*std* for the mean value and standard deviation.

## 5.1 Cognitive states measures

The distribution of NASA-TLX raw score and sub-items (mental, physical, and temporal demand, performance, effort, and frustration) between 1 and 10 for each condition are presented in Fig. 5, with a description of the statistical test used for the analysis.

A *t*-test on the score difference with respect to the condition order showed an order effect (*p* < .001) between the two groups. We then conducted an ANOVA on the NASA-TLX raw score, showing an effect of both the condition (*F* = 6.146, *p* < .05) and the order (*F* = 5.146, *p* < .05) on the score. Overall, a *t*-test on the raw NASA-TLX score resulted in significantly higher score for condition “Perf” (*p* < .001, *d* = 0.705, *m*<sub>Perf</sub> = 5.78±0.88, *m*<sub>Perf+MW</sub> = 5.26±0.81), indicating a lower perceived mental workload after condition “Perf+MW”.

Physical and mental demand were not normally distributed. For physical demand, the data were not normally distributed, so it was analyzed using a Wilcoxon signed-rank test. No significance was found between the conditions for physical demand (*p* = .083, *m*<sub>Perf</sub> = 2.83±1.68, *m*<sub>Perf+MW</sub> = 2.50±1.38), indicating no statistical difference between the two conditions for perceived physical demand. Concerning mental demand, the distribution was slightly skewed. We applied the logarithmic function to the data, which resulted in a normal distribution of the logarithm of mental demand. The results of a paired *t*-test on the logarithmic mental demand led to the same conclusion as a paired *t*-test performed on mental demand. The results presented here are computed from the corrected data. These results highlight significantly higher mental demand for “Perf” (*p* < .001, *d* = 0.800, *m*<sub>Perf</sub> = 2.00 ± 0.20, *m*<sub>Perf+MW</sub> = 1.77 ± 0.28), indicating a lower perceived mental demand after condition “Perf+MW”.

Similar to mental demand, the other NASA-TLX sub-items were significantly higher for “Perf” via a paired *t*-test, with temporal demand (*p* < .01, *d* = 0.644, *m*<sub>Perf</sub> = 7.70±1.74, *m*<sub>Perf+MW</sub> = 6.50±1.53), effort (*p* < .05, *d* = 0.407, *m*<sub>Perf</sub> = 6.53±1.31, *m*<sub>Perf+MW</sub> = 5.93±1.28), and frustration (*p* < .001, *d* = 0.808, *m*<sub>Perf</sub> = 6.03 ± 2.09, *m*<sub>Perf+MW</sub> = 4.57 ± 1.87), indicating lower scores for perceived temporal demand, effort and frustration after condition “Perf+MW”. Additionally, the in-house question regarding self-assessed mental workload was significantly higher for “Perf” (*p* < .001, *d* = 0.933, *m*<sub>Perf</sub> = 6.80 ± 2.04, *m*<sub>Perf+MW</sub> = 5.50 ± 1.57), indicating a lower self-assessed mental workload after condition “Perf+MW”.
<figure><img src="image_7.png" /><figcaption>Fig. 5: Distributions of participants' answers, between 1 and 10, for the NASA-Task Load Index items (mental demand, physical demand, temporal demand, performance, effort, and frustration). Asterisks indicate post hoc significance level.</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>Fig. 6: Distributions of average results to online measures of performance from the sorting task results, between -5 (complete failure) and +5 (complete success), mental workload, and mental fatigue from self-assessment on a 10-point scale. Asterisks indicate post hoc significance level.</figcaption></figure>

The results from online mental workload were symmetrically distributed. The difference between mean online mental workload of each condition followed a normal distribution. A t-test on the score difference with respect to the condition order showed an order effect ($p < .001$) between the two groups. We then conducted an ANOVA on the mean online workload, showing an effect of the order ($F = 6.554$, $p < .05$) on the score. Overall, between the two conditions, the mean value of mental workload was found significantly higher during for “Perf” ($p < .01$, $d = 0.626$, $m_{Perf} = 5.01 \pm 1.76$, $m_{Perf+MW} = 4.52 \pm 1.41$), with mean values of mental workload reported during the task being 10.7% higher than during “Perf+MW”.

The mean online mental fatigue monitored during the experiment did not show an order effect with a t-test regarding the score difference ($p = .33$), and it did not significantly differ between the two conditions with a Wilcoxon signed-rank test ($p = .529$, $m_{Perf} = 4.35 \pm 1.86$, $m_{Perf+MW} = 4.06 \pm 1.73$), indicating similar mental fatigue during the two conditions. Distribution and test results for online measures are shown in Fig. 6.

## 5.2 Performance measures

Within the NASA-TLX evaluation, performance was rated significantly higher for “Perf+MW” ($p < .01$, $d = 0.955$, $m_{Perf} = 4.10 \pm 1.88$, $m_{Perf+MW} = 6.00 \pm 1.78$), indicating a lower perceived performance after condition “Perf”. The performance data monitored online was overall skewed, due to the first difficulty levels being easier. The Wilcoxon signed-rank test resulted in a significantly better performance

<figure><img src="image_9.png" /><figcaption>Fig. 7: Distribution of participants' answers regarding the conditions they felt most successful in, most stressful, and preferred. Each comparison is weighted with the participants' confidence interval in their answers, between 1 and 5. The color of the dots represents the condition chosen by the participants, and the shape of the dots represents the order in which each participant performed the two conditions.</figcaption></figure>

during condition “Perf+MW” ($p < .001$, $d = 0.685$, $m_{Perf} = 2.54 \pm 0.60$, $m_{Perf+MW} = 3.12 \pm 0.97$), with mean values of performance recorded during the task being 22.8% higher than during condition “Perf”, as seen on Fig. 6.

The highest level reached by the participants ranged between 52 and 68 for condition “Perf” ($m = 65.23 \pm 4.00$). For condition “Perf+MW”, the highest level reached ranged between 37 and 68 ($m = 58.13 \pm 9.13$). Regarding the progression of the difficulty, all of the participants reached at least one point where the increase of the difficulty was reduced due to lesser performance or higher mental workload, in both conditions. This change of pace in the difficulty evolution did not happen at the same time for the participants, and sometimes occurred multiple times during the experiment. When comparing the number of trials needed to reach half of the total difficulty levels (corresponding to level 50), it was significantly higher for condition “Perf+MW” ($p < .001$, $d = 0.734$, $m_{Perf} = 9.08 \pm 1.35$, $m_{Perf+MW} = 11.91 \pm 2.73$). This indicates a slower progression for participants during condition “Perf+MW” than during “Perf”.

During condition “Perf”, 17 out of 30 participants reached a plateau (i.e., the repetition of the same difficulty level, with a deviation of 1, during 5 or more steps). Among them, 7 stopped their progression because they reached maximum difficulty and 1 progressed again after the plateau. During condition “Perf+MW”, 6 out of 30 participants reached a plateau, with 1 of them because they reached maximum difficulty and 5 progressing again after the plateau.

## 5.3 Preference and perception of conditions

When asked about the conditions they felt most successful in, 25 out of 30 participants selected condition “Perf+MW”, with an average confidence value of $3.70 \pm 1.09$ (confidence values are between 1 and 5). The 5 other participants did condition “Perf+MW” before condition “Perf”. The condition they felt most stressful was condition “Perf”, for 24 out of 30 participants, with a confidence of $3.90 \pm 0.990$. The 6 other participants did condition “Perf+MW” before condition “Perf”. The condition they mostly preferred was condition “Perf+MW”, for 23 out of 30 participants, with a confidence of $3.97 \pm 0.89$. Among the 7 other participants, 5 of them started with condition “Perf+MW”, and 2 of them started with “Perf”. The detail of their answers, indicating which conditions they performed first, is described in Fig. 7.

The distribution of their answers was tested with a binomial test and could not be attributed to chance, i.e., the probability of success was not 0.5. For each comparison, the *p-value* is respectively $p_{Success} < .001$; $p_{Stress} < .01$; and $p_{Preference} < .01$.

The distribution was also tested with a binomial test for each group to see the influence of the order of conditions. Results of the group that started with condition “Perf” could not be attributed to chance ($p_{Success} < .001$; $p_{Stress} < .001$; and $p_{Preference} < .01$). Results from the group that started with condition “Perf+MW” do not show significant *p-values* ($p_{Success} = .302$; $p_{Stress} = .607$; and $p_{Preference} = .302$).

After each condition, the participants were asked about the influence
of their behavior on the adaptation. No significant differences were found in their answers regarding the weight of their performance (p = 1.00), their mental workload (p = .632), or their mental fatigue (p = .539). The average value between 1: "No, not at all" and 7: "Yes, completely" attributed to the part played by each measure in the adaptation was: 3.40±1.82 for performance, 2.77±1.72 for mental workload, and 2.70±1.74 for mental fatigue.

## 5.4 Simulation sickness and presence

The SSQ was analysed through the Total Severity (TS) score computed from a weighted sum of all the items. The TS value was then compared to the mean reference value of 9.80 ± 15.0. The SSQ was assessed before the first condition, at t₀, after the first condition, at t₁, and after the second condition, at t₂. Using an ANOVA, no significant differences were found between the participants' TS scores at t₀, t₁, and t₂ (p = .79, $m_{t0} = 6.61 \pm 5.97$, $m_{t1} = 9.75 \pm 9.44$, $m_{t2} = 10.38 \pm 9.02$), indicating similar levels of simulation sickness between each step of the experiment.

The answers of the IPQ were compiled in four sub-scores: experienced realism, involvement, presence, and spatial presence, and ranked based on literature's acceptability grades. For the experienced realism, 24 out of 30 participants were above the acceptable threshold, with 14 reaching the "excellent" grade. Regarding the involvement, 22 participants were above the acceptable threshold, including 9 with the "excellent" grade. All the participants were above the satisfactory threshold for the presence and 24 for the spatial presence. 29 participants achieved the "excellent" grade for presence and 9 for spatial presence.

# 6 DISCUSSION

This paper introduces a general method for adaptive VR integrating user ACS, evaluated in a user study. The user study assesses the impact of performance and mental workload-based adaptation on participant behavior and experience.

## 6.1 User study analysis

The results of the statistical analysis show both the significant effect of our experimental condition along with an effect of the order in which the conditions were experienced. This order effect was expected since the task presented for this user study is a cognitive task inducing a learning effect. The participants would benefit from their acquired knowledge during the second condition, which could lessen its perceived difficulty and stress. The order effect had a significant impact on NASA-TLX score, online mental workload, performance, and perception of the most successful, stressful or preferred condition. Nevertheless, it does not entirely counterbalance the impact of the condition between the two groups, resulting in significant differences in most measured outcomes.

The mental workload of participants, measured through the NASA-Task Load Index, additional in-house questions, and the data monitored during the experiment, showed a reduction in the raw NASA-TLX score, mental demand, temporal demand, performance, effort, and frustration, for the experimental condition. Online measures also show a significant improvement in performance, and a decrease in perceived mental workload. The effect size of the questionnaire items revealed which aspects were more impacted by the increased difficulty, with a large effect size for NASA-TLX's mental demand, performance, frustration, and online measure of mental workload. Frustration in particular was often mentioned within the questionnaire comments or during the informal discussion after the experiment, as one of the measures the most affected by the conditions. In these exchanges, the impact of condition "Perf+MW" on mental workload was highlighted, with multiple participants expressing a surprising feeling of mental exhaustion after condition "Perf". This effect was more prominent for participants starting with condition "Perf+MW", even though it was reported by participants who started with a different condition. Only physical demand was not significantly different, which was expected given the cognitive task requiring only simple physical manipulation. Overall, these results support our first hypothesis H1 with both the NASA-TLX raw score and relevant individual sub-items.

Performance as the NASA-TLX sub-item assessed after each condition, and monitored online during each level, was significantly better in condition "Perf+MW", with respectively a moderate and large effect

size. Our second hypothesis H2 is not supported by these results, as we expected from Dey et al.'s study no impact on the performance [11]. The difference in our findings is nuanced by the order effect and the difficulty progression between the two conditions. The progression slope was larger for condition "Perf", resulting in more challenging levels for this condition. These elements could partially explain the improvement of performance in condition "Perf+MW". Additionally, the participants reaching a plateau were more abundant in condition "Perf", where it often meant a stop in their progression. However, most participants who reached a plateau in condition "Perf+MW" then increased the difficulty again, after either reporting a lower mental workload, showing a significant increase of performance, or both. While this phenomenon might not be visible on condition "Perf" due to the short number of trials, it could indicate that our experimental approach is more fitted to the participants' natural progression.

Finally, for most participants, condition "Perf+MW" was the condition they felt most successful in, less stressful, and preferred compared to condition "Perf". All the participants who found themselves more successful in condition "Perf" (N=5) and felt this condition less stressful (N=6) started with condition "Perf+MW". Understanding and integrating unfamiliar instructions could have played a part in their experience, and could be explained by the order effect. Regarding the two participants who preferred condition "Perf", their first condition, their comments about the experiment highlighted that they were familiar with competitive video games and tended to address the cognitive task as a similar challenge. One reported a higher frustration score in condition "Perf+MW", their second condition, which came from the perception that the levels were not as hard as in the first condition. They felt like they could not outperform their first performance. For participants with this personality profile, preferences went toward the most stressful and challenging condition. In contrast, 4 participants explicitly stated that they were comforted by the familiarity of the levels' instructions, which led them to choose their second condition as their preferred, regardless of the adaptation strategy. This highlighted the impact of participants' profiles on their experience of challenging cognitive tasks, and support our third hypothesis H3.

Although the participants' results showed a significant statistical impact of the adaptation strategy on mental workload and performance, they could not perceive the actors of the adaptation. The integration of mental workload of participants was not only beneficial regarding participants' experience and performance but also that this method was imperceptible to them. Such characteristics would allow to adapt a training or rehabilitation VR application with minimal risks of users learning the functioning of the application, leading to biased results concerning their performance.

To supplement our assessment of the participants' experience regarding their cognitive state, the SSQ and IPQ provided metrics and reference scores. Overall, the integration of mental workload in the adaptation did not have a significant effect but confirmed that the experiment was immersive and not detrimental to the participants' well-being.

Repeated measures requiring the participants' input through the experiment (i.e., the subjective scales on mental workload and fatigue) did not seem to damage the experience. It could have been expected from the participants to report a tiredness or boredom from answering two scales about their mental state, approximately every 2 minutes. These interruptions could have broken the flow of the task and reduced the feeling of presence in the virtual environment. Yet, considering the already frustrating nature of the task, our results suggest that it led to limited frustration and overall enjoyable experience. Repeated measures were similarly used in Luong et al.'s study [28], with one scale to answer every 27 seconds. The scales of the repeated measures were displayed on the same virtual monitor as the task instructions, fitting fully in the task context. The participants could not feel a strong disconnection from the current task. The interruptions created by the repeated measures were more likely similar to short breaks from the intense cognitive exercise, and created a clear yet natural separation between two trials where the task instructions could change. The use of such repeated measures to monitor ACS is both valid and feasible.
## 6.2 Implications for the general method

The user study we conducted assesses an implementation of our general method. Our results give important insights about the general method itself. The integration of ACS as adaptive logic input allows to take into account a more nuanced representation of the user. Training and rehabilitation studies that include such affective or cognitive states are still uncommon, but could improve the user experience and performance.

The data collected to feed the user profile can be based on objective measures as well as subjective measures. Objective measures might require cumbersome or intrusive devices, while subjective measures might be less accurate. Yet, in the user study described in this paper, both modalities, with performance measured objectively, and mental workload measured subjectively, were used to gather data in real time, or in an approximation of real time. It shows the viability of both approaches regarding data collection. The type of measures required can vary depending on context, especially for rehabilitation studies: these studies might include participants who cannot use specific types of sensors or reflect on their perceived states. Additionally, the initial state integrated into the user profile allows to weight of the input data according to the user's state prior to the VR experience. It introduces a light history to the user profile without integrating a great amount of data, for a light modeling of the user.

The characterization of performance and ACS feeding the adaptive logic allows to take into account a better representation of the user, and the integration of the function provides better contextual awareness in adaptive VR training applications. The implementation provided in this paper aims to challenge users, while ensuring that users master the exercises and preventing overwork. The user study participants' results and feedback highlights that these objectives were well conveyed in the VR task. While the task remained relatively easy, the time pressure ensured that the participants stayed focused. All of the participants reached one point or more where their progression stalled, when reaching more challenging levels. With these objectives, the function in our method allows to fully integrate the context of use in the adaptation method, while keeping the adaptive logic independent. Finally, the adaptive logic presented here aims to be highly modular, with the characterization of each modification of the virtual environment based on their effect on user's ACS and the virtual environment, making them easy to configure and replace. It allows the method to evolve with future progress regarding our understanding of the role of ACS in training or rehabilitation contexts, with respect to the real-time computation required in VR.

## 6.3 Limitations and perspectives

The main limitation of this work is related to the order effect regarding the conditions in the user study. The participants performed both conditions and could learn from their first experience of the experiment, despite the counterbalancing of condition order. The statistical analysis shows an impact of this order effect on most of the significant results, reducing their validity. While the results for the adaptive strategy assessed in this paper are promising, their generalization cannot be ensured without additional work.

The choice of adaptation input was kept rather simple. Performance was simply defined as a score computed from success and failure, to limit the complexity of the study design and so it could be representative of literature's adaptive strategy, i.e., our control condition. Other indicators of performance (e.g., progression, highest difficulty level) were used for post hoc analysis. We also focused on only one aspect of ACS, with mental workload as a cognitive state. The use of a more complete model of the user performance and ACS would bring more insight on the opportunities and limits of our method.

## 7 CONCLUSION

This paper introduces a method to integrate users' affective and cognitive states in the adaptation of a virtual reality training applications. We present an implementation of this method with the adaptation of difficulty to the user's performance and mental workload, assessed in a user study. The results show that our experimental condition resulted in significantly lower mental workload and higher performance. Overall, the integration of mental workload in addition to performance in the adaptive logic was preferred by the participants over the control condition.

These findings are encouraging results for the adaptation in virtual reality with regards to affective and cognitive states, beyond the scope of this paper. Users of adaptive virtual reality training applications that integrate their affective and cognitive state might feel less overworked and perform better in tasks that challenge their cognitive processing.

## ACKNOWLEDGMENTS

The authors wish to thank Alexandre Audinot, Florian Nouviale, and Adrien Reuzeau for their technical assistance with Xareus during the development of the VR application.

## REFERENCES

[1] H. B. Abdessalem, M. Boukadida, and C. Frasson. Virtual reality game adaptation using neurofeedback. In *FLAIRS 18*, pp. 276-281. AAAI Press, Florida, USA, 2018. 1, 2

[2] S. H. Al-Gburi, K. A. Al-Sammak, I. Marghescu, and C. C. Oprea. State of the Art in Drivers' Attention Monitoring – A Systematic Literature Review. *Karbala International Journal of Modern Science*, 9(1):14-25, Jan. 2023. doi: 10.33640/2405-609X.3278 3

[3] K. A. Bernhardt, D. Poltavski, T. Petros, F. R. Ferraro, T. Jorgenson, C. Carlson, P. Drechsel, and C. Iseminger. The effects of dynamic workload and experience on commercially available EEG cognitive state metrics in a high-fidelity air traffic control environment. *Applied Ergonomics*, 77:83-91, May 2019. doi: 10.1016/j.apergo.2019.01.008 1, 3

[4] D. Bian, J. Wade, Z. Warren, and N. Sarkar. Online Engagement Detection and Task Adaptation in a Virtual Reality Based Driving Simulator for Autism Intervention. In M. Antona and C. Stephanidis, eds., *Universal Access in Human-Computer Interaction. Users and Context Diversity*, pp. 538-547. Springer International Publishing, Cham, 2016. doi: 10.1007/978-3-319-40238-3_51 2

[5] M. M. Bradley and P. J. Lang. Measuring emotion: The self-assessment manikin and the semantic differential. *Journal of Behavior Therapy and Experimental Psychiatry*, 25(1):49-59, Mar. 1994. doi: 10.1016/0005-7916(94)90063-9 2

[6] C. L. F. Chan, E. K. Y. Ngai, P. K. H. Leung, and S. Wong. Effect of the adapted Virtual Reality cognitive training program among Chinese older adults with chronic schizophrenia: a pilot study. *International Journal of Geriatric Psychiatry*, 25(6):643-649, June 2010. doi: 10.1002/gps.2403 2

[7] M. Csikszentmihalyi and M. Csikzentmihaly. *Flow: The psychology of optimal experience*, vol. 1990. Harper & Row New York, 1990. 2

[8] A. Darzi and D. Novak. Automated affect classification and task difficulty adaptation in a competitive scenario based on physiological linkage: An exploratory study. *International Journal of Human-Computer Studies*, 153:102673, 2021. 1

[9] O. De Troyer, F. Kleinermann, and A. Ewais. Enhancing Virtual Reality Learning Environments with Adaptivity: Lessons Learned. In G. Leitner, M. Hitz, and A. Holzinger, eds., *HCI in Work and Learning, Life and Leisure*, pp. 244-265. Springer, Berlin, Heidelberg, 2010. doi: 10.1007/978-3-642-16607-5_16 1

[10] D. De Waard. *The measurement of drivers' mental workload*. PhD thesis, University of Groningen, Groningen, Netherlands, 1996. 2

[11] A. Dey, A. Chatburn, and M. Billingham. Exploration of an EEG-Based Cognitively Adaptive Training System in Virtual Reality. In *2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)*, pp. 220-226. IEEE, New York City, USA, Mar. 2019. ISSN: 2642-5254. doi: 10.1109/VR.2019.8797840 2, 6, 8

[12] M. Egger, M. Ley, and S. Hanke. Emotion Recognition from Physiological Signal Analysis: A Review. *Electronic Notes in Theoretical Computer Science*, 343:35-55, May 2019. doi: 10.1016/j.entcs.2019.04.009 2

[13] A. Graesser, S. D'Mello, P. Chipman, B. King, and B. McDaniel. Exploring relationships between affect and learning with autotutor. In *Proc Int Conf AIED*, 2007. 1

[14] L. Gramoli, F. Nouviale, A. Reuzeau, A. Audinot, M. Risy, T. Marchand-Guerniou, M. Mavromatis, B. Arnaldi, and V. Gouranton. Xareus: a framework to create interactive applications without coding. In *2025 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)*, pp. 1658-1659. IEEE, 2025. 5

[15] F. Grimm, G. Naros, and A. Gharabaghi. Closed-loop task difficulty adaptation during virtual reality reach-to-grasp training assisted with an exoskeleton for stroke rehabilitation. *Frontiers in neuroscience*, 10:183574, 2016. 2

[16] S. G. Hart and L. E. Staveland. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. In P. A. Hancock
and N. Meshkati, eds., *Advances in Psychology*, vol. 52 of *Human Mental Workload*, pp. 139–183. North-Holland, Amsterdam, Netherlands, Jan. 1988. doi: 10.1016/S0166-4115(08)62386-9 2, 6

[17] A. Heloir, F. Nunnari, S. Haudegond, C. Havrez, Y. Lebrun, and C. Kolski. Design and Evaluation of a Self Adaptive Architecture for Upper-Limb Rehabilitation. In *H. M. Fardoun, V. M. R. Penichet, and D. M. Alghazzawi, eds., ICTs for Improving Patients Rehabilitation Research Techniques*, pp. 196–209. Springer, Berlin, Heidelberg, 2015. doi: 10.1007/978-3-662-48645-0_17 2

[18] M. Hoover and E. Winer. Designing Adaptive Extended Reality Training Systems Based on Expert Instructor Behaviors. *IEEE Access*, 9:138160-138173, 2021. doi: 10.1109/ACCESS.2021.3118105 1

[19] E. Hummel, M. Cogné, M. Lange, A. Lécuyer, F. Joly, and V. Gouranton. Vr for vocational and ecological rehabilitation of patients with cognitive impairment: A survey. *IEEE Transactions on Neural Systems and Rehabilitation Engineering*, 31:4167-4178, 2023. 1

[20] C. R. Kelley. What is adaptive training? *Human Factors*, 11(6):547–556, 1969. Place: US Publisher: Human Factors & Ergonomics Society. 1, 2, 3

[21] R. S. Kennedy, N. E. Lane, K. S. Berbaum, and M. G. Lilienthal. Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness. *The International Journal of Aviation Psychology*, 3(3):203–220, July 1993. Publisher: Taylor & Francis_eprint: https://doi.org/10.1207/s15327108ijap0303_3. doi: 10.1207/s15327108ijap0303_3 6

[22] Y. Khasawneh and M. Khasawneh. Cognitive load analysis of adaptive learning technologies in special education classrooms: A quantitative approach. *International Journal of Advanced and Applied Sciences*, 11(12):34–41, 2024. 2

[23] M. D. Kickmeier-Rust, N. Peirce, O. Conlan, D. Schwarz, D. Verpoorten, and D. Albert. Immersive digital games: the interfaces for next-generation e-learning? In *International Conference on Universal Access in Human-Computer Interaction*, pp. 647–656. Springer, 2007. 1

[24] Y. Lang, L. Wei, F. Xu, Y. Zhao, and L.-F. Yu. Synthesizing Personalized Training Programs for Improving Driving Habits via Virtual Reality. In *2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)*, pp. 297–304. IEEE, New York City, USA, Mar. 2018. doi: 10.1109/VR.2018.8448290 2

[25] J. S. Lemmens and C. F. von Münchhausen. Let the beat flow: How game difficulty in virtual reality affects flow. *Acta psychologica*, 232:103812, 2023. 2

[26] J. S. Lerner, Y. Li, P. Valdesolo, and K. S. Kassam. Emotion and Decision Making. *Annual Review of Psychology*, 66(Volume 66, 2015):799–823, Jan. 2015. Publisher: Annual Reviews. doi: 10.1146/annurev-psych-010213-115043 3

[27] L. F. Lui, U. Radhakrishnan, F. Chinello, and K. Koumaditis. Adaptive Immersive VR Training Based on Performance and Self-Efficacy. In *2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)*, pp. 25–29. IEEE, New York City, USA, Mar. 2023. doi: 10.1109/VRW58643.2023.00012 3

[28] T. Luong, F. Argelaguet, N. Martin, and A. Lecuyer. Introducing Mental Workload Assessment for the Design of Virtual Reality Training Scenarios. In *2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)*, pp. 662–671. IEEE, New York City, USA, Mar. 2020. ISSN: 2642-5254. doi: 10.1109/VR46266.2020.00089 2, 8

[29] T. Luong, A. Lecuyer, N. Martin, and F. Argelaguet. A survey on affective and cognitive vr. *IEEE transactions on visualization and computer graphics*, 28(12):5154–5171, 2021. 2

[30] T. Luong, A. Lecuyer, N. Martin, and F. Argelaguet. A Survey on Affective and Cognitive VR. *IEEE Transactions on Visualization and Computer Graphics*, 28(12):5154–5171, Dec. 2022. doi: 10.1109/TVCG.2021.3110459 1

[31] A. Mariani, E. Pellegrini, N. Enayati, P. Kazanzides, M. Vidotto, and E. De Momi. Design and Evaluation of a Performance-based Adaptive Curriculum for Robotic Surgical Training: a Pilot Study. *Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference*, 2018:2162–2165, July 2018. doi: 10.1109/EMBC.2018.8512728 2

[32] Z. Palkova and I. Hatzilygeroudis. Virtual Reality and Its Applications in Vocational Education and Training. In *Handbook of Mobile Teaching and Learning*, pp. 1245–1274. Springer, Singapore, 2019. doi: 10.1007/978-981-13-2766-7_88 1

[33] J. E. Peabody, R. Ryznar, M. T. Ziesmann, and L. Gillman. A Systematic Review of Heart Rate Variability as a Measure of Stress in Medical Professionals.

*Cureus*, 15(1):e34345, Jan. 2023. doi: 10.7759/cureus.34345 3

[34] D. Rodenburg, P. Hungler, S. A. Etemad, D. Howes, A. Szulewski, and J. Mclellan. Dynamically adaptive simulation based on expertise and cognitive load. In *2018 IEEE Games, Entertainment, Media Conference (GEM)*, pp. 1–6. IEEE, 2018. 2

[35] K. R. Scherer. On the Nature and Function of Emotion: A Component Process Approach. In *Approaches To Emotion*. Psychology Press, London, England, 1984. Num Pages: 25. 2

[36] K. R. Scherer. What are emotions? And how can they be measured? *Social science information*, 44(4):695–729, 2005. 1

[37] T. Schubert, F. Friedmann, and H. Regenbrecht. The Experience of Presence: Factor Analytic Insights. *Presence: Teleoperators and Virtual Environments*, 10(3):266–281, June 2001. doi: 10.1162/105474601300343603 6

[38] V. Schwind, P. Knierim, N. Haas, and N. Henze. Using presence questionnaires in virtual reality. In *Proceedings of the 2019 Conference on Human Factors in Computing Systems (CHI)*, pp. 1–12, 2019. 2

[39] J. Sweller. Cognitive load during problem solving: Effects on learning. *Cognitive science*, 12(2):257–285, 1988. 2

[40] J. Sweller. Element interactivity and intrinsic, extraneous, and germane cognitive load. *Educational psychology review*, 22(2):123–138, 2010. 3

[41] D. Tao, H. Tan, H. Wang, X. Zhang, X. Qu, and T. Zhang. A Systematic Review of Physiological Measures of Mental Workload. *International Journal of Environmental Research and Public Health*, 16(15):2716, Jan. 2019. Number: 15 Publisher: Multidisciplinary Digital Publishing Institute. doi: 10.3390/ijerph16152716 2

[42] K. Tcha-Tokey, E. Loup-Escande, O. Christmann, and S. Richir. A questionnaire to measure the user experience in immersive virtual environments. In *Proceedings of the 2016 Virtual Reality International Conference*, pp. 1–5, 2016. 2

[43] M. A. Vidulich and P. S. Tsang. Mental workload and situation awareness. *Handbook of human factors and ergonomics*, pp. 243–273, 2012. 2, 4

[44] B. F. Yuksel, K. B. Oleson, L. Harrison, E. M. Peck, D. Afergan, R. Chang, and R. J. Jacob. Learn piano with bach: An adaptive learning interface that adjusts task difficulty based on brain state. In *Proceedings of the 2016 CHI conference on human factors in computing systems*, pp. 5372–5384, 2016. 2

[45] M. Zahabi and A. M. Abdul Razak. Adaptive virtual reality-based training: a systematic literature review and framework. *Virtual Reality*, 24(4):725–752, Dec. 2020. doi: 10.1007/s10055-020-00434-w 1, 2, 3

[46] Y. Zheng, M. Huang, L. Wang, J. Zhao, C. Zhai, and R. Yang. How Virtual Walking Task Difficulty Design Influences on Task Performance and User Experience. In *2022 8th International Conference on Virtual Reality (ICVR)*, pp. 155–162. IEEE, New York City, USA, May 2022. ISSN: 2331-9569. doi: 10.1109/ICVR55215.2022.9847681 3