# Microgestures to Perform a Situation Awareness Assistive Task in an Autonomous Vehicle: Workload, Ocular Behavior, Acceptance and Takeover Performance

Zoé Lacroux, Christophe Jouffrais, Pierre Vincent Paubel, Céline Lemercier

► To cite this version:

Zoé Lacroux, Christophe Jouffrais, Pierre Vincent Paubel, Céline Lemercier. Microgestures to Perform a Situation Awareness Assistive Task in an Autonomous Vehicle: Workload, Ocular Behavior, Acceptance and Takeover Performance. IHM'25 - 36e Conférence Internationale Francophone sur l'Interaction Humain-Machine, Nov 2025, Toulouse, France. hal-05302492

HAL Id: hal-05302492

https://hal.science/hal-05302492v1

Submitted on 7 Oct 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Microgestures to Perform a Situation Awareness Assistive Task in an Autonomous Vehicle: Workload, Ocular Behavior, Acceptance and Takeover Performance

## Utilisation des Microgestes dans la Réalisation d'une Tâche d'Aide au Maintien de la Conscience de la Situation à bord d'un Véhicule Autonome: Charge de Travail, Comportement Oculaire, Acceptation et Performance de Reprise des Commandes

Zoé Lacroux

CLLE Laboratory, University of Toulouse Jean Jaures, Zoelacroux@univ-tlse2.fr

Christophe Jouffrais

IRIT, CNRS, Christophe.jouffrais@cnrs.fr

Pierre Vincent Paubel

CLLE Laboratory, University of Toulouse Jean Jaures, paubel@univ-tlse2.fr

Céline Lemercier

CLLE Laboratory, University of Toulouse Jean Jaures, celine.lemercier@univ-tlse2.fr

The management of onboard systems in autonomous vehicles relies on various interaction techniques, such as voice recognition and touch interfaces. The present study aims to investigate the use of microgestures, a promising alternative interaction technique that has so far been little explored in this context. Microgestures are rapid and subtle finger movements with advantageous properties, such as being eye-free and requiring minimal physical effort. In this study, we aim to compare the use of a microgesture-based interaction technique with a touch-based interaction technique in the context of onboard system management in an autonomous vehicle. These techniques are used in a parallel task aiming at assisting situation awareness, which consists in answering questions related to the driving environment. Sixty participants are assigned to one of three conditions: a control condition without the parallel task, and two conditions with the parallel task using either a touch interface or microgestures. The study aims to evaluate the effects of the use of these techniques on

situation awareness (SART questionnaire), eye behavior, subjective workload (NASA-TLX), vehicle takeover performance, and perceived acceptance (TAM questionnaire). The results will help assess the effectiveness and acceptance of microgestures for parallel task management in autonomous driving and their impact on the driving supervision.

**CCS CONCEPTS** • Human-Centered computing • Human Computer Interaction (HCI)

**Additional Keywords and Phrases:** Autonomous driving, Interaction techniques, Microgestures, Situation Awareness

La gestion des systèmes embarqués dans les véhicules autonomes repose sur différentes techniques d'interaction,
telles que la reconnaissance vocale ou les interfaces tactiles. La présente étude a pour but de tester l'utilisation des microgestes, une technique d'interaction alternative prometteuse encore peu explorée dans ce contexte. Les microgestes sont des mouvements des doigts rapides et subtils possédant des propriétés avantageuses telles que le fait d'être réalisables sans vision ou d'être peu physiquement coûteux. Dans cette étude, nous comparerons l'utilisation d'une technique d'interaction basée sur les microgestes à l'utilisation d'une technique d'interaction basée sur le toucher dans le cadre de la gestion de systèmes embarqués à bord d'un véhicule autonome. La tâche utilisée sera une tâche parallèle visant à soutenir la conscience de la situation. Il s'agit d'une tâche de question-réponse portant sur l'environnement de conduite. Soixante participants seront répartis dans trois conditions: une condition contrôle sans tâche de soutien, une condition avec tâche de soutien réalisée via une interface tactile, et une condition avec tâche de soutien réalisée par microgestes. La présente étude a pour objectif d'évaluer les effets de ces techniques sur la conscience de la situation (questionnaire SART), le comportement oculaire, la charge mentale subjective (NASA-TLX), les performances de reprise en main du véhicule et l'acceptation perçue (questionnaire TAM). Les résultats permettront d'évaluer l'efficacité et l'acceptation des microgestes pour la gestion de tâches parallèles en conduite autonome, ainsi que leur impact sur la supervision de la conduite.

**Mots-clés additionnels** : Conduite autonome, Techniques d'interaction, Microgestes, Conscience de la Situation

**ACM format:** Zoé Lacroux, Christophe Jouffrais, Pierre Vincent Paubel, Céline Lemercier. 2025. Microgestures to Perform a Situation Awareness Assistive Task in an Autonomous Vehicle: Workload, Ocular Behavior, Acceptance and Takeover Performance. IHM '25 : Actes étendus de la 36e conférence Francophone sur l'Interaction Humain-Machine, Novembre, 4-7, 2025, Toulouse, France

# 1. INTRODUCTION

## 1.1. Autonomous driving

Technological advances in driving and its progressive automation have significantly modified driver's behavior and tasks. The human user's role progressively shifts from driving to supervising the system. One of the objectives of driving automation is to improve road safety by allowing the human driver to delegate some aspects of vehicle control to onboard systems. Currently, six levels of automation are defined by the Society of Automotive Engineers (SAE) (SAE, 2021).

At Level 0, the user is responsible for driving and supervision. The system only provides basic alerts (e.g., blind spot, lane departure warnings) and emergency braking. At Level 1, the user is still responsible for driving and supervision but the system offers increased assistance compared to Level 0. This includes speed control or lane centering support. At Level 2, the system assists with both speed and trajectory control simultaneously, although the user must remain engaged and supervise the system. From Levels 3 and 4, the system can perform the driving task under specific conditions without continuous user supervision. However, the user must be ready to take control if the system requests it (i.e., when conditions no longer allow autonomous driving). At Level 5, full automation is achieved, and the vehicle is capable of handling all driving tasks in all conditions without any intervention or fallback requirement from the human user.

In highly autonomous vehicles, the delegation of the driving task to the system enables the user to allocate time and resources to the onboard systems management (e.g., music player or gps management). The interaction with these systems is achieved through the use of various interaction techniques, such as voice or gesture recognition [1], [2]. Each of these techniques has advantages and
limitations, which make them more or less
appropriate for specific use cases.

## 1.2. Onboard systems management and interaction techniques

The most widespread interaction technique in autonomous vehicles is voice recognition [2]. While it is generally appreciated by users, its usability decreases in noisy environments or when the user's mouth is otherwise occupied (e.g., the user is eating or talking over the phone during autonomous driving) [2]. In the context of manual driving, voice input is considered the safest method, as its use results in fewer longitudinal deviations and less steering angle variability [3], and does not require visual attention [3], [4], [5]. However, voice commands also have drawbacks, such as high cognitive load [4], [5], the need to learn and memorize specific commands, and potential issues with privacy or interference in the presence of passengers. While the advantages of these technique are clear in manual driving, it might not be the most suitable option in the context of autonomous driving. Another interaction technique employed in both manual and autonomous driving is touch-based interaction. However, they are highly demanding in terms of visual resources [3], [4], increase cognitive load, and diminish the driver's situation awareness [6].

Gesture-based interaction techniques offer the advantage of not requiring visual attention [3], [7], [8]. These techniques are considered promising, safe, easy to use, and well-received by users [8], [9]. For effective implementation, they require appropriate feedback and a set of gestures that are easy to learn and remember [9]. Gesture-based techniques, however, consume motor resources and typically require the driver to remove their hands from the steering wheel [2]. Microgestures, a form of gesture-based interaction technique, consisting of quick and subtle finger movements that do not

involve the wrist or the rest of the arm, addresses the issue of muscular fatigue [10], [11]. They also offer greater flexibility than gestures, as it can be performed during grasping, and with the hands remaining on the wheel [7], [8], [12]. Like regular gestures, microgestures are eyes-free, as they rely on proprioception [13]. They also offer the advantage to be discreet, given their low amplitude, and allow confidentiality during the interaction [14], in contrast to voice or other types of gesture-based interfaces.

While voice and touch-based interfaces are already widely used for the task of managing the onboard systems in manual and autonomous driving contexts, microgestures have, so far, received less attention in this field. They appear to be a promising interaction technique for such use cases. In this study the use of microgestures for the control of a parallel task during autonomous driving will be compared to a more commonly used manual interaction technique, namely touch-based interaction. The parallel task that will be performed is a question/response task based on a preliminary study about situation awareness.

## 1.3. Use case: a situation awareness assistive application

As previously mentioned, the automation of driving leads to a shift in the user's role; the driver becomes a supervisor in highly automated vehicles. This shift in role implies changes in the perceptual, cognitive and motor processes associated with the task, particularly in terms of situation awareness [15], [16], [17]. Situation awareness is defined by Endsley (1988) as « the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future" [18]. It is therefore characterized by a three-level hierarchical structure: perception, comprehension, and projection [18], [19]. Several approaches can be
used to assess it, including questionnaires such as the SART, [20], behavioral or physiological measures such as eye behavior and cardiac activity [19].

As vehicle automation increases, the driver engagement in the driving task decreases, leading to a reduced situation awareness. This decline in situation awareness, observed under high levels of system automation, is accompanied by reduced driver alertness [17], [21]. A 2018 study shows that both peripheral detection performance (i.e., vibration on the left arm) and heart rate decrease when transitioning from manual to semi-automated driving [21]. These results suggest that automation comes with a risk of user's disengagement from the supervision task, even at semi-automated levels of driving automation [21]. At high levels of automation, users may even engage in parallel tasks, which has an even more detrimental effect on the supervision task and situation awareness [17], [22]. A 2014 meta-analysis indicates that situation awareness in highly automated driving can be improved when drivers remain actively engaged in the supervision task and attentive to the driving environment. However, if the driver is involved in an annex task, situation awareness deteriorates in comparison to manual driving [22]. This loss of situation awareness can lead to the “out-of-the-loop” phenomenon, which impair the ability to detect errors and cause longer attentional reorientation on the driving task [23], [24]. This may impair the driver’s ability to regain control of the vehicle when required and increase reaction times [24].

The automation of driving also leads to changes in visual strategies [15], [16], [17], [25]. In autonomous driving, drivers spend less cumulative time looking at the road, their visual strategies are less efficient as they are less likely to monitor key visual areas [16], [25], [26]. A 2019 study demonstrated a positive correlation between visual sampling and performance on a monitoring task [27]. Thus, visual attention to the driving scene

appears to play a role both in maintaining situation awareness and in supervision performance. These changes in visual strategy might contribute to the loss of situation awareness and consequently to the decrease in takeover performances.

The loss of situation awareness that comes with the automation of driving and the delegation of tasks to the system is a safety issue since it degrades takeover performance. In a previous preliminary study, authors created a question/response task based on three questions about the driving environment to help users maintain an adequate level of situation awareness, even when they are fully engaged in an annex task. The task is structured such that, in order to answer the questions regarding the driving environment, the user has to watch the road. This adaptation in ocular behavior should improve situation awareness [27]. The authors compared two conditions in an autonomous driving scenario, using an autonomous driving simulator. In one condition, participants received no questions and in the other, they were asked three questions related to the driving environment (e.g., "What is the color of the vehicle in front of you?"). Results show that the participants who received the questions about the environment reported higher levels of situation awareness, measured with the SART questionnaire. These results show that this type of task can be used as an application to ensure an adequate level of situation awareness.

In the present experiment, the same question/response task will be used as a parallel task to perform during autonomous driving phases. The aim is to compare the impact of the interaction techniques (i.e., microgestures and touch) used to perform this task on several variables of interest in the context of autonomous driving. Three conditions will be compared: (1) a control condition with no parallel task and two experimental conditions involving the parallel task: (2) performed via a touch-based interaction technique and (3)
performed via a microgesture-based interaction
technique. The aim is to study how touch-based and
microgesture-based interaction techniques for the
management of a parallel task (i.e., a
question/response task designed to help
maintaining situation awareness) differentially
impact workload, ocular behavior, situation
awareness and takeover performances in a highly
automated vehicle.

## 2. EXPERIMENT

The experiment has two goals: (1) assessing
the effectiveness of the situation awareness assistive
task (i.e., the question/response task) on the level of
situation awareness, ocular behavior, subjective
workload and takeover performance; (2) comparing
the use of touch-based interaction and microgesture-
based interaction techniques to perform the parallel
task on the ocular behavior, subjective workload and
acceptance. The data related to acceptance will be
collected for exploratory purposes.

Based on previous results on driving automation
suggesting that the changes in ocular behavior [27]
and lack of engagement in supervision deteriorates
situation awareness [17], [21], we do the following
hypotheses:

(1) *Hypotheses related to the situation awareness assistive task:*

H1: The level of situation awareness will be higher in the situation awareness assistive task condition.

H2: The total duration of eye gaze (i.e., number and duration of fixations) on the driving scene will be more important in the situation awareness assistive task condition.

H3: The takeover performances (response time and deviation from the trajectory) will be better in the situation awareness assistive task condition.

Based on the literature on the interaction
techniques, assuming touch-based interaction is
more demanding and necessitate visual resources
[3], [4], and considering literature suggesting that
microgesture and gesture-based interaction
techniques are easy to use with the eyes free [14], we
do the following hypotheses:

(2) *Hypotheses related to the interaction technique:*

H1: The subjective workload will be lower in the microgesture condition.

H2: The total duration of eye gaze on the driving scene will be more important in the microgesture condition.

H3: Due to longer total eye gaze duration on the driving scene, the level of situation awareness will be higher in the microgesture condition.

## 2.1. Material and method

2.1.1. Participants and design

We plan to recruit 60 participants having a
driver's licence. We will provide compensation in the
form of gift cards. Two factors are manipulated in a
fully between subjects nested factor design. The first
one is the "situation awareness assistive task
condition", with two levels: "with the assistive task"
(n= 30) and "without the assistive task" (n=30). The
second factor will be the "interaction technique",
with two levels: "touch-based interaction" (n= 15)
and "microgesture-based interaction" (n= 15).

2.1.2. Tasks and interaction techniques

Annex task: During the autonomous driving phases, participants are required to play a game on a tablet. They can pick between three games: « Maître logique 1 », « Brainilis » et “Rullo”. This annex task is used to make the situation as ecologically valid as possible and simulate autonomous driving conditions in which users are engaged in an annex task alongside supervision.
<figure><img src="image_2.png" /><figcaption>Figure 1: Microgesture-based interaction technique mapping</figcaption></figure>

**Primary supervision task:** *Tours, controls releasing and taking over phases:* Two different circuits will be used. The first circuit is a familiarization circuit designed to introduce participants with the equipment. This circuit includes straight roads, a few intersections, and several traffic circles. The driving conditions are relatively simple. Throughout the circuit, multiple phases of steering release and takeover are introduced to help participants get used to them. The duration of the circuit is approximately 15 minutes. The second circuit is the experimental one. It features a varied environment including curves, ascents and descents, mountainous and urban areas and more traffic. Most of the circuit is in autonomous mode, with a single takeover phase followed by 2 minutes of manual driving at the very end. The duration of the circuit is approximately 30 minutes.

**Parallel question/response task** (Situation awareness assistive task): In the assistive task condition, three different questions about the driving environment are presented to the driver: "Which sign is on your right?", "What is the speed limit?" and

"What is the color of the vehicle in front of you?". The participant selects one answer from four available options (see appendix A1).

**Interaction techniques:** The questions are presented in an audiovisual format. The participant hears the question and the four response options are simultaneously displayed on a small screen until the participant answers. In the touch-based interaction condition, the participant selects the answer they consider correct directly by touching the screen. In the microgesture-based interaction condition, participants use the ArcX ring (see apparatus section) to answer. The microgestures are performed using a joystick directly on the ring and pushing it with the thumb in one of the four directions (up, down, right and left). The microgesture responses is logically mapped to the spatial layout of the answer options (see figure 1).

### 2.1.3. Subjective measurements

**SART:** Situation Awareness Rating Technique (Taylor, 1990): The SART [31] is a subjective measurement scale used to assess situation
awareness after task completion. It includes 10 dimensions: familiarity with the situation, attentional focus, amount of information, quality of information, instability of the situation, concentration of attention, complexity of the situation, variability of the situation, arousal, and spare mental capacity. Participants rate their situation awareness on a 7-point Likert scale (from low to high). (see appendix A2)

**NASA-TLX:** NASA Task Load Index: The NASA-TLX [30] is a tool used to measure perceived workload. It includes six subjective subscales related to task demands, strategies, and individual experience: mental demand, physical demand, temporal demand, performance, effort, and frustration. Responses are rated on a scale from 0 to 100 (increasing intensity). A weighted average of the six items yields the overall perceived workload score. After rating each item, participants are presented with item pairs and must select the dimension that most contributed to their workload. In this study, a modified version was used. The dimensions of overall workload and general fatigue were added, the pairwise comparison step was removed, and responses were recorded using continuous scales ranging from 0 to 10. (see appendix A3)

*The basic TAM questionnaire:* The Technology Acceptance Model [28], [29] is a questionnaire used to measure the acceptance of a certain technology. It is based on the Technology Acceptance Model (TAM) developed by Davis [28]. It is based on three dimensions: Perceived Usefulness, Perceived Ease of Use and Behavioural Intention to Use. The participant answers on a Likert scale. (see appendix A4)

### 2.1.4. Apparatus

**Autonomous driving simulator:** The study takes place in the SIMULAUTO driving simulator, part of the Cognition, Behavior and Usages (CCU) platform (Maison de la Recherche, UT2J). The simulator consists of the cockpit of a Peugeot 207, a triptych display providing a 360° field of view, and surround sound with vibration feedback. The simulator allows for the simulation of autonomous driving and mode transitions. During the driving phase, participants do not operate the gear shift but only use the steering wheel, turn signals, accelerator, and brake pedals.

**Eye-Tracker:** The portable eye tracker (SMI Eye Tracking Glasses) allows for the collection of eye data in any type of environment. It captures data such as the participant's gazing path and areas of interest (AOI).

**ArcX smart ring:** To capture microgestures, we plan to use ArcX ring (https://arcx.fit/). It is a wearable electronic device in a ring shape, worn on the finger. It features a four-way directional joystick and a central button (see figure 2). The ring connects to interfaces via Bluetooth and is configured with a dedicated smartphone application.

<figure><img src="image_3.png" /><figcaption>Figure 2: the ArcX Smart Ring,
Picture from: https://arcx.fit/products/arcx-smart-ring</figcaption></figure>

### 2.1.5. Protocol

The experiment takes place at the CCU platform (Maison de la Recherche, UT2J). The session starts with the signature of a consent form and the instruction briefing. The participant is then seated on the simulator and is allowed to adjust the seat, mirrors, and steering wheel to ensure a comfortable
driving position. The usable vehicle controls were explained, including the steering wheel, pedals, turn signals, and the small screen used to answer the questions. The eye-tracker is introduced once the participant is properly seated, ready to proceed with the eye tracking calibration.

In the control condition, the participants start the experimental tour and play the tablet games. They are informed that they have to play the game (annex task) during the autonomous driving phase. Two minutes before the end of the tour, they are asked by the system to take over the commands (manual driving).

In the experimental condition, the experimenter explains that questions related to the environment will be presented during the tour. They hear and read the questions and choose one of the four proposed answers on the small screen located above the main display. They are informed that only one answer is correct. The experimenter explains the interaction technique corresponding to the following condition (touch-based or microgesture-based). Then, the tour starts the same way as in the control condition, with the tablet game. The only difference is that the three questions will be asked at three pre-defined moments during the tour.

At the end of the tour, in both control and experimental conditions, participants are asked to fill three questionnaires: the SART, the NASA-TLX and the basic TAM questionnaire (see appendices A2, A3 and A4).

### 3. EXPECTED RESULTS

About the situation awareness assistive task, we expect a higher level of situation awareness (measured by the SART questionnaire), along with increased frequency and duration of eye gaze directed at the driving scene, and improved take-over performance in the situation awareness task condition. Regarding the interaction technique, we expect lower workload (measured by the NASA TLX questionnaire), increased frequency and duration of eye gaze on the driving scene, and enhanced situation awareness when participants use microgestures to complete the situation awareness assistive task.

Assuming the anticipated effects about the interaction technique are observed, it would be interesting to study their generalizability to other task domains, such as embedded systems management (i.e., music control or GPS navigation).

### ACKNOWLEDGMENTS

The authors thank the ANR MIC in which this research project is undertaken. The authors thank Karine Rakotopare for the question/response task. We acknowledge the use of ChatGPT, an AI language model, for assistance with translation in the preparation of this manuscript.

### REFERENCES

[1] R. Häuslschmid, B. Menrad, et A. Butz, « Freehand vs. micro gestures in the car: Driving performance and user experience », in *2015 IEEE Symposium on 3D User Interfaces (3DUI)*, IEEE, 2015, p. 159-160. Consulté le: 7 décembre 2023. [En ligne]. Disponible sur: https://ieeexplore.ieee.org/abstract/document/7131749/

[2] A. Ataya, W. Kim, A. Elsharkawy, et S. Kim, « How to Interact with a Fully Autonomous Vehicle: Naturalistic Ways for Drivers to Intervene in the Vehicle System While Performing Non-Driving Related Tasks », *Sensors*, vol. 21, n° 6, Art. n° 6, janv. 2021, doi: 10.3390/s21062206.

[3] J. Ma, Z. Gong, et Y. Dong, « Interactive Modes F-ANP Evaluation for In-Vehicle Secondary Tasks », SAE Technical Paper, 2016. Consulté le: 30 novembre 2023. [En ligne]. Disponible sur: https://www.sae.org/publications/technical-papers/content/2016-01-1890/

[4] B. Pfleging, S. Schneegass, et A. Schmidt, « Multimodal interaction in the car: combining speech and gestures on the steering wheel », in *Proceedings of the 4th International Conference on Automotive User Interfaces and Interactive Vehicular Applications*, Portsmouth New Hampshire: ACM, oct. 2012, p. 155-162. doi: 10.1145/2390256.2390282.

[5] C. A. Pickering, K. J. Bumnham, et M. J. Richardson, « Support for modeling interaction with in-vehicle interfaces. », in *2007 2nd Institution of Engineering and Technology International Conference on System Safety*, IET, 2007, p. 223-228. Consulté le: 4 décembre 2023. [En ligne]. Disponible sur: https://ieeexplore.ieee.org/abstract/document/4399937/

[6] R. Tscharn, M. E. Latoschik, D. Löffler, et J. Hurtienne, « "Stop over there": natural gesture and speech interaction for non-critical spontaneous intervention in autonomous driving », in *Proceedings of the 19th ACM International Conference on Multimodal Interaction*, Glasgow UK: ACM, nov. 2017, p. 91-100. doi: 10.1145/3136755.3136787.
[7] L. Angelini et al., « WheelSense: Enabling Tangible Gestures on the Steering Wheel for In-Car Natural Interaction », in *Human-Computer Interaction. Applications and Services*, vol. 8005, M. Kurosu, Éd., in Lecture Notes in Computer Science, vol. 8005., Berlin, Heidelberg: Springer Berlin Heidelberg, 2013, p. 531-540. doi: 10.1007/978-3-642-39262-7_60.

[8] A. Riener, « Gestural Interaction in Vehicular Applications », Computer, vol. 45, n° 4, p. 42-47, avr. 2012, doi: 10.1109/MC.2012.108.

[9] L. Angelini et al., « Opportunistic synergy: a classifier fusion engine for micro-gesture recognition », in *Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications*, in AutomotiveUI '13. New York, NY, USA: Association for Computing Machinery, oct. 2013, p. 30-37. doi: 10.1145/2516540.2516563.

[10] C. Bailly et J. Castet, « WIM-Drone : Combiner un Monde-En-Miniature avec des Microgestes pour Faciliter la Téléopération en Réalité Augmentée », 5 mars 2023. Consulté le: 28 décembre 2023. [En ligne]. Disponible sur: https://hal.science/hal-04014967

[11] G. Li, Y. Liu, W. Song, C. Wang, et Y. Wang, « A New Dataset and Recognition for Egocentric Microgesture Designed by Ergonomists », in *Image and Graphics*, Y. Peng, S.-M. Hu, M. Gabbouj, K. Zhou, M. Elad, et K. Xu, Éd., in Lecture Notes in Computer Science. Cham: Springer International Publishing, 2021, p. 80-89. doi: 10.1007/978-3-030-87358-5_7.

[12] M. Murer, D. Wilfinger, A. Meschtscherjakov, S. Osswald, et M. Tscheligi, « Exploring the back of the steering wheel: text input with hands on the wheel and eyes on the road », in *Proceedings of the 4th International Conference on Automotive User Interfaces and Interactive Vehicular Applications*, Portsmouth New Hampshire: ACM, oct. 2012, p. 117-120. doi: 10.1145/2390256.2390275.

[13] J. Wambecke et al., « M [eye] cro: Eye-gaze+ Microgestures for Multitasking and Interruptions », Proc. ACM Hum.-Comput. Interact., vol. 5, n° EICS, p. 1-22, 2021.

[14] E. Chan, T. Seyed, W. Stuerzlinger, X.-D. Yang, et F. Maurer, « User Elicitation on Single-hand Microgestures », in *Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems*, in CHI '16. New York, NY, USA: Association for Computing Machinery, mai 2016, p. 3403-3414. doi: 10.1145/2858036.2858589.

[15] N. Merat et al., « The “Out-of-the-Loop” concept in automated driving: proposed definition, measures and implications », Cogn. Technol. Work, vol. 21, n° 1, p. 87-98, févr. 2019, doi: 10.1007/s10111-018-0525-8.

[16] C. D. Mole, O. Lappi, O. Giles, G. Markkula, F. Mars, et R. M. Wilkie, « Getting back into the loop: the perceptual-motor determinants of successful transitions out of automated driving », Hum. Factors, vol. 61, n° 7, p. 1037-1065, 2019.

[17] D. Schnebelen, C. Charron, et F. Mars, « Estimating the out-of-the-loop phenomenon from visual strategies during highly automated driving », Accid. Anal. Prev., vol. 148, p. 105776, 2020.

[18] M. R. Endsley, « Design and evaluation for situation awareness enhancement », in *Proceedings of the Human Factors Society annual meeting*, Sage Publications Sage CA: Los Angeles, CA, 1988, p. 97-101. Consulté le: 23 novembre 2023. [En ligne]. Disponible sur: https://journals.sagepub.com/doi/abs/10.1177/154193128803200221

[19] T. Zhang et al., « Physiological measurements of situation awareness: a systematic review », *Hum. Factors*, vol. 65, n° 5, p. 737-758, 2023.

[20] R. M. Taylor, « Situational Awareness Rating Technique (Sart): The Development of a Tool for Aircrew Systems Design », 2017, p. 111-128. doi: 10.4324/9781315087924-8.

[21] F. N. Biondi, M. Lohani, R. Hopman, S. Mills, J. M. Cooper, et D. L. Strayer, « 80 MPH and out-of-the-loop: Effects of real-world semi-automated driving on driver workload and arousal », *Proc. Hum. Factors Ergon. Soc. Annu. Meet.*, vol. 62, n° 1, p. 1878-1882, sept. 2018, doi: 10.1177/1541931218621427.

[22] J. C. F. de Winter, R. Happee, M. H. Martens, et N. A. Stanton, « Effects of adaptive cruise control and highly automated driving on workload and situation awareness: A review of the empirical evidence », *Transp. Res. Part F Traffic Psychol. Behav.*, vol. 27, p. 196-217, nov. 2014, doi: 10.1016/j.trf.2014.06.016.

[23] M. R. Endsley et E. O. Kiris, « The Out-of-the-Loop Performance Problem and Level of Control in Automation », *Hum. Factors J. Hum. Factors Ergon. Soc.*, vol. 37, n° 2, p. 381-394, juin 1995, doi: 10.1518/001872095779064555.

[24] M. Sassman, J. Deniel, T. Bellet, et B. Claverie, « Effets de l'automatisation de la conduite en situations de conduite normales et critiques: de l'étude empirique sur simulateur à la modélisation cognitive. », *Ingénierie Cogn.*, vol. 2, n° 1, 2018, Consulté le: 9 décembre 2024. [En ligne]. Disponible sur: https://hal.science/hal-01880401/document

[25] J. Navarro, M. François, et F. Mars, « Obstacle avoidance under automated steering: Impact on driving and gaze behaviours », *Transp. Res. Part F Traffic Psychol. Behav.*, vol. 43, p. 315-324, 2016.

[26] T. Louw, R. Madigan, O. Carsten, et N. Merat, « Were they in the loop during automated driving? Links between visual attention and crash potential », *Inj. Prev.*, vol. 23, n° 4, p. 281-286, 2017.

[27] J. C. F. De Winter, Y. B. Eisma, C. D. D. Cabrall, P. A. Hancock, et N. A. Stanton, « Situation awareness based on eye movements in relation to the task environment », *Cogn. Technol. Work*, vol. 21, n° 1, p. 99-111, févr. 2019, doi: 10.1007/s10111-018-0527-6.

[28] F. D. Davis, « Perceived usefulness, perceived ease of use, and user acceptance of information technology », MIS Q., p. 319-340, 1989.

[29] M. Turner, B. Kitchenham, D. Budgen, et P. Brereton, « Lessons learnt undertaking a large-scale systematic literature review », in *12th International Conference on Evaluation and Assessment in Software Engineering (EASE)*, BCS Learning & Development, 2008. Consulté le: 27 juin 2025. [En ligne]. Disponible sur: https://www.scienceopen.com/hosted-document?doi=10.14236/ewic/EASE2008.12

[30] S. G. Hart et L. E. Staveland, « Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research », in *Advances in psychology*, vol. 52, Elsevier, 1988, p. 139-183. Consulté le: 17 mars 2025. [En ligne]. Disponible sur: https://www.sciencedirect.com/science/article/pii/S0166411508623869

[31] R. M. Taylor et S. J. Selcon, « Cognitive Quality and Situational Awareness with Advanced Aircraft Attitude Displays », *Proc. Hum. Factors Soc. Annu. Meet.*, vol. 34, n° 1, p. 26-30, oct. 1990, doi: 10.1177/154193129003400107.
# A APPENDICES

## A.1 The question/response task

1. Quel est le panneau qui se trouve sur votre droite?

- Virage à gauche

- Limité à 90 km/h

- Route prioritaire (réponse correcte)

- Cédez le passage

2. Quelle est la vitesse maximale autorisée?

- 110 km/h

- 70 km/h (réponse correcte)

- 80 km/h

- 90 km/h

3. Quelle est la couleur du véhicule qui arrive en face de vous?

- Noire (réponse correcte)

- Rouge

- Verte

- Blanche

## A.2 SART questionnaire

1. Instabilité de la situation : Dans quelle mesure la situation est-elle changeante ? La situation est très instable et susceptible de changer soudainement (7) ou est-elle très stable (1) ?

2. Complexité de la situation : Dans quelle mesure la situation est-elle complexe ? La situation est très complexe avec une multitude d'éléments (7) ou est-elle simple et claire (1) ?

3. Variabilité de la situation : Comment jugez-vous la variabilité de la situation ? Il y a un grand nombre de facteurs qui varient (7) ou très peu de facteurs qui varient (1) ?
4. Explication: Dans quelle mesure la situation est-elle stimulante ? Etes-vous très vigilant et prêt à agir (7) ou peu vigilant (1) ?

5. Focalisation de l'attention : Dans quelle mesure focalisez-vous votre attention sur la situation ? Vous concentrez-vous sur de nombreux aspects (7) ou sur un seul (1) ?

6. Division de l'attention : Dans quelle mesure votre attention est-elle divisée dans la situation ? Vous concentrez-vous sur plusieurs aspects de la situation (7) ou sur un seul (1) ?

7. Ressources mentales disponibles : De quelles ressources mentales disposez vous dans cette situation ? Vous disposez de suffisamment de ressources mentales pour vous occuper de nombreuses variables (7) ou pas assez pour traiter toutes les variables de la tâche (1) ?

8. Quantité d'information : Quelle quantité d'informations avez-vous acquise sur la situation ? Vous avez reçu et compris beaucoup d'éléments (7) ou très peu (1) ?

9. Qualité de l'information : Quelle est la qualité des informations que vous avez reçues sur la situation ? Les informations communiquées sont très utiles (7) ou insuffisantes (1) ?

10. Familiarité avec la situation : Dans quelle mesure êtes-vous familier avec la situation ? Considérez-vous avoir beaucoup d'expérience dans la situation (7) ou s'agit-il d'une situation nouvelle (1) ?

## A.3 NASA TLX questionnaire

1. Exigence mentale : Dans quelle mesure des opérations mentales et perceptives ont-elles été requises ? (ex : penser, décider, calculer, se rappeler, regarder, chercher, etc.) La tâche était-elle plutôt facile (0) ou difficile (100) ?

2. Exigence physique : Dans quelle mesure des opérations physiques ont-elles été requises ? (ex : pousser, tirer, forcer, tourner, activer, etc.) La tâche était-elle sans exigence physique (0) ou beaucoup d'exigence physique (100) ?

3. Exigence temporelle : Quelle quantité de pression liée au temps (rythme, allure des tâches, apparition des éléments de la tâche) avez-vous ressenti ? La tâche était-elle sans pression temporelle (assez de temps pour tout faire) (0) ou avec une pression temporelle importante (très peu de temps pour tout faire) (100) ?

4. Effort : Quelle a été la difficulté à accomplir (mentalement et physiquement) la tâche avec un niveau de performance tel que le vôtre ? La tâche demandait un effort faible (0) ou un effort important (100) ?

5. Performance : Quelle réussite vous attribuez-vous en ce qui concerne l'atteinte des buts de la tâche fixés par l'expérimentateur ou par vous-même ? Dans quelle mesure êtes-vous satisfait de votre performance dans la tâche ? Pas satisfait(e) (0), très satisfait(e) (100) de ma performance ?

6. Frustration : Au cours de la tâche, quel sentiment d'être peu sûr de vous, découragé, irrité, stressé et agacé avez-vous ressenti ? La tâche m'a peu frustré (0) ou très frustré (100) ?

## A.4 TAM questionnaire

Utilité perçue : Utiliser le système améliore ma performance. Pas du tout (1) ou Tout à fait d'accord (7)

Utilité perçue : L'utilisation du système dans mon travail augmente ma productivité. Pas du tout (1) ou Tout à fait d'accord (7)
Utilité perçue : Utiliser le système améliore mon efficacité. Pas du tout (1) ou Tout à fait d'accord (7)

Facilité d'usage perçue : Mon interaction avec le système est claire et compréhensible. Pas du tout (1) ou Tout à fait d'accord (7)

Facilité d'usage perçue : Interagir avec le système ne nécessite pas beaucoup d'effort mental de ma part. Pas du tout (1) ou Tout à fait d'accord (7)

Facilité d'usage perçue : Je trouve que le système est facile à utiliser. Pas du tout (1) ou Tout à fait d'accord (7)

Facilité d'usage perçue : Je trouve facile d'obtenir du système ce que je veux qu'il fasse. Pas du tout (1) ou Tout à fait d'accord (7)

Auto efficacité du dispositif : Je pourrais utiliser ce dispositif...

- Si personne n'est à côté pour m'accompagner

- Si il y a un bouton d'aide pour m'assister

- Si quelqu'un me montre comment faire avec

- Si j'ai déjà utiliser un dispositif similaire avant pour faire le même travail

Caractère ludique de l'interaction : Je pense que je serai créative. En utilisant ce dispositif. Pas du tout (1) ou Tout à fait d'accord (7)

Caractère ludique de l'interaction : Je pense que je serai spontané. En utilisant ce dispositif. Pas du tout (1) ou Tout à fait d'accord (7)

Caractère ludique de l'interaction : Je pense que je serai enjoué. En utilisant ce dispositif. Pas du tout (1) ou Tout à fait d'accord (7)

Caractère ludique de l'interaction : Je pense que je serai sans originalité en utilisant ce dispositif. Pas du tout (1) ou Tout à fait d'accord (7)

Anxiété face au dispositif : Ce dispositif ne me fait pas peur du tout. Pas du tout (1) ou Tout à fait d'accord (7)

Anxiété face au dispositif : Travailler avec ce dispositif me rend nerveux. Pas du tout (1) ou Tout à fait d'accord (7)

Anxiété face au dispositif : Ce dispositif me rend mal à l'aise. Pas du tout (1) ou Tout à fait d'accord (7)

Anxiété face au dispositif : Face à ce dispositif je me sens en difficulté. Pas du tout (1) ou Tout à fait d'accord (7)

Plaisir perçu : Je trouve que l'utilisation du système est agréable. Pas du tout (1) ou Tout à fait d'accord (7)

Plaisir perçu : L'utilisation effective du système est plaisante. Pas du tout (1) ou Tout à fait d'accord (7)

Plaisir perçu : Je m'amuse en utilisant le système. Pas du tout (1) ou Tout à fait d'accord (7)