# Innovative methodological framework for virtual human centered design

Jonathan Deniel, Jean-Charles Bornard, Thierry Bellet, Bernard Claverie

► To cite this version:

Jonathan Deniel, Jean-Charles Bornard, Thierry Bellet, Bernard Claverie. Innovative methodological framework for virtual human centered design. INCOSE - Human System Integration Conference, Sep 2019, Biarritz, France. hal-04984854

HAL Id: hal-04984854

https://hal.science/hal-04984854v1

Submitted on 10 Mar 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# Innovative methodological framework for virtual human centered design: An application to automated driving effects on driver's decisions-making

**Jonathan Deniel**

IFSTTAR - LESCOT
25, av. François Mitterrand
F-69675 Bron Cedex
jonathan.deniel@ifsttar.fr

**Bernard Claverie**

Bordeaux INP - ENSC
109 Avenue Roul
33400 Talence
Bernard.claverie@ensc.fr

**Jean-Charles Bornard**

CIVITEC - ESI-Group
Le Récamier. 70, rue Robert
69458 Lyon
jcb@esi-group.com

**Thierry Bellet**

IFSTTAR - LESCOT
25, av. François Mitterrand
F-69675 Bron Cedex
Thierry.bellet@ifsttar.fr

## ABSTRACT

This paper describes an innovative simulation platform and a live adaptive experimental protocol to include the human centered design approach for the researcher in Automated Vehicle design. With the spreading use of automated driving systems, Human drivers will adjust their activities and habits. This early stage research paper presents an innovative integration of tailored driving situations proposed to human drivers. To address issues about impacts on driver's decision-making in highway lane-change situations after an automated phase, the Virtual Human Centered Design platform is used. It implements the adaptive experimental protocol, including driving simulation, automated vehicles simulation and dynamic situations computation.

## Keywords

Decision making, versatile simulation platform, automated vehicle, adaptive experimental protocol, Human behaviour.

## INTRODUCTION

Development of Automated Vehicles (AV) is already a major topic for car manufacturers and Original Equipment Manufacturers (OEM) [1]. The Society of Automotive Engineers (SAE), regrouping specialists of transport industries, defines since 2014 terms related to driving automation systems [7]. The SAE taxonomy defines levels of automation from 0 to 5, where level 0 describes a car without any driving automation and level 5 a car that doesn't need a steering wheel nor pedals and can drive under all conditions. Between those, Advanced Driver-

Assistance Systems (ADAS) and partial automated systems can coexist with the driver, transforming the driving activity to a human system cooperation [3]. We can already see the appearance of partially automated vehicles on the roads, at SAE levels of 2, and soon can be of level 3. But the step from one to the other get the driver off the driving activity when the feature is engaged. Using a level 3 car, driver can face at any time a "take over request", where the automated driving feature asks and notifies the driver to get back to the driving activity. These transitions, with high time pressure, are studied in many ways [2], but remote effects on driver's decision-making following a period of automated driving are much less so.

This early stage research paper will focus on the description of the V-HCD<fnref n="1" /> simulation platform, a tool providing users capability to create adaptive experimental protocol, and its use around a specific study. The study aims to assess the potential impact of an automated driving phase on a consecutive manual driving phase. Take over requests are not addressed, the story that would best describe these situations would approach a driver driving a vehicle to a part of the route allowing the use of an automated driving system. Then following this automated phase, the driver takes control back to complete the journey.

To properly address these scientific issues, it is necessary to use tools (e.g. the V-HCD simulation platform) and methods (e.g. adaptive experimental protocol) to obtain the relevant information necessary for the analysis, modelling and simulation of observed behaviours. It is thus certain to meet these requests:

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.

HSI2019, September 11-13, 2019, Biarritz, France.

<footnote n="1">¹ Virtual Human Centered Design</footnote>
- Driving situations simulation: simulate scenarios where human drivers can face switch from an automated driving phase to a manual driving phase

- Driving simulator: Human drivers must face these situations using an immersive driving simulator, also allowing simulation of automated driving vehicles. This includes the control by the AV of blinkers of the cabin, the steering wheel angle, etc.

- Human centred data collection: collect and synchronize data using dedicated methodology (e.g. questionnaire or subjective assessments) and specific sensors (e.g. eye tracking or pedals depression), for analysis.

- Experimental protocol complexity: Simulations need an embedded questionnaire, a useful way to manage situation propositions to subjects, and a data management system.

From these needs, we can now describe the experimental protocol that have been implemented in the V-HCD simulation platform for the study, as an example to guide the platform design.

## PROTOCOL

Highway driving is currently the most common driving phase to be highly automated [5]. One aspect of complexity in highway driving is Lane Change (LC) especially when the situation requires the car to change lane by inserting itself in the targeted lane traffic. Understanding how the driver decides to change lane and inserts in between two cars in the traffic is as important [8] as understanding how the driver is impacted by this decision when taken by an AV. The availability of such AV may not only raise questions about LC decisions acceptance but also about potential effects on the driver's LC decisions and behaviour after an extended period of use. To better understand how driver takes LC decisions, accepts these decisions when taken by an AV and is eventually affected by its use, a driving simulator based experimental protocol has been designed.

The analysis of decision-making implies that decision-making processes take place frequently enough to be able to collect a substantial amount of data for as many people as possible. We then need every individual to be exposed to a high number of controlled LC opportunity driving situations where he/she will have to take a LC decision. Additionally, the study of LC capable AV acceptability requires these same LC driving situations to be studied with the aforementioned drivers on board. Finally, the effect of an extended AV use requires every studied individual to both be exposed to a wide range of AV behaviour and a long time of use. It is then possible to design a protocol divided in three phases: Manual driving, Automated driving, Post-Automated driving. The first two phases will only differ on the aspect of manual vs. automated driving mode, whereas the third phase will be shorter and adapted

to every participant (this aspect will be developed further below). This protocol should provide data for driver cognition modelling.

## Driving situations:

To obtain comparable driving situations and controlled conditions, the situations in which lane change decision opportunities will be presented to the participants are standardised as follows:

In the first phase (manual driving), the participant will be instructed to drive at 90km/h on the right lane of a 2x2 lane highway. As this cruise speed is reached a black car platoon overtakes on the left lane at a speed of 110km/h. Among the black platoon cars are two consecutive white ones separated by a determined empty space gap. As the white cars approach, the participant is required to decide whether lane changing and insertion in the empty gap is possible or not. If so, the LC should be executed. An emphasis is put on the instruction about the fact that despite the participant is driving in a simulator he/she is highly encouraged to decide and proceed by adapting his/her trajectory and speed as he/she would usually do in real life open road conditions. No specific pressure is imposed about the necessity to change lane hence ensuring that the participant does not take unusually risky decisions. Every driving situation ends when the LC is completed or when the participant's car is overtaken by the gap closing white car. By the end of each driving situation the driver is instructed to answer a continuous scales questionnaire about the taken decision.

In the second phase (automated driving) the onset of the driving situation is the same as for the previous phase. The main difference is that in all cases the assistance takes the decision to change lane and performs the manoeuvre. The way the manoeuvre is executed by the assistance is rigorously defined. The AV car first accelerates to reach the right lane traffic speed at the median distance point between the gap opener white car (GO) and the gap closer car (GC). Once this condition met in the right lane the car swerves to the left lane ending up in the exact middle of the available space gap at the exact speed of the right lane traffic. This procedure ensures that times to collision (TTC) and times headways (THW) [6] are maintained in safety margins. It is then possible to implement any lane change with a maximized safety possible level if the gap space allows it. To enable a better immersion the AV should not only be able to navigate in the virtual environment but also physically move the steering wheel and activate blinkers of the cabin when the LC decision is taken. Each driving situation ends when the AV reaches the left lane as in manual driving phase, then the participant answers a continuous scale questionnaire about how they experienced the decision.

For both manual and automated driving phases 12 different gap spaces are chosen. Ranging from 0.8s to 3.0s time headway (THW) between GC and GO and by 0.2s steps.
These extrema were chosen considering that the shortest one proposes a LC to the driver that cannot be safely performed and the longest one is so wide that every driver should accept to perform the LC. As participants may be inconsistent in their decisions, it is necessary to repeat each gap size presentation several times. Hence every gap size is presented three times in the manual driving phase as in the AV phase, consequently each of the two first experimental phases consists of 36 randomly ordered driving situations.

Two types of questionnaires are used, a first one dedicated to a general evaluation of driving automation acceptability [4] is completed by the participant before the first phase. A second one consists of questions, proposed at the end of each situation, on 0-100 continuous scales about the decision and execution correctness of the manoeuvre performed by the AV. These questions deal with the situation's criticality estimated by the participant, the acceptance of such a manoeuvre, and correctness of the decision.

Considering these lane change decision situations, one can expect every individual participant to have a range of preferred gap size at which the decision to change lane switch from negative to positive. One can easily imagine a very cautious driver to only change lane if the gap size is large (e.g. > 2.8s) and a more reckless one who would attempt LC with very small gaps (e.g. 1.0s). It seems then that each driver should have a more or less specific Threshold Situation (TS). A threshold situation is a set of scenarios with gap sizes where the driver will mainly refuse to take the LC for the lower gap size, and mainly take the decision to perform the LC for greater gap size.

As this threshold situation is very individual, one can assume that the automated vehicle lane change behaviour may not correspond to the participant's own threshold. Consequences of this discrepancy may not only concern acceptance issues but could also influence the driver in his or her decisions after returning to a manual driving mode. This potential effect does not only concern the time interval of the transition from automated to manual but may also impact later decisions. To address this question, it is necessary to identify the participants threshold situation during the first experimental phase (manual driving mode), then expose them to several LC decisions from the AV during the second experimental phase and finally set a third experimental phase tailored for each individual participant.

This post-automated manual driving phase will expose the participants to three LC driving situations, around their threshold situation. These gaps are identical to those from the first phase but are specifically selected based on the first phase decisions and questionnaire answers. We can thus consider this design as an adaptive experimental protocol and propose an algorithm to achieve it.

Such a protocol provides specification for an experimental platform to be able to manage the experiment. The platform must provide:

- A unified data recording system

- Driving situations order management, with a switch from manual to automated driving mode, and embedded questionnaires

- Highly customizable and controllable driving situations

- Adaptive experimental procedure compatibility (e.g. online threshold situation assessment)

## THE VHCD PLATFORM

As the user (experimenter) need all these specific functions, a dedicated instantiation of the V-HCD platform is built. It consists of the data gathering system, the Experimental Procedure Management System (EPMS), the driving situations simulation and scenarios management, and the threshold situation finding algorithm.

### Data gathering

To fulfil analysis needs, the virtual driving platform must record different sources of data. Data corresponding to dynamic actors in scenarios come from vehicle's dynamic models (position, speed, acceleration, etc.). Data recorded from specific sensors, like pedals depression or steering wheel angle, are recorded synchronously with models'. All these data are synchronized with videos of the inside of the cabin cockpit, and other data sources like the eye-tracking system, managed by a third-party software. Moreover, data from questionnaires are linked to the corresponding situation, allowing cross analysis. EPMS data recording system is managed by the operator through triggers that can be launched live or integrated inside a scenario to switch the recording system ON/OFF.

### Experimental procedure management system

The EPMS is based on a conceptual structure called "scenario". A scenario is a sequence of stages. Each stage is a step where one or more instructions can be programmed. These instructions can be given to control elements to set up a driving situation (e.g. reposition a car), but it can also consist of instructions to launch any external script or algorithm (e.g. eye-tracking recording software), or even be another scenario. The transition from one stage to another can be done by defining transitions conditions. When, in a specific stage, a transition condition is met (e.g. a car reaches a particular position), the next stage is triggered executing its instructions.

The EPMS is in fact a "meta-scenario" consisting in nested scenarios (i.e. scenario of scenarios). In this specific EPMS instantiation 6 main steps are implemented: launching and recording the pre-experiment questionnaire; launching and recording the phase 1 manual driving scenarios; launching and recording the phase 2 automated driving scenarios; running the threshold situation detection algorithm; launching and recording the phase 3 post-automated manual driving scenarios.

Each driving situation phase consists of a list of driving scenarios where the gap space is given for each scenario.
When running this sequential list, the EPMS first runs a driving scenario, at the end of it, EPMS launches the post-situation questionnaire and displays it on the embedded simulator cabin touchscreen. The experimenter oversees the transitions between situations by pressing a defined key ensuring high level control of the overall process. EPMS also allows the experimenter to go back to any questions in the post-situation questionnaires.

## Driving situation scenarios

The cars are controlled through high level customizable instructions such as target speed, lane keeping, lane changing (etc.) and behave through to the use of vehicle models that allow for realistic movement and sounds. Instructions can be configured by the operator to fit the desired LC behaviour. For this experiment the driving scenarios were implemented as prescribed in the protocol and consisted of several stages: Initialize the cars and the environment, set an initial speed and lane keeping instruction for all the cars (except for the participant's car in manual mode), create the proper space gap between GO and GC, let the platoon overtake the participant's car, end the scenario once the participant's decision is taken. In the case of AV driving situations scenario, the LC is programmed according to the protocol prescriptions. Physical cabin blinkers are remotely activated as well as the steering wheel angle movements, the scenario ends once the LC is done or not reachable anymore.

## Threshold situation finding algorithm

At the end of the first two driving phases the threshold situation detection algorithm accesses the participants data to find the TS. The principle of this algorithm is to first go through the participants lane change decisions and proceed to a rolling sum considering each specific gap size with its previous and following sizes. The longest gap size with a total sum inferior or equal to 1 is set as the lower bound (LB), the shortest gap size with a total sum superior or equal to 8 is set as the upper bound (UB). The central bound (CB) is then set as the mean between UB and LB. If the CB is a gap size already used and implemented (e.g. 1.8s) it is then defined as the TS for the post-automated driving phase, if the CB is not already used (e.g. 1.5s) the TS is defined by considering the participants answers to the questionnaires about the immediately inferior and superior gap sizes situations (i.e. 1.4s and 1.6s). Once the TS found, the post-automated phase's three manual driving scenarios are launched with respectively TS value for the first one, TS + 0.2s for the second one and TS - 0.2s for the last one.

## CONCLUSION

V-HCD is a new prototypic simulation tool that provides technical opportunities to allow live individual adaptation of the experimental protocol. This early stage research aims

at describing an example of a dedicated V-HCD platform to the analysis of the impact of AV usage on human decision making, including an innovative way of protocol design and management. Experiment and data analysis will give us feedbacks on the use of live protocol adaptation, potentially validating this new approach of human centered design for experimenters.

## ACKNOWLEDGMENTS

We thank IFSTTAR institute for funding this research, and ESI-Group for their assistance in the technical set up of this research.

## REFERENCES

[1] Badue, C., Guidolini, R., Carneiro, R. V., Azevedo, P., Cardoso, V. B., Forechi, A., ... & Oliveira-Santos, T. (2019). Self-Driving Cars: A Survey. arXiv preprint arXiv:1901.04407.

[2] Bazilinskyy P., Petermeijer S.M., Petrovych V., Dodou D., de Winter J.C.F. (2018). Take-over requests in highly automated driving: A crowdsourcing survey on auditory, vibrotactile, and visual displays, Transportation Research Part F: Traffic Psychology and Behaviour, Vol. 56, P. 82-98, ISSN 1369-8478,

[3] Bellet T., Boverie S., Boy G., Hoc J.M., (2010), De l'interaction à la coopération homme-machine : vers le copilotage automobile., in Kolski C., Interaction homme-machine dans les transports, personnalisation, assistance et informations du voyageur., pp. 151 – 177

[4] Distler V., Lallemand C., and Bellet T. (2018). Acceptability and Acceptance of Automated Mobility on Demand: The Impact of an Immersive Experience. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, New York, NY, USA, Paper 612, 10 pages.

[5] Dokic J., & Müller B., Meyer G., (2015). European Roadmap Smart Systems for Automated Driving, European Technology Platform on Smart Systems Integration (EPoSS)

[6] Green P. (2013). Standard definitions for driving measures and statistics. Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications - AutomotiveUI'13.

[7] SAE International (06/2018) J3016_201806: Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles (Warrendale: SAE International)

[8] Salvucci D., Liu A., (2002), The time course of a lane change: Driver control and eye-movement behavior, Transportation Research Part F: Traffic Psychology and Behaviour, Vol. 5, 2, pp. 123-132