# Cooperation between Driver and Automated Driving System: Implementation and Evaluation

Chunshi Guo, Chouki Sentouh, Jean-Christophe Popieul, Jean-Baptiste Haué, Sabine Langlois, Jean-Jacques Loeillet, Soualmi Boussaad, Thomas Nguyen
That

► To cite this version:

Chunshi Guo, Chouki Sentouh, Jean-Christophe Popieul, Jean-Baptiste Haué, Sabine Langlois, et al. Cooperation between Driver and Automated Driving System: Implementation and Evaluation. Transportation Research Part F: Traffic Psychology and Behaviour, 2019, 61, pp.314-325. 10.1016/j.trf.2017.04.006. hal-01643903

HAL Id: hal-01643903

https://hal.science/hal-01643903v1

Submitted on 14 Jan 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Cooperation between driver and automated driving system: Implementation and evaluation

Chunshi Guo<sup>a,b,c,*</sup>, Chouki Sentouh<sup>b</sup>, Jean-Christophe Popieul<sup>b</sup>, Jean-Baptiste Haué<sup>a</sup>, Sabine Langlois<sup>a,c</sup>, Jean-Jacques Loeillet<sup>c</sup>, Boussaad Soualmi<sup>c</sup>, Thomas Nguyen That<sup>d</sup>

## ABSTRACT

Recent years have witnessed rapid advancement of automated driving technologies. In this context, driver-vehicle cooperation as a new interaction paradigm offers an opportunity to improve the driving performance through the exploitation of human-automation synergy. This paper presents the implementation and evaluation of a new cooperation principle between the driver and the automated driving system. Within a use case concerning high-way merging management, we describe the scenario modelling, system functional and HMI design. We evaluate the cooperation principle and the designed HMI through a user study on driving simulator. Based on test results, we discuss user's perception of the cooperation principle and the potential of this principle to handle highway merging situations. Through this paper, we also aim to demonstrate the strengths of driving simulation in terms of interaction design for automated driving systems.

### Keywords:

Automated driving

Driver-vehicle

cooperation Use case

method

HMI

Scenario modelling

## 1. Introduction

### 1.1. Context and motivation

Considered as a promising solution to improve the safety, efficiency and comfort of our road transport system, automated driving (AD) technologies receive increasingly research efforts and make continuous advancement. According to SAE's taxonomy of levels of vehicle automation (SAE, 2014), AD systems refer to those systems that are capable of performing all driving tasks without human's monitoring, covering the highest three levels of automation (from Level 3 to Level 5). Demonstrations such as autonomous vehicles in the DAPRA Urban Challenge (Urmson et al., 2009) and the Google car (Markoff, 2010) suggest that it may be technically feasible to develop such an AD system. In this context, how a future AD system should interact with a human driver constitutes a key question of research for AD system designers.

For AD systems of SAE Level 3, most human factors studies focus on control transitions between manual driving (MD) and AD (Marinik et al., 2014). Particularly, the transition from AD to MD initiated by the automation in so-called "takeover scenarios" receives a great degree of attention (Blanco et al., 2015; Gold, Damböck, Lorenz, & Bengler, 2013; Lorenz,

<footnote><sup>a</sup>Technocentre Renault, Cognitive Ergonomics & HMI, 1 avenue du Golf, 78280 Guyancourt, France</footnote>

<footnote><sup>b</sup>University of Valenciennes, LAMIH - CNRS UMR 8201, Mont Houy, F-59313 Valenciennes, France</footnote>

<footnote><sup>c</sup>Technological Research Institute SystemX, LRA, 8 Avenue de la Vauve, 91120 Palaiseau, France</footnote>

<footnote><sup>d</sup>OKTAL, 19 Boulevard des Nations Unies, 92190 Meudon, France</footnote>

<footnote>* Corresponding author at: Technocentre Renault, Cognitive Ergonomics & HMI, 1 avenue du Golf, 78280 Guyancourt, France.
E-mail address: chunshi.guo@renault.com (C. Guo).</footnote>
Kerschbaum, & Schumann, 2014; Walch, Lange, Baumann, & Weber, 2015), since this transition could cause safety issues if the driver fails to take over vehicle control. However, few works address the case in which an operating AD system shares the control authority with the human driver.

This paper is a result of our studies on driver-vehicle cooperation (referred to as “cooperation” hereafter), a new interaction paradigm to address the shared authority issue between the AD system and the driver. Through this paradigm, the driver can cooperate with the operating AD system on driving tasks. Our motivations to propose such a paradigm are derived from the following two reasons.

Firstly, a human driver may interfere with the AD system in driving tasks. As users of AD systems, human drivers may have expectations on system's driving behaviors. Current complex safety concepts (Hörwick & Siedersberger, 2010) and risk-based decision strategies (Vanholme, Gruyer, Lusetti, Glaser, & Mammar, 2013) could make an AD vehicle behave over conservatively compared with human drivers. A driver that monitors the automation may have needs to intervene in AD mode, in case that he is not satisfied with the driving behaviors performed by the latter. An interaction paradigm that enables the AD system to share the authority with the driver is expected to meet such potential user needs. It is also expected to improve the usability of the system, because the driver need not disengage the system every time he would like to intervene.

Secondly, we intend to explore synergy between humans and machines. Human driving activities exhibit strong social patterns, e.g. cues like eye contacts and hand gestures, which are difficult for a machine to interpret (Färber, 2016). Compared with a machine, a human driver can detect these patterns easily and make an adequate decision following social rules. Therefore, by allowing the human driver to intervene, the AD system can benefit from the help of the human driver to better handle interactions with other vehicles manually driven by humans.

## 1.2. Theoretical background

Over the past decades, human-machine cooperation has been gaining interest in the academic community as a framework to address the shared authority issue within a human-machine system. According to the definition of Hoc (2001), “cooperation” in the context of human-automation interaction implies that the human and the automation “interfere with each other on goals, resources, procedures, etc.” However, “each one tries to manage the interference to facilitate the individual activities and/or the common task”. Among other fields, the concept of human-machine cooperation has been applied in telerobotics (Sheridan, 1995), air traffic control (Pacaux-Lemoine & Debernard, 2002) and robotic manipulation (Mörtl et al., 2012). In the automotive domain, human-machine cooperation has been implemented within shared control framework for vehicle control (Abbink & Mulder, 2010; Anderson, Karumanchi, Iagnemma, & Walker, 2013; Soualmi, Sentouh, Popieul, & Debernard, 2014). Its application for automotive systems has also been approached from a cognitive perspective (Flemisch, Bengler, Bubb, Winner, & Bruder, 2014; Hoc, Young, & Blosseville, 2009). Recent European projects like HAVEit (Hoeger et al., 2008) and ABV (Sentouh, Popieul, Debernard, & Boverie, 2014) demonstrated the interest of applying human-machine cooperation for AD systems.

In our previous work on the design of cooperation for AD systems (Guo, Sentouh, Soualmi, Haue, & Popieul, 2015), we adopted a top-down approach. We firstly identified a common hierarchy between a classical functional architecture for AD systems (Urmson et al., 2009) and the Michon's model for human driving behavior (Michon, 1985). Then at each level of the hierarchy, we proposed a cooperation principle as a generic interaction method to manage the interference at this level. A cooperation principle is implemented by commands (from the user to the system) and information feedback (from the system to the user). Moreover, a cooperation principle takes a user's perspective with the objective to make the cooperative system easy to understand and use by a user.

The cooperation principle concerned in this paper is situated at the maneuver level, namely deliberative maneuver cooperation. This principle originates from the human supervisory control framework (Sheridan, 1992). In this principle, the cooperation is initiated by the system. The system shows its intended maneuver and plausible alternatives to the driver (information feedback), while the latter can select an alternative if he does not agree with the intention of the system (commands). “Deliberative”, a term borrowed from the domain of multi-agent system (Deliberative agent, 2014), refers to the “thinking ahead” ability exhibited by the human and the automation in this principle. Finally, this principle is consistent with SAE Level 3, i.e., the AD system executes its originally intended maneuver plan in the absence of driver's feedback and it holds the authority in the AD mode.

## 1.3. Objective

The first objective of this paper is to present a use case method for cooperation principle implementation. This use case served as a common basis for driving scenario modelling, system functional and human-machine interface (HMI) design. The use of driving simulators to support the design of AD systems has already aroused the attention from AD developers (Boer, Della Penna, Utz, Pedersen, & Sierhuis, 2015). Within this method, we demonstrate how we implemented the design of system functions and HMI in the driving simulation environment.

As the second objective, we present the evaluation of this cooperation principle through a user study. This study was oriented to test user experiences, since it was the first time to implement such an interaction principle. Hence, a main objective
of the user study was to investigate how future users will perceive the cooperation principle and HMI. What's more, this test allowed us to examine whether cooperation facilitates the management of highway merging situation for AD systems.

## 2. Use case method

Use cases are widely employed in both object-oriented software engineering and HMI design. In this paper, use cases correspond to driving situations. A main advantage of relating use cases to driving situations is that possible behaviors of an actor (the driver or the automation) can be predicted from the knowledge of a few characteristics of the outer environment (the driving situation). Accordingly, the potential user needs and interferences between the driver and the automation can be identified.

### 2.1. Use case definition

Strong interaction exists among road vehicles at highway entry sections. Constrained by the end of acceleration lane, on-ramp vehicles have to merge into the mainline. However, they should give way to those vehicles already on the mainline according to traffic regulations. This special configuration at highway entry sections leads to different interaction patterns between merging and mainline vehicles. A merging vehicle can filter in by forcing a mainline lag vehicle to decelerate. In an inverse case, a mainline vehicle may voluntarily decelerate to facilitate the merge. In congestions, it can be observed in some countries that vehicles alternate between passing and yielding near the lane closure area in a "zipper" fashion (Cassidy & Ahn, 2005).

These complicated interactions among vehicles motivate us to select highway merging management as one use case for cooperation design. In this use case, the AD ego vehicle encounters a merging vehicle at a highway entry section, as shown in Fig. 1. Multiple possible interaction patterns with the merging vehicle could result in the interference of the driver and the automation on maneuver plan (e.g., pass, yield or lane change), thus creating potential user needs to intervene.

Once the use case had been selected, a catalog of merging situations was elaborated for scenario modelling. In this catalog, possible merging situations were classified into nine groups according to the following two criteria:

* Macroscopic criterion: mainline traffic density.

* Microscopic criterion: merging behaviors.<fnref n="1" />

For the traffic density criterion, three qualitative levels are set: *fluid*, *dense* and *congested* levels. In addition, three types of merging behaviors are defined: *nominal*, *hesitant* and *forced* types. Details of behavior description and modelling will be presented in the next section.

### 2.2. Scenario modelling tool

Modelling such a highly interactive merging scenario imposes a new challenge: the scenario must be *interactive* and *reproducible* at the same time. "Interactive" implies that the surrounding vehicles should naturally adapt their behaviors to the ego AD vehicle. But the scenario should be reproducible in the sense that each participant of the user test can encounter the same kind of situation. For the chosen use case, a target merging vehicle needs to be generated such that it always meets the ego vehicle with a configurable relative gap across different test runs. In the meanwhile, its microscopic merging behavior shall be controllable.

To meet this challenge, we developed a generic scenario modelling tool prototype in this study. This tool is based on the traffic and the scenario modules of the SCANER™ studio software (That & Casas, 2011) and meets the following two requirements. First, this tool shall be re-usable for other scenarios or projects. Thus the dependency on the road network and on the scene has to be minimal. Second, it shall be fully configurable, i.e., every parameter controlling the behavior of the actors has to be accessible and modifiable from high-level scripts in the scenario module.

We identified three major functions for the scenario modelling tool: the "meeting control", the "gap control" and the "lane change control". The role of the "meeting control" function is to ensure that the target merging vehicle always meets the ego AD vehicle under configurable conditions including the meeting location, the gap between the vehicles and the speed of the target merging vehicle at the meeting location. The "gap control" function is responsible for controlling the speed of the target merging vehicle so that it keeps resting in proximity in front of the ego AD vehicle. In this way, the predefined *hesitant* behavior can be realized. The last function "lane change control" aims at controlling the decision of the target merging vehicle to merge behind or in front of the ego AD vehicle. Conditions for merging decision vary according to the merging behavior. For the *nominal* merging, only the gap between the vehicles is considered by the target merging vehicle as a decisional factor. For the *hesitant* merging, the deceleration of the ego AD vehicle, showing a cooperative attitude, triggers the decision of merging. At the proximity of the end of the merging lane, a *forced* merging decision can be taken even with smaller gaps than the threshold used by *nominal* merging decision.

<footnote n="1">¹ For the reason of simplicity, it is assumed that there is only a merging vehicle on ramp at the current stage.</footnote>
<figure><img src="image_2.png" /><figcaption>**Fig. 1.** A typical driving scene at a highway entry section.</figcaption></figure>

## 2.3. System functional design

An AD system of SAE L3 was developed in this use case. The system is partitioned into three layers: perception, maneuver planning and control layers. We took the SCANeR™ studio as the driving simulation platform. We employed radar models in SCANeR™ studio to detect traffic vehicles and to obtain their positions and speeds. A longitudinal controller (Guo, Sentouh, Popieul, Soualmi, & Haué, 2015) and a lane-keeping controller (Soualmi et al., 2014) developed in Simulink were easily integrated in the platform thanks to the APIs of SCANeR™ studio. After the functions of perception and control had been realized, maneuver planning was the focus of our design.

Maneuver planning is responsible for generating maneuver plans for vehicle controllers on the one hand and for implementing the cooperation principle on the other hand. To simplify the problem, it was assumed that the system plans maneuvers only in the longitudinal dimension. This simplification can be justified by the fact that interactions between vehicles at a merging section are mainly influenced by their longitudinal positions and dynamics.

We modelled the maneuver planning function using a hierarchical finite state machine (HFSM, Kurt & Özgüner, 2013). The HFSM has two first-level states: “intention phase” and “decision phase”, as shown in Fig. 2. The “intention phase” corresponds to the phase in which the system adapts AD vehicle’s relative position to the merging vehicle according to its intention. In “intended pass” state, the system attempts to reduce the gap with the merging vehicle by keeping the current speed or accelerating if possible. In “intended yield” state, conversely, the system increases the gap by decelerating in order to facilitate the merge of the merging vehicle. Between these two states, the system always enables the one with a lower risk. A “no intention” state is set purposely for congested traffics in which the intention to pass or yield is contextual and influenced mainly by social rules. Therefore, the system in “no intention” state keeps its initial car-following task without interacting with the merging vehicle. The “decision phase” corresponds to the phase in which the system engages an action to terminate the interaction with the merging vehicle. If the gap with the merging vehicle is sufficiently small that the merge is deemed infeasible, the system engages a pass maneuver to surpass the merging vehicle. If the merging vehicle is detected to initiate the merge, the system state transitions to the “engaged yield”. The longitudinal controller then treats the merging vehicle as the new lead vehicle and establishes a new safe gap with it.

The cooperation occurs in the “intention phase”. The principle was implemented by third-level states named “alternative available” (represented by green circles in Fig. 2) and transitions from them (green arrows). When the risk of an alternative maneuver is below a threshold, the corresponding “alternative available” sub-state becomes active. In this case, the driver can trigger the transition to the destination state. As a result, the original alternative becomes the system’s intended maneuver. For instance, the “intended pass” is active and “yield” is an alternative maneuver. If the risk of “yield” (deceleration) is smaller than the threshold, the “yield available” sub-state becomes active and the driver’s input can hence trigger the transition from “intended pass” to “intended yield”. In this principle, the system assumes the final authority. The fact that the driver asks an available alternative does not mean this maneuver will be engaged by the system definitively. This kind of situations may occur in highly dynamic situations where an available maneuver may no more be feasible for the control layer one instant later.

## 2.4. HMI design

In addition to the interaction logic at the system functional level, HMI is another key factor enabling efficient cooperation between the driver and the system. The design decisions are formulated as the following three HMI principles:

1. Showing the driving context. This principle is consistent with the concept of common frame of reference proposed by Hoc (2001). The driving context serves as a common reference on which the driver and the AD system share their intentions. This principle was implemented by a representation of merging scene in a windshield Head-Up Display (HUD,
<figure><img src="image_3.png" /><figcaption>**Fig. 2.** HFSM-based maneuver planning for highway merging context.</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>**Fig. 3.** Designed HMI for deliberative maneuver cooperation principle.</figcaption></figure>

Fig. 3-left) and a yellow semi-transparent rectangle tracking the target merging vehicle (Fig. 3-upper right). This yellow<fnref n="2" /> rectangle is the simulation of augmented reality (AR). The appearance and disappearance of these HMI elements are consistent with the entry and the exit of the HFSM "highway merging context".

(2) *Showing the intention and available alternative.* First, we used triangle to symbolize maneuver, with triangle(s) forwards for “pass” and backwards for “yield”. Then we designed color codes to distinguish maneuver states. Three blue triangles represent an “intended/engaged” maneuver, whereas a single green one represents an “available” alternative. Intention and alternative symbols are shown within the representation of merging scene in the HUD (Fig. 3-left).

(3) Providing a way for the user to choose an alternative. This principle was implemented by two capacitive backlit buttons (Fig. 3-lower right). The up button means “pass” and the down button “yield”. As long as a maneuver is available, the corresponding button twinkles in green and remains active. The press on an active button will trigger a transition to the alternative in the HFSM (Fig. 2). As for an acknowledgement, the pressed button will become blue (lasting 2 s). If the user presses when the green light is off, this button will temporally become red (2 s) to signify a refusal.

Fig. 3 shows a synthesis of the HMI prototypes installed on the driving simulator.

<footnote n="2">² For interpretation of color in Figs. 2 and 3, the reader is referred to the web version of this article.</footnote>
# 3. User study

## 3.1. Objectives

Through a user study based on the driving simulator, we intended to evaluate the following three aspects of the proposed cooperation principle:

* O1: to evaluate the intuitiveness of the proposed cooperation principle and its HMI;

* O2: to assess the user's performance to cooperate with the system through the HMI;

* O3: to assess the effects of cooperation on the interaction between the AD vehicle and the merging vehicle.

## 3.2. Participants

Twenty-two participants, average age of 41.3 years (ranging from 24 to 61) took part in the experiment. They were employees of Renault Technocentre and IRT SystemX. They have a driving license of 22.45 years on average and drove on average 5.8 days/week.

## 3.3. Test setup

### 3.3.1. Driving simulator

The driving simulator "Dr SiHMI" of the IRT SystemX (Dr SiHMI platform, n.d.) was used in the experiment. The simulator uses SCANeR™ studio as a simulation platform in which are integrated different modules such as the AD system, the scenario modelling tool and HMI controllers. The visual system of the simulator is composed of three projectors and a curved screen that can cover a field of view in horizontal 170° and vertical 40°. The simulator cockpit is modular and thus facilitates the prototyping and integration of HMI solutions.

### 3.3.2. Test scenarios

Based on the catalog of merging situations, we have modelled two baseline test scenarios: merging management in a fluid mainline (referred to as Fluid) and in a congested mainline (referred to as Congestion). According to the modal logics in the HFSM, the AD system has its intended maneuver ("intended pass" or "intended yield") in Fluid, while in Congestion the AD system is in the "no intention" state. This implies that the subject has an alternative to choose in Fluid but two alternatives in Congestion. In both scenarios, the AD mode is active by default so that each subject can totally be disengaged from vehicle control. Fig. 4 illustrates two scans of driving scene in Fluid and Congestion.

For this type of scenario in which the ego vehicle needs to interact with other traffic vehicles, it is of interest to vary behaviors of traffic vehicles. Otherwise, a same merging behavior across all the test runs could not only decrease the immersion of subjects but also lead to strong memory effects that may bias the assessment of cooperation performance. As a solution, we used a random variable generator to generate random parameters for the scenario modelling tool. In this way, based on a baseline scenario (used as a template), different test scenarios can be generated automatically.

## 3.4. Procedure and instructions

Prior to the test drive, the subjects were familiar with the AD system and the driving simulator in a training drive. The test drive was divided into two phases. The objective of the first phase (PH1) was to evaluate the intuitiveness of the HMI and the cooperation principle. In order to incite subjects to cooperate with the system, we modelled a hesitant merging vehicle that keeps oscillating ahead. In Fluid, facing this problematic vehicle, the system adopted a conservative maneuver plan—"yield" while leaving the driver a possibility to change to "pass". In this phase, each subject participated in three test runs. Each run consisted of a Fluid and a Congestion scenario. The first run served as a reference in which none of HMI was activated. Before this run, we explained briefly the functionality of the system (full automatic control) and asked the subject to observe the driving scene as a new user. In the second run, we added HMI displays (HUD and the yellow rectangle). We informed the subject of the new added HMI displays but did not explain their meaning. In the third run, we activated the button command interface in addition to HMI displays. We simply indicated the driver that he had a new interface allowing him to change the intention of the system, however, without any instruction on how to use the buttons. As illustrated by the schema in Fig. 5, the second run was set to assess how the subjects understood the HMI displays whilst the third run was dedicated to evaluate their comprehension of the cooperation principle. At the end of PH1, the subjects were asked to fill the prepared questionnaires.

The second phase (PH2) aimed to test subject's performance on cooperation. Therefore, HMI displays and the buttons were both active. Each subject participated in four sequences of Fluid and four sequences of Congestion in a random order. While test scenarios were varied across test sequences within subjects, they remained the same as a whole across subjects.<fnref n="3" />

<footnote n="3">³ In such a way, a subject had four different Fluid scenarios and four different Congestion scenarios. To ensure the consistency of the test, the total eight scenarios remained the same for each subject. This was achieved by using the same random seed for each subject's test (The MathWorks, Inc., n.d.).</footnote>
<figure><img src="image_5.png" /><figcaption>**Fig. 4.** Baseline scenarios: Fluid (left) and Congestion (right).</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>**Fig. 5.** Schema of the procedure in PH1.</figcaption></figure>

Before the start of PH2, we explained the HMI displays and the correct way to use buttons. Furthermore, each subject was instructed to use these buttons according to their needs, i.e., they were not obliged to use them.

### 3.5. Data collection and metrics

Both quantitative and qualitative data were collected. The quantitative data were collected through data log from the simulator and a questionnaire. To obtain the qualitative data, we employed “thinking aloud” technique and conducted interviews. Results were principally derived from the quantitative analysis, while qualitative data were used to give complementary information.

To evaluate how subjects perceived the cooperation principle (01) and their performance on cooperation (02), we proposed two metrics concerning button use. According to the button state when it was pressed, we defined three types of button pressing:

* Pressing Available Alternative maneuver (PAA): a subject pressed the button corresponding to an available alternative. It characterizes a good use.

* Pressing Intended maneuver (PI): a subject pressed the button corresponding to system's intended maneuver. It can be interpreted that the subject shared the same intention with the system.

* Pressing Pre-Intention (PPI): a subject pressed a button before the system showed its intended maneuver (in Fluid) or available alternatives (in Congestion).

The first metric of button use was the distribution of button press types which indicates the successful rate (represented by the ratio of PAA).

The second metric was related to the time on button use. Within each record of a test run, by setting the time when an alternative became available as the origin ($t = 0$), we computed at which time a subject pressed a button for the first time. The statistical information on the time of the first button presses was exploited in different ways in PH1 and PH2. In PH1, this information allowed us to assess whether a subject was aware of the time window of an alternative's availability. In PH2, given that subjects had known the cooperation principle through our explanations, the average time they needed to press the button is a metric of their efficiency on cooperation.

Concerning 03, we queried the data within a time span in which the AD vehicle was interacting with the merging vehicle. The time span starts from the moment when the merging vehicle entered in a zone proximate to the AD vehicle until it quitted this zone either by being surpassed by the AD vehicle or by merging into the mainline. This zone is defined by [-l, 2v] on the acceleration lane, where l is the length of AD car body and the time headway is 2 s. The selected metrics within this time span are shown in Table 1. The interaction duration is characterized by $t_{inter}$. Due to the limited range available for the merg-

<table><caption>**Table 1**

Metrics used to evaluate the driving performance.</caption><thead><tr><td>Metrics</td><td>Description</td></tr></thead><tbody><tr><td>t<sub>inter</sub></td><td>Length of the time span in which the AD vehicle was interacting with the merging vehicle</td></tr><tr><td>v<sub>fin</sub>/v<sub>0</sub></td><td>Ratio of the end speed of the AD vehicle to the initial speed</td></tr><tr><td>Δa</td><td>Difference between the maximum and the minimum accelerations of the AD vehicle</td></tr></tbody></table>
ing vehicle on the acceleration lane, the longer $t_{inter}$ is, the more urgent the merge of the merging vehicle is and hence the more critical the situation is for the AD vehicle. The speed ratio reflects the ability of the AD system to maintain its initial speed during the interaction. At last, the acceleration variation is a metric for comfort.

# 4. Results

## 4.1. Intuitiveness of the HMI and cooperation principle

Subjective evaluation of the HMI intuitiveness was made based on the results of the questionnaire (see Fig. 6). Among all the HMI elements, the simulated AR, the yellow rectangle tracking the merging vehicle, has received the highest rating (M = 3.68, SD = 0.57) in terms of the ease of understanding. The representation of merging context in HUD-HMI was rated as "rather easy" (M = 2.95, SD = 0.90). The rather high SD indicates that the answers were varied. According to the verbal protocols, several subjects were confused with the meaning of arrows. They interpreted the blue arrows as actual accelerations of the ego vehicle. Due to the small size of the pictogram limited by the HUD projection area, 12 subjects out of 22 reported unaware of the green arrow. As for the button command, while the button function were better understood by subjects (M = 3.36, SD = 0.66), the color code of the button was worse rated (M = 2.73, SD = 1.03). Owing to the position of the button interface, several subjects did not associate the twinkling green state of a button with the appearance of the green arrow in the HUD-HMI. Consequently, those subjects were confused with the meaning of the green color.

To assess how users perceived the cooperation principle, we first examined the distribution of the types of button pressing in PH1. As shown in Fig. 7, PAA accounted for only 21% of the presses in Fluid. It suggests that subjects did not well understand the logic to change the intention of the system. Rather, the high ratio of PI (50%) reveals that subjects expected that pressing system's intention would have effect. Compared to Fluid, the logic in Congestion was better perceived by the subjects given the 53% for PAA.

The distribution of the time of the first button presses in PH1 is represented in form of the histogram in Fig. 8. In each histogram, the presses falling within the range of [-SD, SD] were counted and shown. The time window of the availability of the alternative was underlined in green and its initial time served as the origin of time axis. The average time of the first

<figure><img src="image_7.png" /><figcaption>**Fig. 6.** Subjective evaluation of the intuitiveness of the HMI.</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>**Fig. 7.** Results on button press distribution in PH1.</figcaption></figure>
<figure><img src="image_9.png" /><figcaption>**Fig. 8.** Histograms of the time of the first button press in PH1 with the interval of 1 s (left: Fluid; right: Congestion).</figcaption></figure>

<figure><img src="image_10.png" /><figcaption>**Fig. 9.** Results on button press distribution in PH2.</figcaption></figure>

button press as well as the SD were indicated too. It can be clearly seen that the button presses in Fluid were more dispersed, whilst the presses in Congestion were more concentrated to the average value. In view of the one third presses falling out of the time window of the "available pass" in Fluid, it can be inferred that subjects did not well perceive this time windows in Fluid.

## 4.2. Cooperation performance

The overall performance of the subjects on button use was improved in PH2, given the 62% and the 86% for PAA in Fluid and Congestion respectively in Fig. 9. However, PI in Fluid still occupied a non-negligible part (27%) compared with in Congestion. It indicates that several subjects had tendency to confirm the system's intention, even though they were informed of the correct button use—to press an available alternative. With a more important part of PAA, the performance in Congestion was better than in Fluid.

Fig. 10 shows that the distributions of the time of the first button press in both scenarios of PH2 were more concentrated to the average than in PH1. What's more, most button presses fell into the time windows of the alternative's availability. It

<figure><img src="image_11.png" /><figcaption>**Fig. 10.** Histograms of the time of the first button press in PH2 with the interval of 1 s (left: Fluid; right: Congestion).</figcaption></figure>
<table><caption>**Table 2**
Means and standard deviations for the metrics on the performance of the AD system in PH2.</caption><thead><tr><th rowspan="2">Source</th><th rowspan="2">Number<br/>n</th><th colspan="4">Metrics</th></tr><tr><th colspan="2"><i>t</i><sub>inter</sub> (s)<br/>M</th><th colspan="2"><i>v</i><sub>fin</sub>/<i>v</i><sub>0</sub> (%)<br/>M</th><th colspan="2">Δ<i>a</i> (m/s<sup>2</sup>)<br/>M</th></tr><tr><th></th><th></th><th></th><th>SD</th><th></th><th></th><th>SD</th><th>SD</th></tr></thead><tbody><tr><td>Fluid – without cooperation</td><td>9</td><td>20.87</td><td>5.52</td><td>69.26</td><td>5.01</td><td>2.05</td><td>0.77</td></tr><tr><td>Fluid – cooperation</td><td>41</td><td>16.04</td><td>3.00</td><td>98.18</td><td>9.04</td><td>1.40</td><td>0.64</td></tr><tr><td>Congestion – without cooperation</td><td>9</td><td>19.63</td><td>7.00</td><td>78.70</td><td>15.46</td><td>0.88</td><td>0.29</td></tr><tr><td>Congestion – cooperation</td><td>59</td><td>7.17</td><td>4.04</td><td>90.47</td><td>20.45</td><td>1.28</td><td>0.29</td></tr></tbody></table>

<table><caption>**Table 3**
$t$-Test for equality of means (independent samples).</caption><thead><tr><th>Scenario</th><th>Metrics</th><th>t</th><th>df</th><th>p<sup>a</sup></th><th>Mean differences</th></tr></thead><tbody><tr><td rowspan="3">Fluid</td><td><i>t</i><sub>inter</sub> (s)</td><td>-2.54</td><td>9</td><td>0.03</td><td>-4.83</td></tr><tr><td><i>v</i><sub>fin</sub>/<i>v</i><sub>0</sub> (%)</td><td>13.23</td><td>21</td><td>1.19E-11</td><td>28.92</td></tr><tr><td>Δ<i>a</i> (m/s<sup>2</sup>)</td><td>-2.38</td><td>11</td><td>0.04</td><td>-0.65</td></tr><tr><td rowspan="3">Congestion</td><td><i>t</i><sub>inter</sub> (s)</td><td>-5.21</td><td>9</td><td>5.56E-04</td><td>-12.46</td></tr><tr><td><i>v</i><sub>fin</sub>/<i>v</i><sub>0</sub> (%)</td><td>2.03</td><td>13</td><td>0.06</td><td>11.77</td></tr><tr><td>Δ<i>a</i> (m/s<sup>2</sup>)</td><td>3.85</td><td>11</td><td>2.68E-3</td><td>0.40</td></tr></tbody></table>

means that the subjects perceived better this time window and hence understood better the cooperation principle. Coincidentally, the averages of both two scenarios were equal to 3.87 s.

### 4.3. The effects of cooperation on the interaction with the merging vehicle

Table 2 shows the descriptive statistics of the performance metrics of the AD system in interaction with the merging vehicle in PH2. The sample of system's performance without cooperation in either Fluid or Congestion consists of nine randomly generated scenarios; the sample of system's performance with cooperation (41 in Fluid and 59 in Congestion) is made of the test runs with effective cooperation, i.e., with a PAA, performed in these nine scenarios of Fluid or Congestion. An independent-samples *t*-test was conducted to study the effect of cooperation on the system's performance (see Table 3). In Fluid, all the performance metrics yielded better results with cooperation than without cooperation, manifested by shorter interaction time (*t*<sub>inter</sub>, *p* < 0.03), higher speed-keeping ability (*v*<sub>fin</sub>/<i>v</i><sub>0</sub>, *p* < 1.19E-11) and smaller acceleration variation (Δ<i>a</i>, *p* < 0.04). In Congestion, *t*<sub>inter</sub> with cooperation was significantly reduced (*p* < 5.56E-04). The ratio of the final speed to the initial speed (*v*<sub>fin</sub>/<i>v</i><sub>0</sub>) with cooperation was higher, however, without significant difference (*p* < 0.06). The average acceleration variation was stronger with cooperation in Congestion. It can be explained by the fact that the AD vehicle manifested its intention to the merging vehicle by small amounts of acceleration as a result of the driver's intervention.

## 5. Summary and discussion

We summarize the main results from the user study and give a discussion after each result.

Concerning the HMI, the simulated AR in the driving scene—the yellow rectangle that tracks the merging vehicle was considered as the easiest to understand among all the HMI elements. Besides other factors that could influence the intuitiveness of an HMI like graphical grammar, we want to highlight the role of the perceptibility, considering that the HMI is dispatched into three areas in the field of view of the driver in the current configuration. According to our observations, most of the subjects monitored the merging vehicle located on the right part of the scene. The verbal protocol suggested that they tended to infer the system's intention from the change of the distance to the merging vehicle. Consequently, the HUD-HMI (in the center of the scene) and the back-lit buttons (near the gear shift lever) were situated in the peripheral vision of those subjects. The low perceptibility of the HUD-HMI and the button interface may influence subject's average level of understanding on the meanings of their contents. Furthermore, subjects needed to switch attention between the merging vehicle and the button command (to check whether a button became green). It could increase their cognitive demand. Thus, it may be beneficial for the usability of HMI to display the essential information in the driving scene, especially in the zone the driver attends to. In our case, it is of interest to show the system's intention and the button's state near the merging vehicle. To achieve this goal, the AR technology is necessary.

With regard to the cooperation principle, the experiment results indicate that the logic in Congestion (being possible to choose any of the alternatives) was easier to understand than that in Fluid (being possible to choose an alternative other than the system's intention). The average performance of the subjects on button use in PH2 confirms this conclusion too. Therefore, it seems better to provide a way for the driver just to indicate his intention, regardless of the intention of the system. In

<footnote><sup>a</sup> Two tails with a 5% alpha level.</footnote>
case that the driver and the system share the same intention, the system can take the input of the driver as a confirmation. This modification of the cooperation principle will be tested in the future works. As to the temporal aspect of the cooperation performance, the average time needed for the first button press after this button became available was about 3.9 s. This implies that the system needs to offer a time window long enough (at least longer than 3.9 s under the current HMI configuration) so that the driver has enough time to reason and to give his choice. Of course, time window length depends on the behavior of the merging vehicle, but it can be extended if the AD system is able to predict the situation's evolution with a longer look ahead. Therefore, this imposes high requirements on the situation assessment function of the AD system. On the other hand, how to design an intuitive HMI to reduce the driver's reaction time, especially when he is engaging in non-driving tasks, remains another research question of interest.

The performance metrics of the AD vehicle in interaction with the merging vehicle suggest that the intervention from the driver could be beneficial to the AD system in terms of managing merging situations. Nevertheless, it is difficult to generalize the same conclusion to the real world for lack of the knowledge of the validity of these simulated scenarios. As argued by Boer et al. (2015), to evaluate the interaction of the AD vehicle with other road users constitutes one important usage of driving simulators in the AD vehicle design. To achieve this goal, it needs to ensure the realism of other road user's behaviors. One plausible solution is to integrate state-of-the-art microscopic traffic simulation models into a scenario modelling tool as the one developed in this study. In this way, one could make traffic vehicles interactable vis-à-vis the ego AD vehicle while guaranteeing the realism of their behaviors, because those models are already validated by naturalistic data.

## 6. Conclusion

This paper presented the implementation and evaluation of a driver-vehicle cooperation principle. This principle was implemented within a use case: highway merging management. Based on this use case, we designed a scenario modelling tool, a maneuver planning function and a set of HMIs. In a user study on driving simulator, we evaluated the intuitiveness of the cooperation principle, user's performance on cooperation and the effects of the driver-vehicle cooperation on AD vehicle's interaction with the merging vehicle. Test results show the interest of using AR to enhance the perceptibility of HMI. In addition, we discussed user's perception of the proposed cooperation principle and the positive effect of driver-vehicle cooperation on AD vehicle's performance. We also pointed out some future directions to improve the cooperation principle and HMI design.

At last, we would like to highlight the strengths of driving simulation in interaction design for AD systems. Without using driving simulation, it would be quite complicated to prepare the target scenarios in a real environment. A driving simulation environment offers the flexibility to prototype new types of HMI such as the simulation of AR. To design new interaction principles for the future system, driving simulation also renders driver-in-loop tests possible at an early design stage.

## Acknowledgements

This work is part of the project "Localization & Augmented Reality", which is supported by the French government as part of PIA (French acronym for Program of Future Investments) within the Technological Research Institute SystemX.

## References

* Abbink, D. A., & Mulder, M. (2010). Neuromuscular analysis as a guideline in designing shared control. In M. Hosseini (Ed.), *Advances in haptics*. InTech.

* Anderson, S. J., Karumanchi, S. B., Iagnemma, K., & Walker, J. M. (2013). The intelligent copilot: A constraint-based approach to shared-adaptive control of ground vehicles. *IEEE Intelligent Transportation Systems Magazine*, 5(2), 45–54.

* Blanco, M., Atwood, J., Vasquez, H., Trimble, T., Fitchett, V., Radlbeck, J., Ellipsis Sheldon, R. (2015). Human factors evaluation of Level 2 and Level 3 automated driving concepts (No. DOT HS 812 182) (p. 320).

* Boer, E. R., Della Penna, M., Utz, H., Pedersen, L., & Sierhuis, M. (2015). The role of driving simulators in developing and evaluating autonomous vehicles. In *Proceedings of DSC 2015 Europe driving simulation conference & exhibition. Tübingen, Germany* (pp. 3–10).

* Cassidy, M., & Ahn, S. (2005). Driver turn-taking behavior in congested freeway merges. *Transportation Research Record: Journal of the Transportation Research Board*, 1934, 140–147.

* Deliberative agent (2014). In Wikipedia. <https://en.wikipedia.org/w/index.php?title=Deliberative_agent&oldid=638039836> [02.11.2016].

* Dr SiHMI platform. (n.d.). <http://www.irt-systemx.fr/en/valorisation/plateformes/plateforme-dr-sihmi/> [18.12.2016].

* Färber, B. (2016). Communication and communication problems between autonomous vehicles and human drivers. In M. Maurer, J. C. Gerdes, B. Lenz, & H. Winner (Eds.), *Autonomous driving* (pp. 125–144). Berlin Heidelberg: Springer.

* Flemisch, F. O., Bengler, K., Bubb, H., Winner, H., & Bruder, R. (2014). Towards cooperative guidance and control of highly automated vehicles: H-Mode and Conduct-by-Wire. *Ergonomics*, 57(3), 343–360.

* Gold, C., Damböck, D., Lorenz, L., & Bengler, K. (2013). "Take over!" How long does it take to get the driver back into the loop? *Proceedings of the human factors and ergonomics society annual meeting*, 57(1), 1938–1942.

* Guo, C., Sentouh, C., Soualmi, B., Haue, J.-B., & Popieul, J.-C. (2015). A hierarchical cooperative control architecture for automated driving systems. In *Presented at the 22nd ITS World Congress, Bordeaux, France*.

* Guo, C., Sentouh, C., Popieul, J.-C., Soualmi, B., & Haué, J.-B. (2015). Shared control framework applied for vehicle longitudinal control in highway merging scenarios. In *International Conference on Systems, Man, and Cybernetics (SMC)* (pp. 3098–3103). IEEE.

* Hoc, J.-M. (2001). Towards a cognitive approach to human-machine cooperation in dynamic situations. *International Journal of Human-Computer Studies*, 54 (4), 509–540.

* Hoc, J.-M., Young, M. S., & Blosseville, J.-M. (2009). Cooperation between drivers and automation: Implications for safety. *Theoretical Issues in Ergonomics Science*, 10(2), 135–160.

* Hoeger, R., Amditis, A., Kunert, M., Hoess, A., Flemish, F., Krueger, H.-P., Ellipsis Pagle, K. (2008). Highly automated vehicles for intelligent transport: Have-It approach. *Presented at the 15th World Congress on Intelligent Transport Systems and ITS America's 2008 Annual Meeting*.
Hörwick, M., & Siedersberger, K. H. (2010). Strategy and architecture of a safety concept for fully automatic and autonomous driving assistance systems. In *2010 IEEE Intelligent Vehicles Symposium* (pp. 955–960).

Kurt, A., & Özgüner, Ü. (2013). Hierarchical finite state machines for autonomous mobile systems. *Control Engineering Practice*, 21(2), 184–194.

Lorenz, L., Kerschbaum, P., & Schumann, J. (2014). Designing take over scenarios for automated driving: How does augmented reality support the driver to get back into the loop? *Proceedings of the Human Factors and Ergonomics Society Annual Meeting*, 58(1), 1681–1685.

Marinik, A., Bishop, R., Fitchett, V. L., Morgan, J. F., Trimble, T. E., & Blanco, M. (2014). Human factors evaluation of Level 2 and Level 3 automated driving concepts: Concepts of operation (Task Report No. DOT HS 812 044). Blacksburg, VA: Virginia Tech Transportation Institute.

Markoff, J. (2010, October 9). Google cars drive themselves, in traffic. *The New York Times*. <http://www.nytimes.com/2010/10/10/science/10google.html> [24.12.2016].

Michon, J. A. (1985). A critical view of driver behavior models: What do we know, what should we do? In L. Evans & R. C. Schwing (Eds.), *Human behavior and traffic safety* (pp. 485–524). Springer US.

Mörtl, A., Lawitzky, M., Kucukyilmaz, A., Sezgin, M., Basdogan, C., & Hirche, S. (2012). The role of roles: Physical cooperation between humans and robots. *The International Journal of Robotics Research*, 31(13), 1656–1674.

Pacaux-Lemoine, M. P., & Debernard, S. (2002). Common work space for human–machine cooperation in air traffic control. *Control Engineering Practice*, 10 (5), 571–576.

SAE (2014). Taxonomy and definitions for terms related to on-road motor vehicle automated driving systems (Standard No. J3016_201401) (p. 12). SAE International.

Sentouh, C., Popieul, J.-C., Debernard, S., & Boverie, S. (2014). Human-machine interaction in automated vehicle: The ABV Project. *IFAC Proceedings Volumes*, 47(3), 6344–6349.

Sheridan, T. B. (1992). *Telerobotics, automation, and human supervisory control*. Cambridge, MA, USA: MIT Press.

Sheridan, T. B. (1995). Teleoperation, telerobotics and telepresence: A progress report. *Control Engineering Practice*, 3(2), 205–214.

Soualmi, B., Sentouh, C., Popieul, J. C., & Debernard, S. (2014). Automation-driver cooperative driving in presence of undetected obstacles. *Control Engineering Practice*, 24, 106–119.

That, T. N., & Casas, J. (2011). An integrated framework combining a traffic simulator and a driving simulator. *Procedia - Social and Behavioral Sciences*, 20, 648–655.

The MathWorks, Inc. (n.d.). Generate random numbers that are repeatable - MATLAB & Simulink. <https://www.mathworks.com/help/matlab/math/generate-random-numbers-that-are-repeatable.html> [24.12.2016].

Urmson, C., Anhalt, J., Bagnell, D., Baker, C., Bittner, R., Clark, M. N., ... Ferguson, D. (2009). Autonomous driving in urban environments: Boss and the urban challenge. In M. Buehler, K. Lagnemma, & S. Singh (Eds.), *The DARPA urban challenge* (pp. 1–59). Berlin Heidelberg: Springer.

Vanholme, B., Gruyer, D., Lusetti, B., Glaser, S., & Mammar, S. (2013). Highly automated driving on highways based on legal safety. *IEEE Transactions on Intelligent Transportation Systems*, 14(1), 333–347.

Walch, M., Lange, K., Baumann, M., & Weber, M. (2015). Autonomous driving: Investigating the feasibility of car-driver handover assistance. In *Proceedings of the 7th international conference on automotive user interfaces and interactive vehicular applications* (pp. 11–18). New York, NY, USA: ACM.