# Evaluation of Takeover Time Performance of Drivers in Partially Autonomous Vehicles Using a Wearable Sensor

Authors: Daghan Dogan; Seta Bogosyan; Tankut Acarman
Journal: Journal of Sensors
 (2022)

---

## Abstract

Advanced driver assistance systems and conditional automation (SAE level L3) bring full of challenges, central to the human role in such automated driving and are the transitions from automated to manual driving mode: driver monitoring systems should be able to adapt to the detected human state adjusting time given for takeover request. In this study, the human driver’s control authority during the transition from the automated system is evaluated by using a large set of sensors, including the galvanic skin response sensor, the speed, and the current sensor of the electric vehicle. The Local Outlier Factor (LOF) method detects the driver’s readiness to take control during the control authority transition transient stage. Eighteen drivers (8 females and 10 males) between 24 and 65 years of age participate in the experimental evaluation and data collection campaign. Takeover request time analysis evaluates each driver and three driver categories, such as experienced, semiexperienced, and inexperienced, and the questionnaire validates the takeover request time analysis. Galvanic skin response measurements and the LOF approach illustrate successful detection of the driver’s status about his/her readiness to take over and regulate the driving task when the transition of driving authority from the automated system is requested in a real road driving scenario. The takeover request time is personalized for each driver, which may improve the current conditional automated driving technologies’ penetration and acceptance.


---

## Body

## 1. Introduction

The automation level in vehicles is categorized with the SAE J3016 level of driving automation [1]. From a top-down approach, level 5 means a fully autonomous vehicle, and the driver is in an out-of-the-driving loop, and he/she does not have any dynamic driving tasks. At level 4, the driver does not intervene in most driving situations, except for an automatic system failure. Level 4 driving is deployed within a particular area while not exceeding given speed limit, i.e., geofencing is adopted, the driver is alerted to intervene when the automated system fails, and it is toward for the usage in driverless public vehicles, such as taxis and busses. Level 3 is a conditional automation, and control authority transition to the driver is required once the vehicle is out of its operational design domain in which the automated driving system is designed to operate properly. Dynamic driving tasks such as road perception, trajectory, and motion need to be monitored by the driver in order to override the system when it is unable to avoid possible hazards and accidents. The alertness, attentiveness, or stress level, which may be perturbed by no driving-related tasks (NDRTs), need to be tracked by a driver monitoring system to perform manoeuvring tasks during driving control authority transition transients. Advanced driver assistance systems (ADAS) belong to level 2 automation which constitutes lateral and longitudinal control of the vehicle, and we expect the driver to control manoeuvring tasks for any time. A single driving automation system is level 1, and the driver controls the steering when the vehicle is regulated by a system in a longitudinal direction such as an adaptive cruise control system or the driver command traction/braking when the vehicle trajectory is regulated in its lane by a lane-keeping system. Finally, level 0 means full manual control, but some systems may compensate driver’s command without automating the driving task, such as an emergency brake system.Central to the human role in such highly automated driving is the transition from automated to manual driving mode and vice versa. We should ensure that the driver takes over the driving task safely, as long as there are situations where automation cannot handle it. Conditional automation L3 level enforces transition between the driver and the automated system, but this requires monitoring control authority level of the human driver. A human driver can be subject to several factors, such as NDRTs that may cause stress and distraction and weaken the readiness of the driver to command the vehicle subject to a possible safety-critical and challenging manoeuvring task. In addition, the takeover time may depend on the driver’s experience, skill, gender, and age. On the other hand, we can improve the ability of a driver to operate a vehicle system when certain parameters in the vehicle control are adjusted according to the driver’s characteristics [2]. In this study, we aim to improve the driver’s ability when responding to a takeover request generated by an automated driving system subject to its failure to perform complicated manoeuvring tasks in a short period. We personalize and adapt the takeover request (TOR) time of each driver as the safety-critical and time-critical parameter of control authority transient according to the metrics such as time-to-collision (TTC) and distance-to-collision (DTC). Finally, each driver’s TOR time is adapted subject to the driver’s perception and reaction time, assuring an acceptable control authority transition between the automated system and the driver.The focus of this study is to determine at which takeover time the driver takes over vehicle control as a human factor. For this purpose, an electric vehicle with longitudinal autonomy is designed and equipped with sensors. In a campus area, traffic scenarios and driving tasks are examined. The tasks in the experiment are designed to accommodate scenarios of varying complexity, including crossing an intersection, detecting pedestrians oncoming to the road, and following a sudden stopping vehicle. The galvanic skin response sensor (GSR) is used to measure the driver’s stress level, while the speed and the current sensor of the electric vehicle for traction or regenerative braking measures the driving command of the driver. TTC presents the potential hazard or accidents subject to manoeuvring tasks during control authority transition. The LOF method detects the driver’s anomaly. Eighteen drivers (8 females and 10 males) between 24 and 65 years of age participate in the experimental evaluation and data collection campaign. Measurement-based statistical analysis of driver’s status about being capable of commanding the driving task during control authority transition from an automated driving system is tested and analysed subject to challenging driving scenarios in a particular area while not exceeding given speed limit; the driver is alerted to intervene when the automated system is about to fail. Takeover request time is evaluated for each driver and driver categories such as experienced, semiexperienced, and inexperienced and validated by a questionnaire.Four different takeover times (0 s, 2 s, 4 s, and 6 s) were applied in 5 different traffic scenarios and compared. The results illustrate the most appropriate takeover time, because short takeover time leads to an unprepared transition despite providing distance from the critical situation. We include primary and secondary (NDRTs) tasks in traffic scenarios and examine four different TOR times (0 s, 2 s, 4 s, and 6 s). The design of TOR time on a basis of realistic, comprehensive, and various scenario analyses captured by a large set of sensors, including a wearable sensor (GSR), is presented.This research is focused on detecting and extracting the driver’s takeover reaction time (TOT) and TOR time of the automated system for control authority transition compliant with the driver’s takeover perception and reaction time interval. This study contributes to the detection of the authority takeover period and contributes to the safe authority transition processes and studies.This study is presented as follows: after the introduction, the design methodology of the study and experimental design are presented and detailed. Then, the results are discussed, and finally, the conclusions and further works are presented.


### 1.1. Related Studies

Definition of driver’s risk, factors contributing to driving errors, has attracted a lot of attention to predict driving safety. In [3], the driver’s risk of making a road accident is categorized as low, medium, and high; driving behaviour is detected as safe or unsafe by evaluating the gap between the driver’s perception of driving style and measurements that are accelerations on both longitudinal and lateral direction and speed. They conducted a survey on five drivers who drove 1000 km across seven different paths and analysed the perception of driving style versus logged measurements. Definition of driving risk is complicated and subject to several variables. Potential risk factors affecting the driver were analysed by using the naturalistic driving data constituted by 10,800 six-second trip segments from the second Strategic Highway Research Program (SHRP2) [4, 5]. Differences in driver behaviour and surprise, visually distracting secondary tasks, have been detected as the main factors contributing to traffic violations and driving errors. In [6], driving risk in a driving scenario and the risky situation are detected by weighting three traffic safety experts’ knowledge. Driving sessions are performed by the same driver in a realistic truck simulator, and experts’ opinions are combined to label risk describing 18 risky driving situations. In [2], the control authority transition (CAT) has been introduced to avoid a possible collision. Driver’s operating characteristics are assessed while monitoring the driver’s intents, perceiving possible hazards and situations, and the control is modified to assure safe manoeuvring tasks performed by the driver as the main control authority. Considering L3 level conditional automation, TOT of the driver and TOR time plays a crucial role during the CAT for the driver to perform the manoeuvring tasks. We plot the time sequence in Figure 1. TOT defines the time between the takeover request and the takeover action [7], and TOR time defines the driving time between the takeover request and the critical situation [8].Figure 1
Takeover procedure in L3 conditional automated driving.In [8], a design methodology for the TOR time is presented. Although all drivers have the same information, the potential difference exists between individual reaction abilities and the driver characteristics. Reaction time is considered and human-in-the-loop PC-based simulation experiments are conducted to test and evaluate four given TORs to perform conditional automation. A performance-based methodology is presented, and a conditional probability of successful and unnecessary alert in the condition of a collision event or no alert issue is computed. Thirty drivers having a valid driver’s license participated in the simulator experiments and reaction time to the TOR, maximum acceleration, average lane departure distance, the number of collisions, and the average heart rate during TOR constitutes the quantitative results for analysis purposes. The questionnaire is used to collect participants’ acceptance. Statistically significant differences between the four TOR thresholds are detected for individual drivers. Dependency on the accuracy of human, vehicle, and environmental models of this model-based methodology needs to be improved by extensive data and real experimental study. In [9], four TOR-related critical situations are presented; stationary obstacle, slow vehicle, exit highway, and turning right on four-way uncontrolled intersection are the simulated scenarios. Criticality is varied by increasing the speed and traffic and adding an approaching vehicle or a pedestrian crossing intersection. 33 male and 3 female drivers participated, and they give the TOR 7 seconds to the drivers. A generic model in terms of several variables is assumed to give reaction time.In [10], the literature for TOR requests in L3 conditional automated driving is reviewed. Traffic situation, road condition, and transition system from automated to manual driving system are considered being objective complexity, and NDRTs, age, trust, HMI, and situational awareness are considered being subjective complexity affecting the TOR of the individual driver. Metrics are reviewed as well. Lateral and longitudinal metrics, including TTC, the number of actual and successful manoeuvring task to avoid obstacle either by braking or by performing a lane change, cognitive workload, trust, and comfort is presented as the former metrics. Real-world tests are presented as research gaps to be filled to improve the TOR transient procedure.In [11], HMI design for control authority transition and strategies are studied in a simulator of longitudinal autonomous driving with 49 experienced drivers. During a commuting period of 30 minutes, drivers take secondary tasks, and HMIs motivated by the hierarchy of driving skills are investigated toward possible contribution to improvement in situation awareness while resuming control authority during 10 seconds (s) takeover transient. The takeover period may provide benefits, such as traffic safety and readiness to the driver. To model the takeover performance, regression models are created to predict the takeover performance variables such as takeover time, minimum collision time, brake application, and collision probability [12]. These models are created by considering situational and driver-related factors such as time budget, traffic density, nondriving task, repetition, current lane, and driver’s age. In this study, models are designed using 753 takeover data, and 729 takeover validation data accurately captured takeover time, TTC, and collision probability and predicted brake application at a moderate level. Time budget, traffic density, and repetition strongly influenced takeover performance, while other parameters affect less.Considering the TOR procedure, the driver is subject to either a driving task or NDRTs (such as listening to music, reading, talking, and texting on the phone). And the driver may be subject to problems such as skill, experience, fatigue, stress, and NDRTs [13]. Complexity factors fall into two categories: objective and subjective [14]. Objective complexity consists of traffic state, road state, and control transfer. Subjective complexity consists of NDRT, age, trust, the urgency of the situation, HMI, and situational awareness.The effects of NDRTs on driver and takeover time quality are examined using a simulator [15–20]. These studies examine the influence of the different traffic scenarios and NDRTs on the takeover process during automated driving, because NDRTs are the physical, mental, or visual activities that may affect the human factor and endanger driving safety. In [21], the reaction time is examined along with five different simulated traffic scenarios and two different NDRTs. It is determined that 90% of the drivers need at least 8 s for a correct reaction.In [22], an accident scenario is represented on the right lane of a three-lane freeway. The results of 5 s and 7 s TOR times are compared. In [23], they tested how 27 participants performed in a driving simulator after being subjected to authority transition for an emergency. Three transition time conditions were tested: 2 s, 5 s, or 8 s before road hazards. According to the results of the tests, a few drivers in the 2 s condition were able to safely overcome the road hazard situation, while the majority of drivers subject to the condition of 5 s or 8 s were able to pass the hazard safely.


## 1.1. Related Studies

Definition of driver’s risk, factors contributing to driving errors, has attracted a lot of attention to predict driving safety. In [3], the driver’s risk of making a road accident is categorized as low, medium, and high; driving behaviour is detected as safe or unsafe by evaluating the gap between the driver’s perception of driving style and measurements that are accelerations on both longitudinal and lateral direction and speed. They conducted a survey on five drivers who drove 1000 km across seven different paths and analysed the perception of driving style versus logged measurements. Definition of driving risk is complicated and subject to several variables. Potential risk factors affecting the driver were analysed by using the naturalistic driving data constituted by 10,800 six-second trip segments from the second Strategic Highway Research Program (SHRP2) [4, 5]. Differences in driver behaviour and surprise, visually distracting secondary tasks, have been detected as the main factors contributing to traffic violations and driving errors. In [6], driving risk in a driving scenario and the risky situation are detected by weighting three traffic safety experts’ knowledge. Driving sessions are performed by the same driver in a realistic truck simulator, and experts’ opinions are combined to label risk describing 18 risky driving situations. In [2], the control authority transition (CAT) has been introduced to avoid a possible collision. Driver’s operating characteristics are assessed while monitoring the driver’s intents, perceiving possible hazards and situations, and the control is modified to assure safe manoeuvring tasks performed by the driver as the main control authority. Considering L3 level conditional automation, TOT of the driver and TOR time plays a crucial role during the CAT for the driver to perform the manoeuvring tasks. We plot the time sequence in Figure 1. TOT defines the time between the takeover request and the takeover action [7], and TOR time defines the driving time between the takeover request and the critical situation [8].Figure 1
Takeover procedure in L3 conditional automated driving.In [8], a design methodology for the TOR time is presented. Although all drivers have the same information, the potential difference exists between individual reaction abilities and the driver characteristics. Reaction time is considered and human-in-the-loop PC-based simulation experiments are conducted to test and evaluate four given TORs to perform conditional automation. A performance-based methodology is presented, and a conditional probability of successful and unnecessary alert in the condition of a collision event or no alert issue is computed. Thirty drivers having a valid driver’s license participated in the simulator experiments and reaction time to the TOR, maximum acceleration, average lane departure distance, the number of collisions, and the average heart rate during TOR constitutes the quantitative results for analysis purposes. The questionnaire is used to collect participants’ acceptance. Statistically significant differences between the four TOR thresholds are detected for individual drivers. Dependency on the accuracy of human, vehicle, and environmental models of this model-based methodology needs to be improved by extensive data and real experimental study. In [9], four TOR-related critical situations are presented; stationary obstacle, slow vehicle, exit highway, and turning right on four-way uncontrolled intersection are the simulated scenarios. Criticality is varied by increasing the speed and traffic and adding an approaching vehicle or a pedestrian crossing intersection. 33 male and 3 female drivers participated, and they give the TOR 7 seconds to the drivers. A generic model in terms of several variables is assumed to give reaction time.In [10], the literature for TOR requests in L3 conditional automated driving is reviewed. Traffic situation, road condition, and transition system from automated to manual driving system are considered being objective complexity, and NDRTs, age, trust, HMI, and situational awareness are considered being subjective complexity affecting the TOR of the individual driver. Metrics are reviewed as well. Lateral and longitudinal metrics, including TTC, the number of actual and successful manoeuvring task to avoid obstacle either by braking or by performing a lane change, cognitive workload, trust, and comfort is presented as the former metrics. Real-world tests are presented as research gaps to be filled to improve the TOR transient procedure.In [11], HMI design for control authority transition and strategies are studied in a simulator of longitudinal autonomous driving with 49 experienced drivers. During a commuting period of 30 minutes, drivers take secondary tasks, and HMIs motivated by the hierarchy of driving skills are investigated toward possible contribution to improvement in situation awareness while resuming control authority during 10 seconds (s) takeover transient. The takeover period may provide benefits, such as traffic safety and readiness to the driver. To model the takeover performance, regression models are created to predict the takeover performance variables such as takeover time, minimum collision time, brake application, and collision probability [12]. These models are created by considering situational and driver-related factors such as time budget, traffic density, nondriving task, repetition, current lane, and driver’s age. In this study, models are designed using 753 takeover data, and 729 takeover validation data accurately captured takeover time, TTC, and collision probability and predicted brake application at a moderate level. Time budget, traffic density, and repetition strongly influenced takeover performance, while other parameters affect less.Considering the TOR procedure, the driver is subject to either a driving task or NDRTs (such as listening to music, reading, talking, and texting on the phone). And the driver may be subject to problems such as skill, experience, fatigue, stress, and NDRTs [13]. Complexity factors fall into two categories: objective and subjective [14]. Objective complexity consists of traffic state, road state, and control transfer. Subjective complexity consists of NDRT, age, trust, the urgency of the situation, HMI, and situational awareness.The effects of NDRTs on driver and takeover time quality are examined using a simulator [15–20]. These studies examine the influence of the different traffic scenarios and NDRTs on the takeover process during automated driving, because NDRTs are the physical, mental, or visual activities that may affect the human factor and endanger driving safety. In [21], the reaction time is examined along with five different simulated traffic scenarios and two different NDRTs. It is determined that 90% of the drivers need at least 8 s for a correct reaction.In [22], an accident scenario is represented on the right lane of a three-lane freeway. The results of 5 s and 7 s TOR times are compared. In [23], they tested how 27 participants performed in a driving simulator after being subjected to authority transition for an emergency. Three transition time conditions were tested: 2 s, 5 s, or 8 s before road hazards. According to the results of the tests, a few drivers in the 2 s condition were able to safely overcome the road hazard situation, while the majority of drivers subject to the condition of 5 s or 8 s were able to pass the hazard safely.


## 2. The Design Methodology for TOR Time Subject to Individual Driver’s Ability and Readiness

### 2.1. Local Outlier Factor (LOF)

Outlier is a current workspace with categories such as distribution-based, depth-based, and distance-based. Outlier studies discover valid and new knowledge in large datasets. LOF is an anomaly detection algorithm presented by Breunig et al. to show the anomaly degree of a point for multidimensional datasets and the LOF algorithm states that the degree of isolation of the object from the surrounding neighbourhood is significant [24]. Thus, the LOF is local because the degree depends on how isolated the object is from the surrounding neighbourhood. The LOF algorithm can detect significant local outliers that previous approaches were unable to find. This approach of finding local outliers is efficient for datasets where nearest neighbour queries are supported by index structures, and practical for very large datasets [25]. Results are improved with respect to global approaches, particularly a small distance to the dense cluster as a local neighbourhood advantage. On the other hand, since this algorithm does not have a certain threshold value, choosing an outlier is user-dependent. In general, the object is considered the outlier when the LOFscore>>1. However, the algorithm does not have a certain threshold value. Overall, the LOF algorithm performs like the KNN classification algorithm. While the KNN tries to find out who the desired point is, the LOF tries to find out who is no more like its neighbours.


## 2.1. Local Outlier Factor (LOF)

Outlier is a current workspace with categories such as distribution-based, depth-based, and distance-based. Outlier studies discover valid and new knowledge in large datasets. LOF is an anomaly detection algorithm presented by Breunig et al. to show the anomaly degree of a point for multidimensional datasets and the LOF algorithm states that the degree of isolation of the object from the surrounding neighbourhood is significant [24]. Thus, the LOF is local because the degree depends on how isolated the object is from the surrounding neighbourhood. The LOF algorithm can detect significant local outliers that previous approaches were unable to find. This approach of finding local outliers is efficient for datasets where nearest neighbour queries are supported by index structures, and practical for very large datasets [25]. Results are improved with respect to global approaches, particularly a small distance to the dense cluster as a local neighbourhood advantage. On the other hand, since this algorithm does not have a certain threshold value, choosing an outlier is user-dependent. In general, the object is considered the outlier when the LOFscore>>1. However, the algorithm does not have a certain threshold value. Overall, the LOF algorithm performs like the KNN classification algorithm. While the KNN tries to find out who the desired point is, the LOF tries to find out who is no more like its neighbours.


## 3. Experimental Design

### 3.1. TOR Scenarios

In the study, each driver tests five cases. Tasks need to be executed while approaching or crossing the intersection. The first case is crossing the intersection in campus traffic and taking the control authority of the vehicle as a baseline experiment. Then, the driver was asked to follow the same test route but he/she is subject to NDRTs (such as listening to music) and dynamic obstacles such as a pedestrian intending to cross the road along the oncoming path of the ego vehicle and other participating vehicles. These five cases are described in Table1 and illustrated on a route of the ego vehicle on a campus road in Figure 2.Table 1
Tasks, task descriptions, and task photos.
CaseDescription/taskTask photos1Crossing the intersection: at the end of autonomous driving on road 1 (see also Figure2), the driver takes over control when the TOR period expires. The driver drives the vehicle across the junction (denoted by road 2 in Figure 2).2Crossing at the intersection where the pedestrian crosses: at the end of automated driving on road 3, the driver takes over control when the TOR period expires. The driver drives the vehicle across the junction on road 4 where the pedestrian crosses.3Crossing the intersection with the sound of music: during autonomous driving on road 5, the human-machine interface (HMI) starts playing music as a nondriving-related task. At the end of autonomous driving on road 5, the driver takes over control when the TOR period expires. The driver drives the vehicle across the junction on road 6.4Crossing the intersection with the vehicle approaching from the side road (pink vehicle): at the end of autonomous driving on road 5 on the arrival, the driver takes over control when the TOR period expires. The driver commands the vehicle across the junction on road 4 where another vehicle approaches from the side road.5Crossing the intersection where another vehicle is stopped and the driver overtakes: at the end of autonomous driving on road 3 on the arrival, the driver takes over control when the TOR period expires. The driver drives the vehicle across the junction on road 2 where a vehicle is stopped at the intersection.Figure 2
Driving route and task points in the study.


### 3.2. Variables and Methods

Two methods are used. The first method is to obtain and interpret takeover time quality measures. Thus, it is to determine the most ideal takeover time for each driver.Takeover quality measures are determined according to previous studies [25, 26] and test routes. Our TOR points and test route are located on a straight road. The longitudinal takeover measures for the takeover quality have been chosen, and extra quality measures for the takeover quality are described in Table 2.Table 2
Takeover time quality measures.
LetterNameReferenceDescriptionaThe mean longitudinal accelerationIndependent metrics (a, b, c, d, e) [26, 27]Average acceleration or deceleration value during manual driving (MD) after automated driving (AD) (5 values).bThe maximum velocityMaximum velocity value during MD after AD.cThe minimum velocityMinimum velocity value during MD after AD.dDistance to collision (DTC)Minimum distance to the collision before changing lanes.eTime to collision (TTC)Time to the collision before changing lanes.fStandard deviation (SD) value of the currentA custom new metric for the studyThe SD of the 3 current values (first value before the authority transition (AT), at the time of transition to authority and the first value after the AT).fxaSD value of the current multiplied by mean longitudinal acceleration valuesA custom new dependent metric for the studyThe SD of the 3 current values (first value before the authority transition (AT), at the time of transition to authority, the first value after the AT) multiplied by the average of five consecutive acceleration or deceleration values during MDgSD value of the GSR valuesA custom new metric for the studyThe SD of 3 GSR values (first value before the AT, at the time of transition to authority, the first value after the AT).Considering the second TOR design method, the outlier values from the GSR sensor data sought the ideal TOR time. We can consider authority transition as a disturbance input for the driver in the traffic environment, because there is a critical situation that requires a change of authority. Naturally, the reflection is extracted from the sensor data recorded by the human driver.In our previous study, it was found that there is a relationship between the driver’s stress level and the galvanic skin response of the driver [28]. In this study, different takeover times are applied to the same driver in the same traffic scenarios. We determine the TOR time interval for the individual driver while monitoring the GSR sensor data fluctuation during the authority transition. By applying the LOF algorithm to the GSR data, it is determined whether the driver status at the transition stage differs from the general excitement set of the driver.The takeover time between the automated driving system and the human driver is determined by various road scenarios and different drivers. Eighteen drivers have been driving the prototype electric vehicle equipped with a suite of sensors and a human-machine interface system in five different road scenarios with different takeover times. We conducted the experimental study on a campus area populated by 30,000 students shown in Figure3.Figure 3
Istanbul Technical University campus map and route (red line).By analysing 360 (=18numberofdrivers×5numberofscenarios×4numberofdifferenttakeovertimes) cases recorded from eighteen different drivers using five different road scenarios and four different takeover times (0 s, 2 s, 4 s, and 6 s), appropriate takeover times are determined. For instance, 6 s TOR time is a countdown from 6 s to 0 s, and at the end of this countdown period of 6 seconds, the driver intervenes the vehicle. Hence, a 6-second perception time and being prepared to intervene the vehicle are allowed to the driver. Unlike 6 s, 0 s TOR time does not permit the driver to perceive, and the driver is enforced to command the vehicle promptly.These analyses are made using takeover quality measures for the sensor data, and we evaluated particularly the impact of the GSR sensor, which shows the excitement and psychological state of the driver.


### 3.3. Experimental Procedure

The route shown in Figure3 is 1 km long and is located at the Istanbul Technical University campus. The electric vehicle with longitudinal autonomy has a fixed velocity of 14 km/h in automated driving, and it can accelerate to a maximum of 25 km/h in manual driving.The blue vehicle shown in Figure2 is the electric vehicle used for TOR testing and data collection purposes (ego vehicle) with longitudinal autonomy, and it is located at the starting point of the experimental test route. The pink vehicle, on the other hand, is a conventional internal combustion vehicle driven by a human driver that we use for two different tasks as a participating vehicle to the experimental study.The green dashed lines on the road represent the autonomous driving path, and the red solid lines represent the manual driving path. Each driver drives four times because four different TOR values were used on this test route. TOR times used on this route are 6 s, 4 s, 2 s, and 0 s.


### 3.4. Human-Machine Interface (HMI) Design

An HMI informs TOR time as a timely message to the driver. Depending on the urgency of the request, the interaction must be dynamic, the message must be in a prominent place in the vehicle to attract the driver’s attention. The way the message is transmitted can be classified as visual, auditory, and tactile [9]. In our study, we use a visual and acoustic HMI system via a mobile phone application to provide authority transitions at the same point on the specified route. Designed HMI system allows the driver to keep track of the transition points and the TOR time countdown. The reason for using the visual and acoustic HMI system: the visual system causes distraction as well as shows too much information on the screen. The audio system, on the other hand, does not require eye contact, but may not give the warning signal very clearly [9]. We aimed to eliminate possible negative outcomes by using both visual and acoustic HMI output.We designed an application in Android for the HMI. We embed this application in the smartphone mounted on the vehicle, as shown in Figure4(a).Figure 4
(a) Mounted mobile phone with HMI system, (b) bounding boxes on the route, and (c) TOR input screen on the HMI system.
(a)(b)(c)Location-based bounding boxes are plotted on the visual HMI to alert when the vehicle entered the intersection zone created by the TOR time and autonomous vehicle speed. Audible and visual TOR countdowns were started when the vehicle reached that location (see Figure4(b)). The red regions represent the bounding boxes on the forward route and the blue the bounding boxes on the return route. The TOR is selected at the beginning of the test at the interface, and the HMI is started while pressing the start button, as shown in Figure 4(c).The TOR countdown is audible and visual to the drive; the TOR countdown is initiated from the same point on each test. This function excites an NDRT to each participating driver by playing music to perform the third case in Table1.


### 3.5. Participants

Eighteen participants (8 females and 10 males) between 24 and 65 years of age joined in this study. The average and standard deviation (SD) of the ages are 40.3 and 14.7 years, respectively. All participants have a valid driver’s license, at least a high school degree, and one year of driver experience (mean: 19.8, SD: 15.3) at least. Participants consist of people who drive at different frequencies (daily: 11 drivers, once a week: 2 drivers, once a month: 4 drivers, once a year: 1 driver). Five of the participants were involved in a traffic accident. None of the participants had any visual and cognitive impairment that would affect their driving.


### 3.6. Apparatus

Equipment used in this simultaneous data acquisition system:(i)
A prototype electric vehicle with longitudinal autonomy (BLDC 750 W).(ii)
A 10-turn Vishay 860 potentiometer detects steering angle(iii)
Tomura 300A AC/DC hall-effect current sensor used for current (torque) data(iv)
Pololu FSR to detect force on the steering wheel and the brake pedal(v)
Seeedstudio GSR sensor to detect stress(vi)
MinIMU-9 v3 IMU to detect gyro, accelerometer, and compass values(vii)
GY-NEO6MV2 GPS to detect chassis velocity(viii)
Scancon 2RM 200 encoder to detect wheel velocity(ix)
Techsmart dash cam to detect environment and driver behaviour(x)
Lidar (light detection and ranging) to detect the objects and measure the distance(xi)
Laptop and electronic card to collect the data(xii)
A HMI (human-machine interface) system is embedded in the mobile phone to properly ensure authority transitionsThe above-mentioned sensors and electric vehicle constitute the data collection system, as shown in Figure5.Figure 5
Data collection system.


## 3.1. TOR Scenarios

In the study, each driver tests five cases. Tasks need to be executed while approaching or crossing the intersection. The first case is crossing the intersection in campus traffic and taking the control authority of the vehicle as a baseline experiment. Then, the driver was asked to follow the same test route but he/she is subject to NDRTs (such as listening to music) and dynamic obstacles such as a pedestrian intending to cross the road along the oncoming path of the ego vehicle and other participating vehicles. These five cases are described in Table1 and illustrated on a route of the ego vehicle on a campus road in Figure 2.Table 1
Tasks, task descriptions, and task photos.
CaseDescription/taskTask photos1Crossing the intersection: at the end of autonomous driving on road 1 (see also Figure2), the driver takes over control when the TOR period expires. The driver drives the vehicle across the junction (denoted by road 2 in Figure 2).2Crossing at the intersection where the pedestrian crosses: at the end of automated driving on road 3, the driver takes over control when the TOR period expires. The driver drives the vehicle across the junction on road 4 where the pedestrian crosses.3Crossing the intersection with the sound of music: during autonomous driving on road 5, the human-machine interface (HMI) starts playing music as a nondriving-related task. At the end of autonomous driving on road 5, the driver takes over control when the TOR period expires. The driver drives the vehicle across the junction on road 6.4Crossing the intersection with the vehicle approaching from the side road (pink vehicle): at the end of autonomous driving on road 5 on the arrival, the driver takes over control when the TOR period expires. The driver commands the vehicle across the junction on road 4 where another vehicle approaches from the side road.5Crossing the intersection where another vehicle is stopped and the driver overtakes: at the end of autonomous driving on road 3 on the arrival, the driver takes over control when the TOR period expires. The driver drives the vehicle across the junction on road 2 where a vehicle is stopped at the intersection.Figure 2
Driving route and task points in the study.


## 3.2. Variables and Methods

Two methods are used. The first method is to obtain and interpret takeover time quality measures. Thus, it is to determine the most ideal takeover time for each driver.Takeover quality measures are determined according to previous studies [25, 26] and test routes. Our TOR points and test route are located on a straight road. The longitudinal takeover measures for the takeover quality have been chosen, and extra quality measures for the takeover quality are described in Table 2.Table 2
Takeover time quality measures.
LetterNameReferenceDescriptionaThe mean longitudinal accelerationIndependent metrics (a, b, c, d, e) [26, 27]Average acceleration or deceleration value during manual driving (MD) after automated driving (AD) (5 values).bThe maximum velocityMaximum velocity value during MD after AD.cThe minimum velocityMinimum velocity value during MD after AD.dDistance to collision (DTC)Minimum distance to the collision before changing lanes.eTime to collision (TTC)Time to the collision before changing lanes.fStandard deviation (SD) value of the currentA custom new metric for the studyThe SD of the 3 current values (first value before the authority transition (AT), at the time of transition to authority and the first value after the AT).fxaSD value of the current multiplied by mean longitudinal acceleration valuesA custom new dependent metric for the studyThe SD of the 3 current values (first value before the authority transition (AT), at the time of transition to authority, the first value after the AT) multiplied by the average of five consecutive acceleration or deceleration values during MDgSD value of the GSR valuesA custom new metric for the studyThe SD of 3 GSR values (first value before the AT, at the time of transition to authority, the first value after the AT).Considering the second TOR design method, the outlier values from the GSR sensor data sought the ideal TOR time. We can consider authority transition as a disturbance input for the driver in the traffic environment, because there is a critical situation that requires a change of authority. Naturally, the reflection is extracted from the sensor data recorded by the human driver.In our previous study, it was found that there is a relationship between the driver’s stress level and the galvanic skin response of the driver [28]. In this study, different takeover times are applied to the same driver in the same traffic scenarios. We determine the TOR time interval for the individual driver while monitoring the GSR sensor data fluctuation during the authority transition. By applying the LOF algorithm to the GSR data, it is determined whether the driver status at the transition stage differs from the general excitement set of the driver.The takeover time between the automated driving system and the human driver is determined by various road scenarios and different drivers. Eighteen drivers have been driving the prototype electric vehicle equipped with a suite of sensors and a human-machine interface system in five different road scenarios with different takeover times. We conducted the experimental study on a campus area populated by 30,000 students shown in Figure3.Figure 3
Istanbul Technical University campus map and route (red line).By analysing 360 (=18numberofdrivers×5numberofscenarios×4numberofdifferenttakeovertimes) cases recorded from eighteen different drivers using five different road scenarios and four different takeover times (0 s, 2 s, 4 s, and 6 s), appropriate takeover times are determined. For instance, 6 s TOR time is a countdown from 6 s to 0 s, and at the end of this countdown period of 6 seconds, the driver intervenes the vehicle. Hence, a 6-second perception time and being prepared to intervene the vehicle are allowed to the driver. Unlike 6 s, 0 s TOR time does not permit the driver to perceive, and the driver is enforced to command the vehicle promptly.These analyses are made using takeover quality measures for the sensor data, and we evaluated particularly the impact of the GSR sensor, which shows the excitement and psychological state of the driver.


## 3.3. Experimental Procedure

The route shown in Figure3 is 1 km long and is located at the Istanbul Technical University campus. The electric vehicle with longitudinal autonomy has a fixed velocity of 14 km/h in automated driving, and it can accelerate to a maximum of 25 km/h in manual driving.The blue vehicle shown in Figure2 is the electric vehicle used for TOR testing and data collection purposes (ego vehicle) with longitudinal autonomy, and it is located at the starting point of the experimental test route. The pink vehicle, on the other hand, is a conventional internal combustion vehicle driven by a human driver that we use for two different tasks as a participating vehicle to the experimental study.The green dashed lines on the road represent the autonomous driving path, and the red solid lines represent the manual driving path. Each driver drives four times because four different TOR values were used on this test route. TOR times used on this route are 6 s, 4 s, 2 s, and 0 s.


## 3.4. Human-Machine Interface (HMI) Design

An HMI informs TOR time as a timely message to the driver. Depending on the urgency of the request, the interaction must be dynamic, the message must be in a prominent place in the vehicle to attract the driver’s attention. The way the message is transmitted can be classified as visual, auditory, and tactile [9]. In our study, we use a visual and acoustic HMI system via a mobile phone application to provide authority transitions at the same point on the specified route. Designed HMI system allows the driver to keep track of the transition points and the TOR time countdown. The reason for using the visual and acoustic HMI system: the visual system causes distraction as well as shows too much information on the screen. The audio system, on the other hand, does not require eye contact, but may not give the warning signal very clearly [9]. We aimed to eliminate possible negative outcomes by using both visual and acoustic HMI output.We designed an application in Android for the HMI. We embed this application in the smartphone mounted on the vehicle, as shown in Figure4(a).Figure 4
(a) Mounted mobile phone with HMI system, (b) bounding boxes on the route, and (c) TOR input screen on the HMI system.
(a)(b)(c)Location-based bounding boxes are plotted on the visual HMI to alert when the vehicle entered the intersection zone created by the TOR time and autonomous vehicle speed. Audible and visual TOR countdowns were started when the vehicle reached that location (see Figure4(b)). The red regions represent the bounding boxes on the forward route and the blue the bounding boxes on the return route. The TOR is selected at the beginning of the test at the interface, and the HMI is started while pressing the start button, as shown in Figure 4(c).The TOR countdown is audible and visual to the drive; the TOR countdown is initiated from the same point on each test. This function excites an NDRT to each participating driver by playing music to perform the third case in Table1.


## 3.5. Participants

Eighteen participants (8 females and 10 males) between 24 and 65 years of age joined in this study. The average and standard deviation (SD) of the ages are 40.3 and 14.7 years, respectively. All participants have a valid driver’s license, at least a high school degree, and one year of driver experience (mean: 19.8, SD: 15.3) at least. Participants consist of people who drive at different frequencies (daily: 11 drivers, once a week: 2 drivers, once a month: 4 drivers, once a year: 1 driver). Five of the participants were involved in a traffic accident. None of the participants had any visual and cognitive impairment that would affect their driving.


## 3.6. Apparatus

Equipment used in this simultaneous data acquisition system:(i)
A prototype electric vehicle with longitudinal autonomy (BLDC 750 W).(ii)
A 10-turn Vishay 860 potentiometer detects steering angle(iii)
Tomura 300A AC/DC hall-effect current sensor used for current (torque) data(iv)
Pololu FSR to detect force on the steering wheel and the brake pedal(v)
Seeedstudio GSR sensor to detect stress(vi)
MinIMU-9 v3 IMU to detect gyro, accelerometer, and compass values(vii)
GY-NEO6MV2 GPS to detect chassis velocity(viii)
Scancon 2RM 200 encoder to detect wheel velocity(ix)
Techsmart dash cam to detect environment and driver behaviour(x)
Lidar (light detection and ranging) to detect the objects and measure the distance(xi)
Laptop and electronic card to collect the data(xii)
A HMI (human-machine interface) system is embedded in the mobile phone to properly ensure authority transitionsThe above-mentioned sensors and electric vehicle constitute the data collection system, as shown in Figure5.Figure 5
Data collection system.


## 4. Experimental Results

### 4.1. Takeover Request (TOR) Time Results for Case 1, Case 2, and Case 3

Before using the data, there was a need to ask the drivers a question because the vehicle is not a conventional vehicle but is a prototype electric vehicle. We asked drivers the following question (see the Appendix): “Do you think your driving today is representative of your normal driver behaviour?” According to the answers (Likert Scale: 1-5 (not at all–pretty much), 83.3% of the drivers (15 drivers) answered “pretty much” (Likert 5), 11.1% of drivers (2 drivers) answered “much” (4), and 5.5% of drivers (1 driver) answered “very little” (2).The following criteria for case 1 have been calculated for each driver and different TOR times; in Table3 (fxa), which is the current standard deviation (f) multiplied by the average value of the five acceleration measurements (a) and the standard deviation (g) of the GSR values determine whether the authority transition has been executed smoothly by the driver.Table 3
Analysis results for case 1.
DTORa (m/s2)b (km/h)c (km/h)d (m)e (s)f (A)f.a (Am/s2)gRQ16 s0.85122429.397.9832.532 s2 s4 s0.9013.767.6793.627.947.156.802 s1.0114.137.76155.414.834.883.210 s1.2210.436.43217.606.537.9738.5026 s0.9813.674.4152.227.637.484.582 s4 s4 s0.9910.954.87114.257.016.943.602 s1.0015.36.69176.564.554.550.570 s1.1119.064.722415.097.578.402.0836 s1.0212.20.0251.626.766.907.374 s2 s4 s0.3012.396.07103.23.280.984.612 s0.578.564.02168.335.693.2450 s0.898.783.67239.028.287.375.6846 s0.9611.655.4341.754.974.779.292 s2 s4 s1.0113.788.0792.242.692.729.862 s0.5411.396.02166.302.041.103.460 s0.8910.064.592410.1211.7710.487.6356 s1.109.355.4352.137.588.333.644 s4 s4 s1.1712.86.95103.526.807.953.052 s1.1417.459.02166.408.479.664.160 s1.1617.335.70238.366.897.993.6066 s0.627.393.3952.495.773.585.294 s2 s4 s0.678.325.13115.285.133.433.212 s0.6610.55.78176.275.953.935.130 s0.5915.767.872310.57.924.624.0476 s0.769.33.8752.113.692.805.294 s4 s4 s0.8273.15105.151.961.611.732 s0.959.724.17168.886.796.452.080 s0.9011.55.43229.957.426.682.5786 s1.279.894.8562.184.826.121.522 s4 s4 s0.4311.24.82114.8014.346.170.572 s0.9111.24.70175.444.584.170.570 s0.8012.35.82248.826.965.571.5296 s1.0213.525.2462.982.282.3324 s4 s4 s1.0310.484.87124.043.403.500.572 s0.9610.466.33175.855.064.853.050 s0.9811.375.67247.893.633.551106 s0.6115.785.1562.647.504.581.154 s4 s4 s0.7411.454.91125.424.793.541.152 s0.8411.546.02188.1412.9810.902.510 s0.5210.986.83258.205.052.632.08116 s0.5713.15.1741.2811.26.38102 s0 s4 s0.9317.677.54103.227.657.116.242 s0.5511.574.93177.553.872.132.50 s0.6310.741.28257.7118.7911.841.15126 s1.0111.193.8751.68.148.2225.52 s2 s4 s0.90137.56103.0813.0911.788.082 s0.8511.136.87165.168.096.880.570 s0.7616.638.5724616.8412.801.15136 s0.998.504.4162.668.768.673.780 s2 s4 s0.5010.33.89124.197.893.951.522 s0.2910.285.82176.743.050.884.040 s0.5410.155.72248.863.892.101.15146 s0.826.831.5252.6312.3310.111.572 s2 s4 s0.8911.35113.6112.4511.081.152 s0.8412.043.46175.483.583.011.150 s0.7910.133.37249.192.992.361.52156 s1.0216.88.5061.7511.6211.853.512 s2 s4 s0.8713.114.89123.2914.0912.261.522 s0.8213.416.67185.537.986.541.520 s0.7713.854.87256.578.626.6311.67166 s1.0313.545.7062.537.417.6310.110 s2 s4 s0.4811.024.70113.594.262.042.082 s1.2015.7610.95174.6410.7212.860.570 s0.5412.636.06246.834.102.210.57176 s1.4714.636.3051.655.508.094.152 s2 s4 s0.1813.566.4893.7012.742.291.522 s0.4912.393.72164.652.811.381.210 s0.6513.484.80246.4017.7611.542.08186 s0.7211.355.3251.5810.227.3615.030 s4 s4 s0.6614.372.301236.264.132.082 s0.8510.263.75186.38.226.991.520 s0.5312.762.522474.582.431.52The critical situation for case 1 is the control authority transition from the automated system to the driver while approaching the intersection. In Table3, d denotes the distance, the average velocity during the transition, which is denoted by vtransition, and gives time to the intersection, denoted by e=d/vtransition in Table 3. In the following, a denotes the average value of the five consecutive acceleration measurements collected by the TOR trigger. During the transition, the values of maximum velocity (b) and minimum velocity (c) are extracted. And the authority and the readiness of the driver subject to transition while approaching the intersection are discovered.Driver 1, the smoothest transition, occurred with a TOR about 2 s. Pulse deviation validates this: (g) value is the lowest value of 3.21 for TOR 2 s, whereas 32.53 for TOR 6 s, 6.80 for TOR 4 s, and 38.50 for TOR 0 s. The metric, denoted by (fxa), gives the lowest value in the same manner for TOR 2 s, which also validates the control authority of the driver 1 who does not excite by braking or accelerating the electric vehicle, instead of his/her smooth commanding capability at the TOR transient stage. For TOR 2 s, the distance to the critical situation, denoted by DTC, is 15 m, and the time to reach the intersection, denoted by TTC, is 5.41 s. Driver 1 performs smoothly the manoeuvring task to approach the intersection. When driver 1 takes the control, acceleration is reached at 1.01 m/s2 and has a comparably higher maximum and minimum velocity value than other TOR cases, denoted by (b) and (c) in Table 3, respectively. This also shows the confidence of the driver 1 while commanding the vehicle after the control authority transition from the automated system.Considering driver 2, the pulse deviation metric (g) and (fxa) takes a lower value for TOR 2 s with respect to the test metric values of other TOR such as 0 s, 4 s, and 6 s. Experiment with a TOR 2 s, the DTC is 17 m, and TTC is 6.56 s. This provides sufficient distance and time for the driver to command the vehicle. After the transition, the driver accelerates with 1 m/s2. The vehicle reached a maximum speed of 15.3 km/h and a minimum speed of 6.69 km/h. Comparing the speed values (d) for different TORs in Table 3, driver 2 has the authority over the vehicle while commanding traction/braking with more confidentiality after the TOR request about 2 s.Analysing the TOR experiment results of driver 3, the (g) and (fxa) metric values are lower for TOR 4 s in comparison with other TOR request times. For TOR 4 s, DTC is 10 m, and TTC is 3.2 s. After the transition, driver 3 drives with minimum average acceleration, higher maximum velocity, and minimum velocity metric values, showing his/her control over the vehicle while approaching the intersection.Driver 5, driver 6, driver 7, and driver 10 performs smooth authority transition, causing a lower (fxa) and (g) value for the experiment with TOR 4 s. Driver 4, driver 8, driver 11, driver 12, driver 15, and driver 17 perform smooth authority transition subject to TOR 2 s. Driver 16 and driver 18 perform also smooth authority transition for TOR 0 s in terms of (fxa) and (g) metric values.The lower pulse deviation of driver 9 is assured when TOR is 4 s, and lower (fxa) occurs when TOR is 6 s. Since acceleration and velocity values are also close to each other, DTC (d) and TTC (e) values in Table 3 have been compared. For TOR 4 s, the metric value (d) and (e) is higher with respect to other TOR times.Driver 13 generates lower pulse deviation for TOR 0 s whereas (fxa) is lower for the experiment with TOR 2 s. The (fxa) value of TOR 0 s is lower in comparison to its value for TOR 2 s, and DTC (d) and TTC (e) are higher; in consequence, TOR 0 s has been detected higher confidence in transition for driver 13.For driver 14, lower pulse deviation is given for TOR 4 s and TOR 2 s whereas the lower (fxa) occurs when TOR is 0 s. However, (fxa) is lower for the experiment with TOR 2 s in comparison with TOR 4 s. For this reason, it would be better to compare TOR 0 s and TOR 2 s experiments. DTC (d) and TTC (e) values assure safe driving for both experiments, and minimum velocities are equal, which are 3.46 km/h and 3.37 km/h. The TOR 2 s has a higher velocity, and the driver tends to accelerate more. Hence, TOR 2 s gives a higher confidence in control authority transition for driver 14.For theR column (results column) in Table 3, the ideal TOR value of 9 drivers is 2 s, the ideal TOR value of 6 drivers is 4 s, and TOR 0 s is the ideal for 3 drivers. Accordingly, TOR 6 s is very close to the critical situation and can cause a higher (fxa) and (g). Hence, it is not detected as the ideal transition time for all TOR cases in the experimental data. On the other hand, the TOR 0 s is a sudden transition; the driver can be caught unready and may cause a higher (fxa) and (g). However, this transition turned out to be the ideal transition for 3 drivers, see the R column result of driver 13, driver 16, and driver 18 in Table 3, who prefers 0 s for TOR.After the road experiment, the answers to the question of what is your preferred TOR time, which is asked to each driver (see the questionnaire in the Appendix), are given in theQ column in Table 3. According to the questionnaire, none of the drivers prefer TOR 6 s, two drivers prefer TOR 0 s, and other drivers prefer TOR 4 s and TOR 2 s to authorize control transition. When the TOR values of the analysis and the questionnaire are examined, the TORs of 10 drivers are the same as a result of the analysis and questionnaire.We visualize the data of 18 drivers for their ideal TOR time with a box plot in Figure6. According to this plot, DTC, denoted by (d), varies largely in its distribution with respect to other TOR quality measures. And in the test case 1, the DTC (d) value can be between 10 and 25 m subject to the driver’s preferred TOR time as given in Table 3.Figure 6
Box plot of analysis results for the test drive case 1 for 18 drivers.As in case 1, we conducted analyses for case 2 and case 3. The critical situation for case 2 is a pedestrian walking on the oncoming vehicle route, and for case 3, a NDRT disturbs the driver while approaching the intersection. Table4 presents the TOR time obtained for case 2 and case 3 as a result of a similar analysis conducted for case 1.Table 4
TOR results for case 1, case 2, case 3, and questionnaire.
DriverTOR case 1TOR case 2TOR case 3TOR questionnaire1∗2 s2 s4 s2 s22 s2 s4 s4 s34 s2 s4 s2 s42 s2 s4 s2 s54 s4 s4 s4 s64 s2 s2 s2 s74 s4 s4 s4 s82 s2 s2 s4 s94 s4 s2 s4 s10∗4 s4 s4 s4 s11∗2 s4 s0 s0 s122 s4 s4 s2 s130 s0 s2 s2 s142 s0 s0 s2 s152 s4 s2 s2 s160 s0 s0 s2 s172 s2 s0 s2 s180 s0 s2 s4 sAccording to results in Table4, TOR 6 s has not been adopted by the drivers for any case. For case 2, the ideal TOR time of 7 drivers is 2 s, the ideal TOR time of 7 drivers is 4 s, and the ideal TOR time of 4 drivers is 0 s. For case 3, the ideal TOR time of 6 drivers is 2 s, the ideal TOR time of 8 drivers is 4 s, and the ideal TOR time of 4 drivers is 0 s.According to the descriptive questionnaire, the experienced driver is driver 10 who has 42 years of driving experience and commutes on a daily frequency. The inexperienced is driver 11, with a driving frequency of one trip per year and had a driving license about one month ago. The semiexperienced driver is driver 1, having five years of driving experience with a driving frequency of one trip per week. If we compare the TOR results of these three drivers in Table4, the TOR for the expert driver (driver 10) is TOR 4 s in all drives and the questionnaire. For inexperienced driver (driver 11), the TOR value differs for different cases. The TOR for the semiexperienced driver (driver 1) is TOR 2 s for case 1, case 2, and questionnaire and TOR 4 s for case 3.The GSR data of the expert driver, inexperienced driver, and semiexperienced driver are plotted in Figures7(a)–7(c). Box plots of GSR voltage of expert driver, inexperienced driver, and semiexperienced driver are plotted in Figures 7(d)–7(f). The magnitude of the drivers’ GSR values do not provide information because skin conductivity may vary from person to person. But instead, the spread of the distributions of the GSR values and the large deviations can be considered as a measure of the level in stress.Figure 7
For different TOR times: (a) GSR values of driver 10, (b) GSR values of driver 11, (c) GSR values of driver 1, (d) box plot of GSR voltage of driver 10, (e) box plot of GSR voltage of driver 11, and (f) box plot of GSR voltage of driver 1.
(a)(b)(c)(d)(e)(f)In Figure7(d), although the voltage values of the expert driver are higher than the inexperienced driver and semiexperienced driver, GSR voltage variation is bounded by 4% where the GSR value of the inexperienced is subject to a variation about 350% in its distribution, and the GSR value of the inexperienced is subject to a variation about 30%. Figure 7(e) presents the GSR values of the inexperienced driver, which vary largely despite lowest voltage levels generated in comparison to the experienced driver and semiexperienced driver.


### 4.2. Fixed Takeover Request (TOR) Time Analysis for Case 4 and Case 5

The data of 3 drivers (expert, semiexperienced, and inexperienced) are analysed and compared. While analysing the cases, they are analysed as integrated rather than separately because the cases are sequential. Tasks in case 4 and case 5 are described as follows:Case 4 task: the test vehicle is approaching the intersection on the main road, and another vehicle is approaching the same intersection on a perpendicular road (see, for instance, the pink vehicle in Figure2 describing case 4). A screenshot from the case 4 experiment is plotted in Figure 8.Figure 8
Case 4 task; the participating vehicle causing a possible collision at the intersection is coloured by red.Case 5 task: after crossing the intersection, another participating vehicle is stopped in a lane along the test route, and the control authority is transferred to the test driver; he/she stops the test vehicle or overtakes that stopping vehicle. A screenshot of the experiment is plotted in Figure9.Figure 9
Case 5 task; the other vehicle stopped in a lane is coloured red.According to the TOR results in Table4, the drivers’ preference is 2 s or 4 s. For this reason, analyses are made for case 4 and case 5 using a constant TOR time of 2 s while TOR time is varied from 0 s to 6 s with a resolution of 2 s for case 1, case 2, and case 3 in each experiment. Overall, each driver test-drive four times with a constant TOR 2 s for case 4 and case 5; the test results are compared in Table 5.Table 5
Results for case 4 and case 5.
DriverDriving numberTOR case 1, case 2, and case 3TOR case 4 and case 5(fxa) (Am/s2) for case 4 and case 5(g) for case 4 and case 5Expert16 s2 s9.731.2324 s2 s5.641.1832 s2 s9.273.1840 s2 s7.621.47Semiexperienced16 s2 s11.499.2524 s2 s13.326.8932 s2 s8.505.4040 s2 s9.311.25Inexperienced16 s2 s10.826.9124 s2 s16.473.3332 s2 s17.069.1340 s2 s12.701.45Toward analysis of experiments for case 4 and case 5, (fxa) and (g) metric values are given and compared for the expert, semiexperienced, and inexperienced drivers in Table 5. The expert driver experiments case 4 and case 5 on his/her second test drive with a lower (fxa) and (g) value in comparison to the other three test drive results. And the same driver completed the other test drive cases, which are case 1, case 2, and case 3, and achieved a lower value in (fxa) and (g) with a TOR 4 s.The lower (fxa) and (g) values for the semiexperienced and inexperienced drivers are performed on different driving tests. According to the results in Table 5, TOR values minimizing the (fxa) and (g) metric values vary.Comparing these three drivers, in case 4 and case 5 subject to a stationary and approaching vehicle to the path of the test vehicle, a higher level of driver experience improves the performance of control authority transition in conditional automation. For instance, in the second test drive, denoted by driving number 2 in Table5, the expert driver ensures lower (fxa) and (g) values with the TOR time 4 s in case 1, case 2, and case 3 and continues performing lower values when TOR time is given 2 s in case 4 and case 5.


### 4.3. Local Outlier Factor (LOF) Results

In this section, we have applied the LOF algorithm to the GSR data of the experienced driver, semiexperienced driver, and inexperienced driver. Firstly, the raw GSR sensor data of the drives are transformed into data in terms of voltages (voltagedata=rawdata/1024×5). The mean and standard deviation of each two consecutive GSR sensor data are calculated, and feature vectors are given. LOF algorithm is applied to these feature vectors, and LOF scores are obtained. A LOF score close to 1 indicates that the driver is less excited and relaxed.The semiexperienced driver data are analysed. According to these analyses, the LOF results of the GSR data are obtained by using the maximum 9th neighbourhood in the LOF algorithm for driver 1 and TOR 6 s. The GSR measurements are plotted in Figure10(a), LOF score responses are plotted in Figure 10(b), and the mean versus standard deviation of the feature is plotted in Figure 10(c). In Figure 10(b), the maximum local outlier score value is 3.9. In Figure 10(c), the distribution of mean and standard deviation feature of LOF values is plotted. LOF score values are coloured; the maximum score value being 3.9 is coloured by red and scaled.Figure 10
(a) GSR voltage values of driver 1 for 6 s TOR, (b) GSR data LOF scores of driver 1 for 6 s TOR, and (c) visual graph of GSR data LOF scores of the driver 1 for 6 s TOR.
(a)(b)(c)For the experiment with a TOR 4 s, the LOF score responses are plotted in Figure11, and the maximum LOF score is 6.3. LOF results of the GSR data are obtained by using the maximum 12th neighbourhood in the LOF algorithm.Figure 11
(a) GSR voltage values of driver 1 for 4 s TOR, (b) GSR data LOF scores of driver 1 for 4 s TOR, and (c) visual graph of GSR data LOF scores of the driver 1 for 4 s TOR.
(a)(b)(c)For TOR 2 s, LOF results of the GSR data and maximum LOF score value of 2.3 is obtained by using the maximum 10th neighbourhood in the LOF algorithm in Figure12.Figure 12
(a) GSR voltage values of driver 1 for 2 s TOR, (b) GSR data LOF scores of driver 1 for 2 s TOR, and (c) visual graph of GSR data LOF scores of the driver 1 for 2 s TOR.
(a)(b)(c)For TOR 0 s, LOF results of the GSR data are obtained by using the maximum 11th neighbourhood in the LOF algorithm shown in Figure13. Figure 13(b) shows the maximum LOF value reaching at 3.6. Figure 13(c) plots the distribution of mean and standard deviation feature of LOF values with respect to the scaled LOF score value.Figure 13
(a) GSR voltage values of driver 1 for 0 s TOR, (b) GSR data LOF scores of driver 1 for 0 s TOR, and (c) visual graph of GSR data LOF scores of the driver 1 for 0 s TOR.
(a)(b)(c)When the analysis results of the semiexperienced driver 1 are examined, the LOF score is minimally varied between 1 and 2.2 for TOR 2 s as plotted in Figure12(b) whereas the LOF score is increased to 6.5 when TOR is given 4 s. Considering the experiment of the semiexperienced driver, TOR 2 s data involves bounded deviation with respect to other TOR datasets.In Figure14, the LOF score responses of the semiexperienced driver at the control authority transition instances are marked with a dot for cases 1, 2, and 3, and cases 4 and 5 transition responses are marked with a circle. Case 5 and case 4 LOF scores are considered continuous in a given time interval because overtaking the stopped vehicle manoeuvre is a continuation of avoiding the participating vehicle cornering into the path of the oncoming test vehicle. Figure 14 presents LOF score responses subject to different TOR time responses from 0 s to 6 s from Figure 14(a) through Figure 14(d).Figure 14
(a) LOF scores of the cases for TOR 6 s (driver 1), (b) LOF scores of the cases for TOR 4 s (driver 1), (c) LOF scores of the cases for TOR 2 s (driver 1), and (d) LOF scores of the cases for TOR 0 s (driver 1).
(a)(b)(c)(d)While evaluating the results in Figure14, it is necessary to examine each dataset within itself. Because the LOF score in the dataset is a local value, it is valid for the particular dataset. Comparing the LOF score magnitude, the driver is excited and not comfortable with all authority transitions in the data set of TOR 6 s because the LOF score is higher with respect to other TOR LOF scores. The driver performed a comfortable transition at the TOR 4 s in case 2 and case 3. The driver is not comfortable in case 1 and case 4 and case 5 for TOR 4 s. The driver made a comfortable transition at the TOR 2 s in case 1 and case 2. The driver is not comfortable in case 3 and case 4 and case 5 for TOR 2 s and is excited and uncomfortable subject to all authority transitions with the TOR 0 s. Comfortable and smooth transitions cannot be directly compared with LOF values because they are local values. Comfortable and uncomfortable means also the level of readiness of the driver to take the control of the vehicle subject to possible driving hazards and situations when the transition from the automated system is asked to be executed.We analyse the inexperienced driver data in Figure15. According to these analyses, the LOF score results of the GSR data are obtained by using the maximum 9th neighbourhood in the LOF algorithm for TOR 6 s of driver 11. In Figure 15(b), the LOF value increases considerably between feature vectors 33 and 37, reaching the maximum LOF value of 5.1.Figure 15
(a) GSR voltage values of driver 11 for 6 s TOR, (b) GSR data LOF scores of driver 11 for 6 s TOR, and (c) visual graph of GSR data LOF scores of the driver 11 for 6 s TOR.
(a)(b)(c)For TOR 4 s and 2 s, the LOF results of the GSR data are obtained by using the maximum 8th neighbourhood in the LOF algorithm for driver 11. The GSR measurements are plotted in Figures16(a)and 17(a), LOF score responses are plotted in Figures 16(b) and 17(b), and the mean versus standard deviation of feature is plotted in Figures 16(c) and 17(c), respectively. In Figure 16(b), feature vector 17 reaches at 12.4, and the driver is overexcited and stressed at this instance. In Figure 17(b), the driver’s GSR values increase to the LOF of 4.3 at the beginning of the experiment, and GSR values are also coloured; the maximum is coloured by red and scaled in Figure 17(c).Figure 16
(a) GSR voltage values of driver 11 for 4 s TOR, (b) GSR data LOF scores of driver 11 for 4 s TOR, and (c) visual graph of GSR data LOF scores of the driver 11 for 4 s TOR.
(a)(b)(c)Figure 17
(a) GSR voltage values of driver 11 for 2 s TOR, (b) GSR data LOF scores of driver 11 for 2 s TOR, and (c) visual graph of GSR data LOF scores of the driver 11 for 2 s TOR.
(a)(b)(c)For TOR 0 s, the LOF results of the GSR data are obtained by using the maximum 10th neighbourhood, and we plot them in Figure18. In Figure 18(b), the LOF value increases considerably between feature vectors 25 and 32, reaching the maximum LOF value of 5.2. In Figure 18(c), the distribution of mean and standard deviation feature of LOF values is plotted.Figure 18
(a) GSR voltage values of driver 11 for 0 s TOR, (b) GSR data LOF scores of driver 11 for 0 s TOR, and (c) visual graph of GSR data LOF scores of the driver 11 for 0 s TOR.
(a)(b)(c)When the analysis results of the inexperienced driver 11 are examined, the LOF score is minimally varied between 1 and 4.2 for TOR 2 s as plotted in Figure17(b), whereas the LOF score is increased to 12 when TOR is given 4 s.While evaluating the results in Figure19, the driver made a comfortable transition for case 2 in the dataset of TOR 6 s. The driver made a comfortable transition for case 1, case 2, and case 3 subject to the TOR 4 s dataset. The driver made a comfortable transition for case 1 and case 2 in the TOR 2 s dataset. The driver has been subject to a higher LOF score for TOR 0 s.Figure 19
(a) LOF scores of the cases for TOR 6 s (driver 11), (b) LOF scores of the cases for TOR 4 s (driver 11), (c) LOF scores of the cases the TOR 2 s (driver 11), and (d) LOF scores of the cases for TOR 0 s (driver 11).
(a)(b)(c)(d)Lastly, we analyse expert driver data in Figure20. According to this analysis, the LOF results of the GSR data are obtained by using the maximum 15th neighbourhood in the LOF algorithm for driver 10 (expert driver) for TOR 6 s.Figure 20
(a) GSR voltage values of driver 10 for 6 s TOR, (b) GSR data LOF scores of driver 10 for 6 s TOR, and (c) visual graph of GSR data LOF scores of the driver 10 for 6 s TOR.
(a)(b)(c)In Figure20, the expert driver performs small deviations in the response of LOF scores while comparing with respect to inexperienced and semiexperienced drivers. The expert driver data for the TOR 6 s is plotted in Figure 21. In Figure 21(b), the distribution of mean and standard deviation feature of LOF values is plotted. The maximum LOF value is 3.5, and LOF score values are also coloured; the maximum is coloured by red and scaled.Figure 21
(a) GSR voltage values of driver 10 for 6 s TOR (wide scale) and (b) visual graph of GSR data LOF scores of the driver 10 for 6 s TOR (wide scale).
(a)(b)For TOR 4 s, the LOF results of the GSR data are obtained by using the maximum 9th neighbourhood for driver 10, and we plot them in Figure22. The maximum LOF value is 3.7 coloured by red and scaled in Figure 22(b) presenting the distribution of mean and standard deviation feature of LOF values.Figure 22
(a) GSR voltage values of driver 10 for 4 s TOR (wide scale) and (b) visual graph of GSR data LOF scores of the driver 10 for 4 s TOR (wide scale).
(a)(b)For TOR 2 s, the LOF results of the GSR data are obtained by using the maximum 9th neighbourhood for driver 10, and we plot them in Figure23. Figure 23(a) plots the GSR voltage responses, and they are subject to small deviations. In Figure 23(b), the distribution of mean and standard deviation feature of LOF values is plotted, the maximum LOF value is reached at 8.3, LOF score values are also coloured, and the maximum is coloured by red and scaled.Figure 23
(a) GSR voltage values of driver 10 for 2 s TOR (wide scale) and (b) visual graph of GSR data LOF scores of the driver 10 for 2 s TOR (wide scale).
(a)(b)For TOR 0 s, the LOF results of the GSR data are obtained by using the maximum 12th neighbourhood for driver 10, and we plot them in Figure24. According to coloured and scaled bar of the LOF score plotted in Figure 24(b), the maximum LOF value of driver 10 is 2.2.Figure 24
(a) GSR voltage values of driver 10 for 0 s TOR (wide scale) and (b) visual graph of GSR data LOF scores of the driver 10 for 0 s TOR (wide scale).
(a)(b)When the analysis results of the expert driver 10 are examined, the LOF score is minimally varied between 1 and 2.1 for TOR 0 s as plotted in Figure24(b), whereas the LOF score is increased to 8 when TOR is given 2 s. Considering the experiment of an expert driver, TOR 0 s data involves bounded deviation with respect to other TOR datasets.In Figure25, the driver made a comfortable transition for case 1, case 2, and case 3 for each given TOR 6 s, 4 s, 2 s, and 0 s.Figure 25
(a) LOF scores of the cases for the driver 10 and TOR 6 s, (b) LOF scores of the cases for the driver 10 and TOR 4 s, (c) LOF scores of the cases for the driver 10 and TOR 2 s, and (d) LOF scores of the cases for the driver 10 and TOR 0 s.
(a)(b)(c)(d)


### 4.4. Generalized Results for the Dataset

The GSR deviation values are examined. In Figure26, the GSR deviation values of drivers for the ideal TOR times in Table 4 are compared with box plots based on case 1, case 2, and case 3. According to Figure 26, in the distribution of the GSR deviations of the drivers, the median and minimum values are close in all 3 cases. However, the distribution of the GSR deviations is higher in case 2 where the pedestrian crosses. We can say that the drivers are very excited in case 2.Figure 26
Box plot of GSR results for the ideal TOR and different cases.8 female and 10 male drivers participated in the experimental study. The ideal TOR values of female and male drivers vary in different cases. The responses of TOR value for female and male participants are plotted in Figure27. According to Figure 27(a), lower limits and median values are the same for the 3 cases in the distributions of female drivers for TOR time detected by driving experiments, i.e., subject to metrics given in Table 3. The distributions of female drivers for TOR time are the same for case 2 and case 3. Pedestrian crossing and telephone conversation situations have the same effect on the control authority’s transition performance. In case 1, female drivers generally prefer shorter TOR times when crossing the intersection without being perturbed by another vehicle, pedestrian, or NDRTs. In Figure 27(b), the lower limits and upper limits are plotted for the 3 cases. The TOR distributions of male drivers are the same for case 1 and case 3. Male drivers prefer higher TOR times according to the distributions in Figure 27.Figure 27
(a) Box plot of the ideal TOR results of the female drivers for the different cases and (b) box plot of the ideal TOR results of the male drivers for the different cases.
(a)(b)In the study, drivers who have more than 30 years of experience and drive every day are considered experienced drivers (6 drivers are experienced among 18 drivers). Participating drivers who drive once a month or once a year are considered as an inexperienced driver (4 drivers are inexperienced among 18 drivers), and 8 drivers are semiexperienced drivers. According to Figure28(a), lower limits and median values are the same for the 3 cases while comparing TOR time distributions of the inexperienced drivers. The TOR time distribution of inexperienced drivers does not vary for case 2 and case 3. According to Figure 28(b), the median TOR time distribution for semiexperienced drivers is 2 seconds for case 1. The TOR time distribution varies significantly, and the TOR times are increased from case 1 to case 3. According to Figure 28(c), the mean of TOR time distribution for the experienced drivers is 4 seconds for case 1. The interquartile range of TOR time distribution of semiexperienced and experienced drivers involved with case 1 takes the median value of the distribution because these drivers are not affected by another vehicle, pedestrian, or NDRTs while crossing the intersection. As the experience level of the driver increases, TOR distributions become more homogeneous, and higher TOR times are preferred by the driver for the authority transition.Figure 28
(a) Box plot of the ideal TOR results of the inexperienced drivers for the different cases, (b) box plot of the ideal TOR results of the semi-experienced drivers for the different cases, and (c) box plot of the ideal TOR results of the experienced drivers for the different cases.
(a)(b)(c)


## 4.1. Takeover Request (TOR) Time Results for Case 1, Case 2, and Case 3

Before using the data, there was a need to ask the drivers a question because the vehicle is not a conventional vehicle but is a prototype electric vehicle. We asked drivers the following question (see the Appendix): “Do you think your driving today is representative of your normal driver behaviour?” According to the answers (Likert Scale: 1-5 (not at all–pretty much), 83.3% of the drivers (15 drivers) answered “pretty much” (Likert 5), 11.1% of drivers (2 drivers) answered “much” (4), and 5.5% of drivers (1 driver) answered “very little” (2).The following criteria for case 1 have been calculated for each driver and different TOR times; in Table3 (fxa), which is the current standard deviation (f) multiplied by the average value of the five acceleration measurements (a) and the standard deviation (g) of the GSR values determine whether the authority transition has been executed smoothly by the driver.Table 3
Analysis results for case 1.
DTORa (m/s2)b (km/h)c (km/h)d (m)e (s)f (A)f.a (Am/s2)gRQ16 s0.85122429.397.9832.532 s2 s4 s0.9013.767.6793.627.947.156.802 s1.0114.137.76155.414.834.883.210 s1.2210.436.43217.606.537.9738.5026 s0.9813.674.4152.227.637.484.582 s4 s4 s0.9910.954.87114.257.016.943.602 s1.0015.36.69176.564.554.550.570 s1.1119.064.722415.097.578.402.0836 s1.0212.20.0251.626.766.907.374 s2 s4 s0.3012.396.07103.23.280.984.612 s0.578.564.02168.335.693.2450 s0.898.783.67239.028.287.375.6846 s0.9611.655.4341.754.974.779.292 s2 s4 s1.0113.788.0792.242.692.729.862 s0.5411.396.02166.302.041.103.460 s0.8910.064.592410.1211.7710.487.6356 s1.109.355.4352.137.588.333.644 s4 s4 s1.1712.86.95103.526.807.953.052 s1.1417.459.02166.408.479.664.160 s1.1617.335.70238.366.897.993.6066 s0.627.393.3952.495.773.585.294 s2 s4 s0.678.325.13115.285.133.433.212 s0.6610.55.78176.275.953.935.130 s0.5915.767.872310.57.924.624.0476 s0.769.33.8752.113.692.805.294 s4 s4 s0.8273.15105.151.961.611.732 s0.959.724.17168.886.796.452.080 s0.9011.55.43229.957.426.682.5786 s1.279.894.8562.184.826.121.522 s4 s4 s0.4311.24.82114.8014.346.170.572 s0.9111.24.70175.444.584.170.570 s0.8012.35.82248.826.965.571.5296 s1.0213.525.2462.982.282.3324 s4 s4 s1.0310.484.87124.043.403.500.572 s0.9610.466.33175.855.064.853.050 s0.9811.375.67247.893.633.551106 s0.6115.785.1562.647.504.581.154 s4 s4 s0.7411.454.91125.424.793.541.152 s0.8411.546.02188.1412.9810.902.510 s0.5210.986.83258.205.052.632.08116 s0.5713.15.1741.2811.26.38102 s0 s4 s0.9317.677.54103.227.657.116.242 s0.5511.574.93177.553.872.132.50 s0.6310.741.28257.7118.7911.841.15126 s1.0111.193.8751.68.148.2225.52 s2 s4 s0.90137.56103.0813.0911.788.082 s0.8511.136.87165.168.096.880.570 s0.7616.638.5724616.8412.801.15136 s0.998.504.4162.668.768.673.780 s2 s4 s0.5010.33.89124.197.893.951.522 s0.2910.285.82176.743.050.884.040 s0.5410.155.72248.863.892.101.15146 s0.826.831.5252.6312.3310.111.572 s2 s4 s0.8911.35113.6112.4511.081.152 s0.8412.043.46175.483.583.011.150 s0.7910.133.37249.192.992.361.52156 s1.0216.88.5061.7511.6211.853.512 s2 s4 s0.8713.114.89123.2914.0912.261.522 s0.8213.416.67185.537.986.541.520 s0.7713.854.87256.578.626.6311.67166 s1.0313.545.7062.537.417.6310.110 s2 s4 s0.4811.024.70113.594.262.042.082 s1.2015.7610.95174.6410.7212.860.570 s0.5412.636.06246.834.102.210.57176 s1.4714.636.3051.655.508.094.152 s2 s4 s0.1813.566.4893.7012.742.291.522 s0.4912.393.72164.652.811.381.210 s0.6513.484.80246.4017.7611.542.08186 s0.7211.355.3251.5810.227.3615.030 s4 s4 s0.6614.372.301236.264.132.082 s0.8510.263.75186.38.226.991.520 s0.5312.762.522474.582.431.52The critical situation for case 1 is the control authority transition from the automated system to the driver while approaching the intersection. In Table3, d denotes the distance, the average velocity during the transition, which is denoted by vtransition, and gives time to the intersection, denoted by e=d/vtransition in Table 3. In the following, a denotes the average value of the five consecutive acceleration measurements collected by the TOR trigger. During the transition, the values of maximum velocity (b) and minimum velocity (c) are extracted. And the authority and the readiness of the driver subject to transition while approaching the intersection are discovered.Driver 1, the smoothest transition, occurred with a TOR about 2 s. Pulse deviation validates this: (g) value is the lowest value of 3.21 for TOR 2 s, whereas 32.53 for TOR 6 s, 6.80 for TOR 4 s, and 38.50 for TOR 0 s. The metric, denoted by (fxa), gives the lowest value in the same manner for TOR 2 s, which also validates the control authority of the driver 1 who does not excite by braking or accelerating the electric vehicle, instead of his/her smooth commanding capability at the TOR transient stage. For TOR 2 s, the distance to the critical situation, denoted by DTC, is 15 m, and the time to reach the intersection, denoted by TTC, is 5.41 s. Driver 1 performs smoothly the manoeuvring task to approach the intersection. When driver 1 takes the control, acceleration is reached at 1.01 m/s2 and has a comparably higher maximum and minimum velocity value than other TOR cases, denoted by (b) and (c) in Table 3, respectively. This also shows the confidence of the driver 1 while commanding the vehicle after the control authority transition from the automated system.Considering driver 2, the pulse deviation metric (g) and (fxa) takes a lower value for TOR 2 s with respect to the test metric values of other TOR such as 0 s, 4 s, and 6 s. Experiment with a TOR 2 s, the DTC is 17 m, and TTC is 6.56 s. This provides sufficient distance and time for the driver to command the vehicle. After the transition, the driver accelerates with 1 m/s2. The vehicle reached a maximum speed of 15.3 km/h and a minimum speed of 6.69 km/h. Comparing the speed values (d) for different TORs in Table 3, driver 2 has the authority over the vehicle while commanding traction/braking with more confidentiality after the TOR request about 2 s.Analysing the TOR experiment results of driver 3, the (g) and (fxa) metric values are lower for TOR 4 s in comparison with other TOR request times. For TOR 4 s, DTC is 10 m, and TTC is 3.2 s. After the transition, driver 3 drives with minimum average acceleration, higher maximum velocity, and minimum velocity metric values, showing his/her control over the vehicle while approaching the intersection.Driver 5, driver 6, driver 7, and driver 10 performs smooth authority transition, causing a lower (fxa) and (g) value for the experiment with TOR 4 s. Driver 4, driver 8, driver 11, driver 12, driver 15, and driver 17 perform smooth authority transition subject to TOR 2 s. Driver 16 and driver 18 perform also smooth authority transition for TOR 0 s in terms of (fxa) and (g) metric values.The lower pulse deviation of driver 9 is assured when TOR is 4 s, and lower (fxa) occurs when TOR is 6 s. Since acceleration and velocity values are also close to each other, DTC (d) and TTC (e) values in Table 3 have been compared. For TOR 4 s, the metric value (d) and (e) is higher with respect to other TOR times.Driver 13 generates lower pulse deviation for TOR 0 s whereas (fxa) is lower for the experiment with TOR 2 s. The (fxa) value of TOR 0 s is lower in comparison to its value for TOR 2 s, and DTC (d) and TTC (e) are higher; in consequence, TOR 0 s has been detected higher confidence in transition for driver 13.For driver 14, lower pulse deviation is given for TOR 4 s and TOR 2 s whereas the lower (fxa) occurs when TOR is 0 s. However, (fxa) is lower for the experiment with TOR 2 s in comparison with TOR 4 s. For this reason, it would be better to compare TOR 0 s and TOR 2 s experiments. DTC (d) and TTC (e) values assure safe driving for both experiments, and minimum velocities are equal, which are 3.46 km/h and 3.37 km/h. The TOR 2 s has a higher velocity, and the driver tends to accelerate more. Hence, TOR 2 s gives a higher confidence in control authority transition for driver 14.For theR column (results column) in Table 3, the ideal TOR value of 9 drivers is 2 s, the ideal TOR value of 6 drivers is 4 s, and TOR 0 s is the ideal for 3 drivers. Accordingly, TOR 6 s is very close to the critical situation and can cause a higher (fxa) and (g). Hence, it is not detected as the ideal transition time for all TOR cases in the experimental data. On the other hand, the TOR 0 s is a sudden transition; the driver can be caught unready and may cause a higher (fxa) and (g). However, this transition turned out to be the ideal transition for 3 drivers, see the R column result of driver 13, driver 16, and driver 18 in Table 3, who prefers 0 s for TOR.After the road experiment, the answers to the question of what is your preferred TOR time, which is asked to each driver (see the questionnaire in the Appendix), are given in theQ column in Table 3. According to the questionnaire, none of the drivers prefer TOR 6 s, two drivers prefer TOR 0 s, and other drivers prefer TOR 4 s and TOR 2 s to authorize control transition. When the TOR values of the analysis and the questionnaire are examined, the TORs of 10 drivers are the same as a result of the analysis and questionnaire.We visualize the data of 18 drivers for their ideal TOR time with a box plot in Figure6. According to this plot, DTC, denoted by (d), varies largely in its distribution with respect to other TOR quality measures. And in the test case 1, the DTC (d) value can be between 10 and 25 m subject to the driver’s preferred TOR time as given in Table 3.Figure 6
Box plot of analysis results for the test drive case 1 for 18 drivers.As in case 1, we conducted analyses for case 2 and case 3. The critical situation for case 2 is a pedestrian walking on the oncoming vehicle route, and for case 3, a NDRT disturbs the driver while approaching the intersection. Table4 presents the TOR time obtained for case 2 and case 3 as a result of a similar analysis conducted for case 1.Table 4
TOR results for case 1, case 2, case 3, and questionnaire.
DriverTOR case 1TOR case 2TOR case 3TOR questionnaire1∗2 s2 s4 s2 s22 s2 s4 s4 s34 s2 s4 s2 s42 s2 s4 s2 s54 s4 s4 s4 s64 s2 s2 s2 s74 s4 s4 s4 s82 s2 s2 s4 s94 s4 s2 s4 s10∗4 s4 s4 s4 s11∗2 s4 s0 s0 s122 s4 s4 s2 s130 s0 s2 s2 s142 s0 s0 s2 s152 s4 s2 s2 s160 s0 s0 s2 s172 s2 s0 s2 s180 s0 s2 s4 sAccording to results in Table4, TOR 6 s has not been adopted by the drivers for any case. For case 2, the ideal TOR time of 7 drivers is 2 s, the ideal TOR time of 7 drivers is 4 s, and the ideal TOR time of 4 drivers is 0 s. For case 3, the ideal TOR time of 6 drivers is 2 s, the ideal TOR time of 8 drivers is 4 s, and the ideal TOR time of 4 drivers is 0 s.According to the descriptive questionnaire, the experienced driver is driver 10 who has 42 years of driving experience and commutes on a daily frequency. The inexperienced is driver 11, with a driving frequency of one trip per year and had a driving license about one month ago. The semiexperienced driver is driver 1, having five years of driving experience with a driving frequency of one trip per week. If we compare the TOR results of these three drivers in Table4, the TOR for the expert driver (driver 10) is TOR 4 s in all drives and the questionnaire. For inexperienced driver (driver 11), the TOR value differs for different cases. The TOR for the semiexperienced driver (driver 1) is TOR 2 s for case 1, case 2, and questionnaire and TOR 4 s for case 3.The GSR data of the expert driver, inexperienced driver, and semiexperienced driver are plotted in Figures7(a)–7(c). Box plots of GSR voltage of expert driver, inexperienced driver, and semiexperienced driver are plotted in Figures 7(d)–7(f). The magnitude of the drivers’ GSR values do not provide information because skin conductivity may vary from person to person. But instead, the spread of the distributions of the GSR values and the large deviations can be considered as a measure of the level in stress.Figure 7
For different TOR times: (a) GSR values of driver 10, (b) GSR values of driver 11, (c) GSR values of driver 1, (d) box plot of GSR voltage of driver 10, (e) box plot of GSR voltage of driver 11, and (f) box plot of GSR voltage of driver 1.
(a)(b)(c)(d)(e)(f)In Figure7(d), although the voltage values of the expert driver are higher than the inexperienced driver and semiexperienced driver, GSR voltage variation is bounded by 4% where the GSR value of the inexperienced is subject to a variation about 350% in its distribution, and the GSR value of the inexperienced is subject to a variation about 30%. Figure 7(e) presents the GSR values of the inexperienced driver, which vary largely despite lowest voltage levels generated in comparison to the experienced driver and semiexperienced driver.


## 4.2. Fixed Takeover Request (TOR) Time Analysis for Case 4 and Case 5

The data of 3 drivers (expert, semiexperienced, and inexperienced) are analysed and compared. While analysing the cases, they are analysed as integrated rather than separately because the cases are sequential. Tasks in case 4 and case 5 are described as follows:Case 4 task: the test vehicle is approaching the intersection on the main road, and another vehicle is approaching the same intersection on a perpendicular road (see, for instance, the pink vehicle in Figure2 describing case 4). A screenshot from the case 4 experiment is plotted in Figure 8.Figure 8
Case 4 task; the participating vehicle causing a possible collision at the intersection is coloured by red.Case 5 task: after crossing the intersection, another participating vehicle is stopped in a lane along the test route, and the control authority is transferred to the test driver; he/she stops the test vehicle or overtakes that stopping vehicle. A screenshot of the experiment is plotted in Figure9.Figure 9
Case 5 task; the other vehicle stopped in a lane is coloured red.According to the TOR results in Table4, the drivers’ preference is 2 s or 4 s. For this reason, analyses are made for case 4 and case 5 using a constant TOR time of 2 s while TOR time is varied from 0 s to 6 s with a resolution of 2 s for case 1, case 2, and case 3 in each experiment. Overall, each driver test-drive four times with a constant TOR 2 s for case 4 and case 5; the test results are compared in Table 5.Table 5
Results for case 4 and case 5.
DriverDriving numberTOR case 1, case 2, and case 3TOR case 4 and case 5(fxa) (Am/s2) for case 4 and case 5(g) for case 4 and case 5Expert16 s2 s9.731.2324 s2 s5.641.1832 s2 s9.273.1840 s2 s7.621.47Semiexperienced16 s2 s11.499.2524 s2 s13.326.8932 s2 s8.505.4040 s2 s9.311.25Inexperienced16 s2 s10.826.9124 s2 s16.473.3332 s2 s17.069.1340 s2 s12.701.45Toward analysis of experiments for case 4 and case 5, (fxa) and (g) metric values are given and compared for the expert, semiexperienced, and inexperienced drivers in Table 5. The expert driver experiments case 4 and case 5 on his/her second test drive with a lower (fxa) and (g) value in comparison to the other three test drive results. And the same driver completed the other test drive cases, which are case 1, case 2, and case 3, and achieved a lower value in (fxa) and (g) with a TOR 4 s.The lower (fxa) and (g) values for the semiexperienced and inexperienced drivers are performed on different driving tests. According to the results in Table 5, TOR values minimizing the (fxa) and (g) metric values vary.Comparing these three drivers, in case 4 and case 5 subject to a stationary and approaching vehicle to the path of the test vehicle, a higher level of driver experience improves the performance of control authority transition in conditional automation. For instance, in the second test drive, denoted by driving number 2 in Table5, the expert driver ensures lower (fxa) and (g) values with the TOR time 4 s in case 1, case 2, and case 3 and continues performing lower values when TOR time is given 2 s in case 4 and case 5.


## 4.3. Local Outlier Factor (LOF) Results

In this section, we have applied the LOF algorithm to the GSR data of the experienced driver, semiexperienced driver, and inexperienced driver. Firstly, the raw GSR sensor data of the drives are transformed into data in terms of voltages (voltagedata=rawdata/1024×5). The mean and standard deviation of each two consecutive GSR sensor data are calculated, and feature vectors are given. LOF algorithm is applied to these feature vectors, and LOF scores are obtained. A LOF score close to 1 indicates that the driver is less excited and relaxed.The semiexperienced driver data are analysed. According to these analyses, the LOF results of the GSR data are obtained by using the maximum 9th neighbourhood in the LOF algorithm for driver 1 and TOR 6 s. The GSR measurements are plotted in Figure10(a), LOF score responses are plotted in Figure 10(b), and the mean versus standard deviation of the feature is plotted in Figure 10(c). In Figure 10(b), the maximum local outlier score value is 3.9. In Figure 10(c), the distribution of mean and standard deviation feature of LOF values is plotted. LOF score values are coloured; the maximum score value being 3.9 is coloured by red and scaled.Figure 10
(a) GSR voltage values of driver 1 for 6 s TOR, (b) GSR data LOF scores of driver 1 for 6 s TOR, and (c) visual graph of GSR data LOF scores of the driver 1 for 6 s TOR.
(a)(b)(c)For the experiment with a TOR 4 s, the LOF score responses are plotted in Figure11, and the maximum LOF score is 6.3. LOF results of the GSR data are obtained by using the maximum 12th neighbourhood in the LOF algorithm.Figure 11
(a) GSR voltage values of driver 1 for 4 s TOR, (b) GSR data LOF scores of driver 1 for 4 s TOR, and (c) visual graph of GSR data LOF scores of the driver 1 for 4 s TOR.
(a)(b)(c)For TOR 2 s, LOF results of the GSR data and maximum LOF score value of 2.3 is obtained by using the maximum 10th neighbourhood in the LOF algorithm in Figure12.Figure 12
(a) GSR voltage values of driver 1 for 2 s TOR, (b) GSR data LOF scores of driver 1 for 2 s TOR, and (c) visual graph of GSR data LOF scores of the driver 1 for 2 s TOR.
(a)(b)(c)For TOR 0 s, LOF results of the GSR data are obtained by using the maximum 11th neighbourhood in the LOF algorithm shown in Figure13. Figure 13(b) shows the maximum LOF value reaching at 3.6. Figure 13(c) plots the distribution of mean and standard deviation feature of LOF values with respect to the scaled LOF score value.Figure 13
(a) GSR voltage values of driver 1 for 0 s TOR, (b) GSR data LOF scores of driver 1 for 0 s TOR, and (c) visual graph of GSR data LOF scores of the driver 1 for 0 s TOR.
(a)(b)(c)When the analysis results of the semiexperienced driver 1 are examined, the LOF score is minimally varied between 1 and 2.2 for TOR 2 s as plotted in Figure12(b) whereas the LOF score is increased to 6.5 when TOR is given 4 s. Considering the experiment of the semiexperienced driver, TOR 2 s data involves bounded deviation with respect to other TOR datasets.In Figure14, the LOF score responses of the semiexperienced driver at the control authority transition instances are marked with a dot for cases 1, 2, and 3, and cases 4 and 5 transition responses are marked with a circle. Case 5 and case 4 LOF scores are considered continuous in a given time interval because overtaking the stopped vehicle manoeuvre is a continuation of avoiding the participating vehicle cornering into the path of the oncoming test vehicle. Figure 14 presents LOF score responses subject to different TOR time responses from 0 s to 6 s from Figure 14(a) through Figure 14(d).Figure 14
(a) LOF scores of the cases for TOR 6 s (driver 1), (b) LOF scores of the cases for TOR 4 s (driver 1), (c) LOF scores of the cases for TOR 2 s (driver 1), and (d) LOF scores of the cases for TOR 0 s (driver 1).
(a)(b)(c)(d)While evaluating the results in Figure14, it is necessary to examine each dataset within itself. Because the LOF score in the dataset is a local value, it is valid for the particular dataset. Comparing the LOF score magnitude, the driver is excited and not comfortable with all authority transitions in the data set of TOR 6 s because the LOF score is higher with respect to other TOR LOF scores. The driver performed a comfortable transition at the TOR 4 s in case 2 and case 3. The driver is not comfortable in case 1 and case 4 and case 5 for TOR 4 s. The driver made a comfortable transition at the TOR 2 s in case 1 and case 2. The driver is not comfortable in case 3 and case 4 and case 5 for TOR 2 s and is excited and uncomfortable subject to all authority transitions with the TOR 0 s. Comfortable and smooth transitions cannot be directly compared with LOF values because they are local values. Comfortable and uncomfortable means also the level of readiness of the driver to take the control of the vehicle subject to possible driving hazards and situations when the transition from the automated system is asked to be executed.We analyse the inexperienced driver data in Figure15. According to these analyses, the LOF score results of the GSR data are obtained by using the maximum 9th neighbourhood in the LOF algorithm for TOR 6 s of driver 11. In Figure 15(b), the LOF value increases considerably between feature vectors 33 and 37, reaching the maximum LOF value of 5.1.Figure 15
(a) GSR voltage values of driver 11 for 6 s TOR, (b) GSR data LOF scores of driver 11 for 6 s TOR, and (c) visual graph of GSR data LOF scores of the driver 11 for 6 s TOR.
(a)(b)(c)For TOR 4 s and 2 s, the LOF results of the GSR data are obtained by using the maximum 8th neighbourhood in the LOF algorithm for driver 11. The GSR measurements are plotted in Figures16(a)and 17(a), LOF score responses are plotted in Figures 16(b) and 17(b), and the mean versus standard deviation of feature is plotted in Figures 16(c) and 17(c), respectively. In Figure 16(b), feature vector 17 reaches at 12.4, and the driver is overexcited and stressed at this instance. In Figure 17(b), the driver’s GSR values increase to the LOF of 4.3 at the beginning of the experiment, and GSR values are also coloured; the maximum is coloured by red and scaled in Figure 17(c).Figure 16
(a) GSR voltage values of driver 11 for 4 s TOR, (b) GSR data LOF scores of driver 11 for 4 s TOR, and (c) visual graph of GSR data LOF scores of the driver 11 for 4 s TOR.
(a)(b)(c)Figure 17
(a) GSR voltage values of driver 11 for 2 s TOR, (b) GSR data LOF scores of driver 11 for 2 s TOR, and (c) visual graph of GSR data LOF scores of the driver 11 for 2 s TOR.
(a)(b)(c)For TOR 0 s, the LOF results of the GSR data are obtained by using the maximum 10th neighbourhood, and we plot them in Figure18. In Figure 18(b), the LOF value increases considerably between feature vectors 25 and 32, reaching the maximum LOF value of 5.2. In Figure 18(c), the distribution of mean and standard deviation feature of LOF values is plotted.Figure 18
(a) GSR voltage values of driver 11 for 0 s TOR, (b) GSR data LOF scores of driver 11 for 0 s TOR, and (c) visual graph of GSR data LOF scores of the driver 11 for 0 s TOR.
(a)(b)(c)When the analysis results of the inexperienced driver 11 are examined, the LOF score is minimally varied between 1 and 4.2 for TOR 2 s as plotted in Figure17(b), whereas the LOF score is increased to 12 when TOR is given 4 s.While evaluating the results in Figure19, the driver made a comfortable transition for case 2 in the dataset of TOR 6 s. The driver made a comfortable transition for case 1, case 2, and case 3 subject to the TOR 4 s dataset. The driver made a comfortable transition for case 1 and case 2 in the TOR 2 s dataset. The driver has been subject to a higher LOF score for TOR 0 s.Figure 19
(a) LOF scores of the cases for TOR 6 s (driver 11), (b) LOF scores of the cases for TOR 4 s (driver 11), (c) LOF scores of the cases the TOR 2 s (driver 11), and (d) LOF scores of the cases for TOR 0 s (driver 11).
(a)(b)(c)(d)Lastly, we analyse expert driver data in Figure20. According to this analysis, the LOF results of the GSR data are obtained by using the maximum 15th neighbourhood in the LOF algorithm for driver 10 (expert driver) for TOR 6 s.Figure 20
(a) GSR voltage values of driver 10 for 6 s TOR, (b) GSR data LOF scores of driver 10 for 6 s TOR, and (c) visual graph of GSR data LOF scores of the driver 10 for 6 s TOR.
(a)(b)(c)In Figure20, the expert driver performs small deviations in the response of LOF scores while comparing with respect to inexperienced and semiexperienced drivers. The expert driver data for the TOR 6 s is plotted in Figure 21. In Figure 21(b), the distribution of mean and standard deviation feature of LOF values is plotted. The maximum LOF value is 3.5, and LOF score values are also coloured; the maximum is coloured by red and scaled.Figure 21
(a) GSR voltage values of driver 10 for 6 s TOR (wide scale) and (b) visual graph of GSR data LOF scores of the driver 10 for 6 s TOR (wide scale).
(a)(b)For TOR 4 s, the LOF results of the GSR data are obtained by using the maximum 9th neighbourhood for driver 10, and we plot them in Figure22. The maximum LOF value is 3.7 coloured by red and scaled in Figure 22(b) presenting the distribution of mean and standard deviation feature of LOF values.Figure 22
(a) GSR voltage values of driver 10 for 4 s TOR (wide scale) and (b) visual graph of GSR data LOF scores of the driver 10 for 4 s TOR (wide scale).
(a)(b)For TOR 2 s, the LOF results of the GSR data are obtained by using the maximum 9th neighbourhood for driver 10, and we plot them in Figure23. Figure 23(a) plots the GSR voltage responses, and they are subject to small deviations. In Figure 23(b), the distribution of mean and standard deviation feature of LOF values is plotted, the maximum LOF value is reached at 8.3, LOF score values are also coloured, and the maximum is coloured by red and scaled.Figure 23
(a) GSR voltage values of driver 10 for 2 s TOR (wide scale) and (b) visual graph of GSR data LOF scores of the driver 10 for 2 s TOR (wide scale).
(a)(b)For TOR 0 s, the LOF results of the GSR data are obtained by using the maximum 12th neighbourhood for driver 10, and we plot them in Figure24. According to coloured and scaled bar of the LOF score plotted in Figure 24(b), the maximum LOF value of driver 10 is 2.2.Figure 24
(a) GSR voltage values of driver 10 for 0 s TOR (wide scale) and (b) visual graph of GSR data LOF scores of the driver 10 for 0 s TOR (wide scale).
(a)(b)When the analysis results of the expert driver 10 are examined, the LOF score is minimally varied between 1 and 2.1 for TOR 0 s as plotted in Figure24(b), whereas the LOF score is increased to 8 when TOR is given 2 s. Considering the experiment of an expert driver, TOR 0 s data involves bounded deviation with respect to other TOR datasets.In Figure25, the driver made a comfortable transition for case 1, case 2, and case 3 for each given TOR 6 s, 4 s, 2 s, and 0 s.Figure 25
(a) LOF scores of the cases for the driver 10 and TOR 6 s, (b) LOF scores of the cases for the driver 10 and TOR 4 s, (c) LOF scores of the cases for the driver 10 and TOR 2 s, and (d) LOF scores of the cases for the driver 10 and TOR 0 s.
(a)(b)(c)(d)


## 4.4. Generalized Results for the Dataset

The GSR deviation values are examined. In Figure26, the GSR deviation values of drivers for the ideal TOR times in Table 4 are compared with box plots based on case 1, case 2, and case 3. According to Figure 26, in the distribution of the GSR deviations of the drivers, the median and minimum values are close in all 3 cases. However, the distribution of the GSR deviations is higher in case 2 where the pedestrian crosses. We can say that the drivers are very excited in case 2.Figure 26
Box plot of GSR results for the ideal TOR and different cases.8 female and 10 male drivers participated in the experimental study. The ideal TOR values of female and male drivers vary in different cases. The responses of TOR value for female and male participants are plotted in Figure27. According to Figure 27(a), lower limits and median values are the same for the 3 cases in the distributions of female drivers for TOR time detected by driving experiments, i.e., subject to metrics given in Table 3. The distributions of female drivers for TOR time are the same for case 2 and case 3. Pedestrian crossing and telephone conversation situations have the same effect on the control authority’s transition performance. In case 1, female drivers generally prefer shorter TOR times when crossing the intersection without being perturbed by another vehicle, pedestrian, or NDRTs. In Figure 27(b), the lower limits and upper limits are plotted for the 3 cases. The TOR distributions of male drivers are the same for case 1 and case 3. Male drivers prefer higher TOR times according to the distributions in Figure 27.Figure 27
(a) Box plot of the ideal TOR results of the female drivers for the different cases and (b) box plot of the ideal TOR results of the male drivers for the different cases.
(a)(b)In the study, drivers who have more than 30 years of experience and drive every day are considered experienced drivers (6 drivers are experienced among 18 drivers). Participating drivers who drive once a month or once a year are considered as an inexperienced driver (4 drivers are inexperienced among 18 drivers), and 8 drivers are semiexperienced drivers. According to Figure28(a), lower limits and median values are the same for the 3 cases while comparing TOR time distributions of the inexperienced drivers. The TOR time distribution of inexperienced drivers does not vary for case 2 and case 3. According to Figure 28(b), the median TOR time distribution for semiexperienced drivers is 2 seconds for case 1. The TOR time distribution varies significantly, and the TOR times are increased from case 1 to case 3. According to Figure 28(c), the mean of TOR time distribution for the experienced drivers is 4 seconds for case 1. The interquartile range of TOR time distribution of semiexperienced and experienced drivers involved with case 1 takes the median value of the distribution because these drivers are not affected by another vehicle, pedestrian, or NDRTs while crossing the intersection. As the experience level of the driver increases, TOR distributions become more homogeneous, and higher TOR times are preferred by the driver for the authority transition.Figure 28
(a) Box plot of the ideal TOR results of the inexperienced drivers for the different cases, (b) box plot of the ideal TOR results of the semi-experienced drivers for the different cases, and (c) box plot of the ideal TOR results of the experienced drivers for the different cases.
(a)(b)(c)


## 5. Conclusions

In this study, different TOR times are tested on five different driving cases with 18 participating drivers. Three of these cases are used to detect the TOR time of drivers, while the other two scenarios are used sequentially to increase the effects of other participating vehicles such as a vehicle approaching the intersection and then is stopped on a lane along the test route causing a possible hazard situation.According to the analysis, drivers do not prefer the authority transition that is very close to the critical situation (TOR 6 s). Because in TOR 6 s, due to the approaching the critical situation, the (g) and (fxa) values are higher, and a smooth transition does not occur. It has been observed that most of the drivers make the comfortable and smooth authority transition for TOR 4 s and TOR 2 s. The experienced drivers prefer TOR 4 for authority transition. Since the TOR 0 s authority transition is also a sudden transition, the driver is unready and causes a higher (g) and (fxa). In other words, even if the drivers are far from the critical situation, they do not prefer a sudden authority transition.Takeover request time is evaluated for each driver and three driver categories such as experienced, semiexperienced, and inexperienced, and validated by a questionnaire. The TOR time is extracted and personalized for each driver, which may improve the current conditional automated driving technologies’ penetration and acceptance. As the experience of the driver increases, more stable results are obtained. The TOR time for inexperienced drivers varies for each case. This is visible in the driver’s GSR response and sensor output during the road experiments.



---

*Source: 7924444-2022-07-22.xml*